Bug 1393 - examples/l3fwd: in eventmode RX queue setup doesn't obey numa
Summary: examples/l3fwd: in eventmode RX queue setup doesn't obey numa
Status: UNCONFIRMED
Alias: None
Product: DPDK
Classification: Unclassified
Component: examples (show other bugs)
Version: 23.11
Hardware: All All
: Normal normal
Target Milestone: ---
Assignee: dev
URL:
Depends on:
Blocks:
 
Reported: 2024-03-01 14:56 CET by Konstantin Ananyev
Modified: 2024-03-01 14:56 CET (History)
1 user (show)



Attachments

Description Konstantin Ananyev 2024-03-01 14:56:23 CET
Reproducible with latest DPDK main branch (24.03.rc1)

Starting l3fwd in event mode with all cores/NICs on numa-socket #1 will fail:

dpdk-l3fwd --lcores=49,51,53,57 -n 6 \
-a ca:00.0 -a ca:00.1 -a cb:00.0 -a cb:00.1 -s 0x8000000000000 --vdev event_sw0 -- \
-L -P -p f --rx-queue-size 1024 --tx-queue-size 1024 --mode eventdev \
--eventq-sched=ordered --rule_ipv4=test/l3fwd_lpm_v4_u1.cfg \
--rule_ipv6=test/l3fwd_lpm_v6_u1.cfg
....
...
ETHDEV: Ambiguous Rx mempools configuration
EAL: Error - exiting with code: 1
  Cause: rte_eth_rx_queue_setup: err=-22, port=0, eth_qid: 0

Reason for that:
all lcores I specified are from numa-socket #1, so there is no mempool for numa-socket #0, while in event mode l3fwd_eth_dev_port_setup() *always* tries to use mempool for numa-socket 0.
examples/l3fwd/l3fwd_event.c:
static void
l3fwd_eth_dev_port_setup(struct rte_eth_conf *port_conf)
{
   ...
   for (eth_qid = 0; eth_qid < evt_rsrc->eth_rx_queues;
                     eth_qid++) {
                        if (!evt_rsrc->per_port_pool)
                                ret = rte_eth_rx_queue_setup(port_id, eth_qid,
                                        nb_rxd, 0, &rxconf,
                                        evt_rsrc->pkt_pool[0][0]);
                        else
                                ret = rte_eth_rx_queue_setup(port_id, eth_qid,
                                        nb_rxd, 0, &rxconf,
                                        evt_rsrc->pkt_pool[port_id][0]);

For proper numa support, I expect it to be something like:

int socket_id = l3fwd_event_deduce_rx_service_socket_id(...);
ret = rte_eth_rx_queue_setup(..., evt_rsrc->pkt_pool[port_id][socket_id]);

To overcome the problem, use '--no-numa' l3fwd cmd-line flag.

Note You need to log in before you can comment on or make changes to this bug.