Bug 887 - mlx5: cannot probe ConnectX-6 Dx if socket-limit is too small on NUMA socket 0
Summary: mlx5: cannot probe ConnectX-6 Dx if socket-limit is too small on NUMA socket 0
Status: UNCONFIRMED
Alias: None
Product: DPDK
Classification: Unclassified
Component: ethdev (show other bugs)
Version: 21.08
Hardware: x86 Linux
: Normal normal
Target Milestone: ---
Assignee: dev
URL:
Depends on:
Blocks:
 
Reported: 2021-11-17 20:16 CET by Junxiao Shi
Modified: 2022-08-28 22:42 CEST (History)
3 users (show)


Attachments
Add an attachment (proposed patch, testcase, etc.)

Description Junxiao Shi 2021-11-17 20:16:25 CET 
Our client has a ConnectX-6 Dx Ethernet adapter, installed on a PCI slot belonging to NUMA socket 1.
When DPDK is initialized with flags "--socket-limit 1,2048 -a c1:00.0", PCI probe fails with this message:

EAL: No available 2048 kB hugepages reported
EAL: No free 2048 kB hugepages reported on node 0
EAL: No free 2048 kB hugepages reported on node 1
EAL: No available 2048 kB hugepages reported
mlx5_net: probe of PCI device 0000:c1:00.0 aborted after encountering an error: Cannot allocate memory
mlx5_common: Failed to load driver mlx5_eth
EAL: Requested device 0000:c1:00.0 cannot be used
EAL: Bus (pci) probe failed.

I suspect somewhere in the driver is trying to allocate certain objects on NUMA socket 0, but I don't know which.
It should be changed to allow allocation on any NUMA socket.
Comment 1 Lior 2022-02-08 16:54:41 CET 
Can you try to reproduce with the following patch?
https://git.dpdk.org/dpdk/commit/?id=147f6fb42b
Comment 2 Asaf Penso 2022-08-28 22:42:39 CEST 
Hello, have you tried the patch Lior mentioned? 
Do you see the issue also with upstream version?
Note You need to log in before you can comment on or make changes to this bug.