[dpdk-users] Mellanox 100G NIC, VF SR-IOV, docker container, EAL could not detect the device

[Adrien Mazarguil]

On Tue, May 23, 2017 at 05:39:36AM +0000, Wu, Xiaoban wrote:
> Dear DPDK users,
> 
> 
> I am trying to use the VF of the Mellanox 100G NIC enabled by SR-IOV. The following is what I have done.
> 
> 
>      1. Add "intel_iommu=on iommu=pt" to kernel command line option, update-grub, and reboot
> 
>      2. Install the MLNX-OFED, reboot
> 
>      3. In default the card is in infiband mode, so I switched it to ethernet mode (in order to run DPDK application) and reboot.
> 
>      4. mst start
> 
>          mlxconfig -d /dev/mst/mt4115_pciconf0 q #query
>          mlxconfig -d /dev/mst/mt4115_pciconf0 set SRIOV_EN=1 NUM_OF_VFS=1
>          reboot
>      5. echo 1 > /sys/bus/pci/devices/0000\:83\:00.0/mlx5_num_vfs
>      6. modprobe vfio-pci
>      7. dpdk-devbind.py --status
>      8. dpdk-devbind.py -b vfio-pci 0000:83:00.1
>      9. ls -al /dev/vfio
>      10. docker run -it --privileged --device=/dev/vfio/54:/dev/vfio/54 --device=/dev/vfio/vfio:/dev/vfio/vfio -v /mnt/huge/:/dev/hugepages/ -v /var/run:/var/run  $IMAGEID bash
>      11. $dpdkapp -c 0x01 --socket-mem=128,128 --file-prefix="docker1"
> 
> However, in the EAL part, it does not list any usable devices
> EAL: Detected 12 lcore(s)
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> PMD: bnxt_rte_pmd_init() called for (null)
> Error, rte_eth_dev_configure() returns negative!
> 
> Can anybody please point out any possible solution? Looking forward to your reply. Thanks very much for your help.

Seems like the issue is not related to your mlx5 device. From the above log
it appears that you also have a bnxt device on that system which DPDK
detects and attempts to use as it is running in blacklist mode. Perhaps that
device was not configured properly.

Try to white-list the devices you want to use by explicitly providing their
PCI bus addresses through -w arguments instead.

-- 
Adrien Mazarguil
6WIND