[dpdk-users] DPDK mlx4 PMD on virtual function

[Adrien Mazarguil]

Hi Trevor,

On Wed, Jun 07, 2017 at 04:12:27PM +0000, Trevor  Highland wrote:
> Hi all,
> 
> I’m trying to attach the dpdk mlx4 pmd to a virtual function in a virtualized environment. I installed the latest version of OFED (4.0-2.0.0.1), and I compiled/installed the latest code in the github Mellanox/dpdk-dev-libibverbs repository. I’m trying to start by testing with testpmd. During initialization, I encounter an IBV permission error. Ethtool shows firmware below the recommended version, however I thought the OFED installation automatically upgraded the firmware. Any help diagnosing the error would be appreciated.
> 
> Testpmd output:
> ./build/app/test-pmd/testpmd -c 0xFF -n 4 -- --nb-cores=4 --rxq=2 --txq=0 --forward-mode=rxonly --port-topology=chained -a
> EAL: Detected 8 lcore(s)
> EAL: Probing VFIO support...
> EAL: PCI device 0002:00:02.0 on NUMA socket 0
> EAL:   probe driver: 15b3:1004 net_mlx4
> PMD: net_mlx4: PCI information matches, using device "mlx4_0" (VF: true)
> PMD: net_mlx4: 1 port(s) detected
> PMD: net_mlx4: port 1 MAC address is 00:0d:3a:f9:40:21
> Set rxonly packet forwarding mode
> Auto-start selected
> Warning: nb_rxq=2 enables RSS configuration, but nb_txq=0 will prevent to fully test it.
> USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=203456, size=2176, socket=0
> Configuring Port 0 (socket 0)
> PMD: net_mlx4: 0xcd0000: RX queues number update: 0 -> 2
> PMD: net_mlx4: 0xcd0000: QP state to IBV_QPS_INIT failed: Operation not permitted
> Fail to configure port 0
> EAL: Error - exiting with code: 1
>   Cause: Start ports failed
> 
> Ethtool output:
> ethtool -i enP2p0s2
> driver: mlx4_en
> version: 4.0-2.0.0 (28 Mar 2017)
> firmware-version: 2.38.5000

Probably due to the firmware version indeed. Here's mine:

driver: mlx4_en
version: 4.0-2.0.0 (28 Mar 2017)
firmware-version: 2.40.7000

You can attempt to run mlxup [1][2] if MLNX_OFED is unable to update it.

However if your adapter is not part of the above list, it's probably made by
a different vendor, in that case neither MLNX_OFED nor mlxup will be able to
update it since you normally need a firmware update from this specific
vendor. Perhaps this [3] can help? (may void your warranty though)

[1] http://www.mellanox.com/page/mlxup_firmware_tool
[2] http://www.mellanox.com/page/firmware_table_ConnectX3EN
[3] https://forums.servethehome.com/index.php?threads/flash-oem-ibm-mellanox-connectx-3-en-with-stock-firmware.8689/

-- 
Adrien Mazarguil
6WIND