[dpdk-dev] vfio-pci blocking issue

[Lin, Xueqin]

Hi Rivet,
Using your patch to include "eal_vfio.h" header file, vfio-pci could work normally on my environment. Thanks.
http://dpdk.org/ml/archives/dev/2017-October/080746.html

Best regards,
Xueqin

-----Original Message-----
From: Gaëtan Rivet [mailto:gaetan.rivet at 6wind.com] 
Sent: Monday, October 30, 2017 4:14 PM
To: Xu, Qian Q <qian.q.xu at intel.com>
Cc: Lin, Xueqin <xueqin.lin at intel.com>; Zhang, Helin <helin.zhang at intel.com>; Thomas Monjalon <thomas at monjalon.net>; Pei, Yulong <yulong.pei at intel.com>; Peng, Yuan <yuan.peng at intel.com>; dev at dpdk.org; Glynn, Michael J <michael.j.glynn at intel.com>; Liu, Yu Y <yu.y.liu at intel.com>
Subject: Re: vfio-pci blocking issue

Hi Xueqin,

Thank you for the thorough test report.
Multiple filings for this bug, please refer to the thread

http://dpdk.org/ml/archives/dev/2017-October/080746.html

Which will focus discussion. A first fix and a new proposition are available, please tell us if it works for you or if there are any other problem with either of the solutions.

On Mon, Oct 30, 2017 at 07:25:20AM +0000, Xu, Qian Q wrote:
> We have found the blocking issue about vfio-pci in RC2, which blocked many test cases. It is a blocking issue for 17.11 release. Please help check.
> 
> Add more failed case as below: it will also cause the VF port can't be probed.
> 
> ?  Components version
> dpdk 17.11 rc2
> ?  Defect component version (release version or repo/branch/commit ID) 
> ?  Environment components version (QEMU, OVS, etc) ?  HW platform info 
> fvl10g ?  Test Steps and Test Result 1. Created VFs and bind to 
> vfio-pci ./usertools/dpdk-devbind.py --bind=igb_uio 81:00.0 81:00.1 
> echo 1 > /sys/bus/pci/devices/0000:81:00.0/max_vfs
> echo 1 > /sys/bus/pci/devices/0000:81:00.1/max_vfs
> ./usertools/dpdk-devbind.py -s
> 0000:05:02.0 'XL710/X710 Virtual Function 154c' if= drv=i40evf 
> unused=igb_uio
> 0000:05:06.0 'XL710/X710 Virtual Function 154c' if= drv=i40evf 
> unused=igb_uio ./usertools/dpdk-devbind.py --bind=vfio-pci 81:02.0 
> 81:06.0 ./usertools/dpdk-devbind.py -s
> 0000:81:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572' 
> drv=igb_uio unused=vfio-pci
> 0000:81:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572' 
> drv=igb_uio unused=vfio-pci
> 0000:81:02.0 'XL710/X710 Virtual Function 154c' drv=vfio-pci 
> unused=igb_uio
> 0000:81:06.0 'XL710/X710 Virtual Function 154c' drv=vfio-pci 
> unused=igb_uio 2. up PF with testpmd, 
> ./x86_64-native-linuxapp-gcc/app/testpmd -l 23-27 -n 4 
> --socket-mem=1024,1024 --file-prefix=pf -w 81:00.0 -w 81:00.1 - -i 
> --rxq=4 --txq=4 3. start VFs with testpmd, 
> ./x86_64-native-linuxapp-gcc/app/testpmd -l 28-32 -n 4 
> --socket-mem=1024,1024 --file-prefix=vf -w 81:02.0 -w 81:06.0 - -i 
> --rxq=4 --txq=4
> EAL: Detected 88 lcore(s)
> EAL: Some devices want iova as va but pa will be used because.. EAL: 
> vfio-noiommu mode configured
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: PCI device 0000:81:02.0 on NUMA socket 1
> EAL: probe driver: 8086:154c net_i40e_vf
> EAL: Requested device 0000:81:02.0 cannot be used
> EAL: PCI device 0000:81:06.0 on NUMA socket 1
> EAL: probe driver: 8086:154c net_i40e_vf
> EAL: Requested device 0000:81:06.0 cannot be used
> EAL: No probed ethernet devices
> Interactive-mode selected
> USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=179456, 
> size=2176, socket=0
> USER1: create a new mbuf pool <mbuf_pool_socket_1>: n=179456, 
> size=2176, socket=1 Done
> testpmd>
> it can be seen "No probed ethernet devices"
> 
> 
> From: Lin, Xueqin
> Sent: Monday, October 30, 2017 1:59 PM
> To: gaetan.rivet at 6wind.com; Zhang, Helin <helin.zhang at intel.com>
> Cc: Xu, Qian Q <qian.q.xu at intel.com>; Pei, Yulong 
> <yulong.pei at intel.com>; Peng, Yuan <yuan.peng at intel.com>
> Subject: vfio-pci
> 
> Hi Rivet,
> 
> Some cases need to bind port to vfio-pci, for example, test inline inpsec.
> 
> 1.      modprobe vfio-pci
> 
> 2.      bind port to vfio-pci
> 
> ./usertools/dpdk-devbind.py -b vfio-pci 81:00.0 81:00.1
> 
> 3.      Run ipsec
> sudo ./build/ipsec-secgw -l 20,21   -w 81:00.0 -w 81:00.1 --vdev "crypto_null" --log-level 8 --socket-mem 1024,1   -- -p 0xf -P -u 0x2 -j 9000\
>         --config="(0,0,20),(1,0,21)" -f ./enc.cfg Find error as below, 
> vfio-pci can't be used.
> 
> EAL:   probe driver: 8086:10fb net_ixgbe
> EAL: Requested device 0000:81:00.0 cannot be used
> EAL: PCI device 0000:81:00.1 on NUMA socket 1
> EAL:   probe driver: 8086:10fb net_ixgbe
> EAL: Requested device 0000:81:00.1 cannot be used
> EAL: Search driver crypto_null to probe device crypto_null
> CRYPTODEV: [crypto_null] - Creating cryptodev crypto_null
> 
> Find your patch results in the error, block lots of vfio-pci cases, which is urgent.
> DPDK-4026<https://jira01.devtools.intel.com/browse/DPDK-4026> should meet same issue.
> If any wrong to use vfio-pci, could you give a guide?
> 
> commit 279b581c897d4bf01f2b35c7327e767400c3ee80
> Author: Gaetan Rivet <gaetan.rivet at 6wind.com<mailto:gaetan.rivet at 6wind.com>>
> Date:   Thu Oct 26 12:05:52 2017 +0200
> 
>     vfio: expose functions
> 
>     The following symbols are used by vfio implementations within the PCI bus.
>     They need to be publicly available for the PCI bus to be outside the
>     EAL.
> 
> 
> Best regards,
> Xueqin
> 

--
Gaëtan Rivet
6WIND