[dpdk-dev] vfio detection

[Richardson, Bruce]

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Richardson, Bruce
> Sent: Tuesday, June 17, 2014 9:29 AM
> To: Burakov, Anatoly; dev at dpdk.org
> Subject: Re: [dpdk-dev] vfio detection
> 
> > -----Original Message-----
> > From: Burakov, Anatoly
> > Sent: Tuesday, June 17, 2014 1:40 AM
> > To: Richardson, Bruce; dev at dpdk.org
> > Subject: RE: vfio detection
> >
> > Hi Bruce,
> >
> > > I have a number of NIC ports which were working correctly yesterday and
> are
> > > bound correctly to the igb_uio driver - and I want to keep using them
> > > through the igb_uio driver for now, not vfio. However, whenever I run a
> > > dpdk application today, I find that the vfio kernel module is getting loaded
> > > each time - even after I manually remove it, and verify that it has been
> > > removed by checking lsmod. Is this expected? If so, why are we loading the
> > > vfio driver when I just want to continue using igb_uio which works fine?
> >
> > Can you elaborate a bit on what do you mean by "loading vfio driver"? Do you
> > mean the vfio-pci kernel gets loaded by DPDK? I certainly didn't put in any code
> > that would automatically load that driver, and certainly not binding devices to
> it.
> 
> The kernel module called just "vfio" is constantly getting reloaded, and there is
> always a "/dev/vfio" directory, which triggers the vfio code handling every time I
> run dpdk.
> 
> >
> > > Secondly, then, when testpmd or any other app loads, it automatically tries
> > > to map the NIC using vfio and then aborts on the very first NIC port when it
> > > fails to do so.
> >
> > This shouldn't happen, unless you have a device bound to VFIO and have
> another
> > device in the same IOMMU group that is bound to something else. Can you
> > provide a log of what you are seeing?
> 
> Log of testpmd run attached.

Log got stripped from mail, including below instead.

Script started on Tue 17 Jun 2014 17:23:54 IST

bruce at silpixa00372841:dpdk.org$ ./tools/dpdk_nic_bind.py --status

Network devices using DPDK-compatible driver
============================================
0000:84:00.0 'Ethernet Server Adapter X520-4' drv=igb_uio unused=ixgbe
0000:87:00.0 'Ethernet Server Adapter X520-4' drv=igb_uio unused=ixgbe
0000:8b:00.0 'Ethernet Server Adapter X520-4' drv=igb_uio unused=ixgbe
0000:8e:00.0 'Ethernet Server Adapter X520-4' drv=igb_uio unused=ixgbe

Network devices using kernel driver
===================================
0000:04:00.0 'I350 Gigabit Network Connection' if=em0 drv=igb unused=igb_uio *Active*
0000:04:00.1 'I350 Gigabit Network Connection' if=ens2f1 drv=igb unused=igb_uio 
0000:04:00.2 'I350 Gigabit Network Connection' if=ens2f2 drv=igb unused=igb_uio 
0000:04:00.3 'I350 Gigabit Network Connection' if=ens2f3 drv=igb unused=igb_uio 

Other network devices
=====================
0000:0a:00.1 'DH8900CC Null Device' unused=igb_uio
0000:0b:00.1 'DH8900CC Null Device' unused=igb_uio
0000:0c:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' unused=ixgbe,igb_uio
0000:0c:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' unused=ixgbe,igb_uio
0000:84:00.1 'Ethernet Server Adapter X520-4' unused=ixgbe,igb_uio
0000:87:00.1 'Ethernet Server Adapter X520-4' unused=ixgbe,igb_uio
0000:8b:00.1 'Ethernet Server Adapter X520-4' unused=ixgbe,igb_uio
0000:8e:00.1 'Ethernet Server Adapter X520-4' unused=ixgbe,igb_uio


bruce at silpixa00372841:dpdk.org$ sudo ./x86_64-native-linuxapp-gcc/app/testpmd -c F00 -n 4 -- -i
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 6 on socket 0
EAL: Detected lcore 7 as core 7 on socket 0
EAL: Detected lcore 8 as core 0 on socket 1
EAL: Detected lcore 9 as core 1 on socket 1
EAL: Detected lcore 10 as core 2 on socket 1
EAL: Detected lcore 11 as core 3 on socket 1
EAL: Detected lcore 12 as core 4 on socket 1
EAL: Detected lcore 13 as core 5 on socket 1
EAL: Detected lcore 14 as core 6 on socket 1
EAL: Detected lcore 15 as core 7 on socket 1
EAL: Detected lcore 16 as core 0 on socket 0
EAL: Detected lcore 17 as core 1 on socket 0
EAL: Detected lcore 18 as core 2 on socket 0
EAL: Detected lcore 19 as core 3 on socket 0
EAL: Detected lcore 20 as core 4 on socket 0
EAL: Detected lcore 21 as core 5 on socket 0
EAL: Detected lcore 22 as core 6 on socket 0
EAL: Detected lcore 23 as core 7 on socket 0
EAL: Detected lcore 24 as core 0 on socket 1
EAL: Detected lcore 25 as core 1 on socket 1
EAL: Detected lcore 26 as core 2 on socket 1
EAL: Detected lcore 27 as core 3 on socket 1
EAL: Detected lcore 28 as core 4 on socket 1
EAL: Detected lcore 29 as core 5 on socket 1
EAL: Detected lcore 30 as core 6 on socket 1
EAL: Detected lcore 31 as core 7 on socket 1
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 32 lcore(s)
EAL: 1024 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size
EAL: Setting up memory...
EAL: Ask a virtual area of 0x80000000 bytes
EAL: Virtual area found at 0x7fe5c0000000 (size = 0x80000000)
EAL: Ask a virtual area of 0x80000000 bytes
EAL: Virtual area found at 0x7fe500000000 (size = 0x80000000)
EAL: Requesting 2 pages of size 1024MB from socket 0
EAL: Requesting 2 pages of size 1024MB from socket 1
EAL: TSC frequency is ~2693503 KHz
EAL: Master core 8 is ready (tid=e877c880)
EAL: Core 11 is ready (tid=e6741700)
EAL: Core 10 is ready (tid=e6f42700)
EAL: Core 9 is ready (tid=e7743700)
EAL: PCI device 0000:04:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   unknown IOMMU driver!
EAL:   0000:04:00.0 cannot open VFIO container!
EAL: Error - exiting with code: 1
  Cause: Requested device 0000:04:00.0 cannot be used


bruce at silpixa00372841:dpdk.org$ exit
exit

Script done on Tue 17 Jun 2014 17:25:29 IST