[dpdk-users] Unable to bind intel NIC 82599ES to vfio-pci, uio_pci_generic

[Shyam Shrivastav]

Thanks Ferruh, I figured my mistake immediately afterwards. I am able to
work with uio_pci_generic and proceeding with my project pilot for now,
have not spent much time on debugging vfio-pci issue.

Here is one more try to bind vfio-pci, includes dmesg


[root at unassigned dpdk-16.11]# dpdk-devbind --status

Network devices using DPDK-compatible driver
============================================
0000:01:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection'
drv=uio_pci_generic unused=ixgbe,igb_uio,vfio-pci

Network devices using kernel driver
===================================
0000:02:00.0 'I210 Gigabit Network Connection' if=eno1 drv=igb
unused=igb_uio,vfio-pci,uio_pci_generic
0000:03:00.0 'I210 Gigabit Network Connection' if=eno2 drv=igb
unused=igb_uio,vfio-pci,uio_pci_generic

Other network devices
=====================
0000:01:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection'
unused=ixgbe,igb_uio,vfio-pci,uio_pci_generic

Crypto devices using DPDK-compatible driver
===========================================
<none>

Crypto devices using kernel driver
==================================
<none>

Other crypto devices
====================
<none>

[root at unassigned dpdk-16.11]# dpdk-devbind -b vfio-pci 01:00.1
Error: bind failed for 0000:01:00.1 - Cannot bind to driver vfio-pci

[root at unassigned dpdk-16.11]# dmesg | tail -n 10
[ 1989.451194] VFIO - User Level meta-driver version: 0.3
[16700.417269] VFIO - User Level meta-driver version: 0.3
[16708.620870] rte_kni: loading out-of-tree module taints kernel.
[16708.621016] rte_kni: module verification failed: signature and/or
required key missing - tainting kernel
[16762.831711] igb_uio: Use MSIX interrupt by default
[378754.897617] Intel(R) 10GbE PCI Express Linux Network Driver - version
4.5.4
[378754.897623] Copyright(c) 1999 - 2016 Intel Corporation.
[491762.064211] Intel(R) 10GbE PCI Express Linux Network Driver - version
4.5.4
[491762.064213] Copyright(c) 1999 - 2016 Intel Corporation.
[515594.211309] VFIO - User Level meta-driver version: 0.3

[root at unassigned dpdk-16.11]# ulimit -l
unlimited


dmesg for initialisation check

[root at unassigned dpdk-16.11]# dmesg | grep DMAR
[    0.000000] ACPI: DMAR 000000007d705e98 00070 (v01 INTEL      SKL
00000001 INTL 00000001)
[    0.000000] DMAR: IOMMU enabled
[    0.027917] DMAR: Host address width 39
[    0.027918] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[    0.027923] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap
d2008c40660462 ecap f050da
[    0.027923] DMAR: RMRR base: 0x0000007d190000 end: 0x0000007d1affff
[    0.027925] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed90000 IOMMU 0
[    0.027926] DMAR-IR: HPET id 0 under DRHD base 0xfed90000
[    0.027927] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out
bit.
[    0.027927] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the
BIOS setting.
[    0.028288] DMAR-IR: Enabled IRQ remapping in xapic mode
[    0.511886] DMAR: No ATSR found
[    0.511942] DMAR: dmar0: Using Queued invalidation
[    0.511954] DMAR: Hardware identity mapping for device 0000:00:00.0
[    0.511955] DMAR: Hardware identity mapping for device 0000:00:01.0
[    0.511956] DMAR: Hardware identity mapping for device 0000:00:14.0
[    0.511957] DMAR: Hardware identity mapping for device 0000:00:14.2
[    0.511958] DMAR: Hardware identity mapping for device 0000:00:16.0
[    0.511959] DMAR: Hardware identity mapping for device 0000:00:17.0
[    0.511959] DMAR: Hardware identity mapping for device 0000:00:1d.0
[    0.511960] DMAR: Hardware identity mapping for device 0000:00:1d.1
[    0.511961] DMAR: Hardware identity mapping for device 0000:00:1d.2
[    0.511962] DMAR: Hardware identity mapping for device 0000:00:1f.0
[    0.511963] DMAR: Hardware identity mapping for device 0000:00:1f.2
[    0.511964] DMAR: Hardware identity mapping for device 0000:00:1f.4
[    0.511965] DMAR: Hardware identity mapping for device 0000:01:00.0
[    0.511966] DMAR: Hardware identity mapping for device 0000:01:00.1
[    0.511969] DMAR: Hardware identity mapping for device 0000:02:00.0
[    0.511971] DMAR: Hardware identity mapping for device 0000:03:00.0
[    0.511972] DMAR: Setting RMRR:
[    0.511973] DMAR: Ignoring identity map for HW passthrough device
0000:00:14.0 [0x7d190000 - 0x7d1affff]
[    0.511974] DMAR: Prepare 0-16MiB unity mapping for LPC
[    0.511975] DMAR: Ignoring identity map for HW passthrough device
0000:00:1f.0 [0x0 - 0xffffff]
[    0.511976] DMAR: Intel(R) Virtualization Technology for Directed I/O
[root at unassigned dpdk-16.11]#





On Mon, Feb 13, 2017 at 10:31 PM, Ferruh Yigit <ferruh.yigit at intel.com>
wrote:

> On 2/8/2017 7:19 AM, Shyam Shrivastav wrote:
> > Hi All
> >
> > I am not able to bind 82599 to either uio_pci_generic or vfio-pci
> > successfully. Any help greatly appreciated, I am completely stuck at this
> > initial step.
> >
> > *1) uio_pci_generic* : tools/dpdk-devbind.py script reports success but
> it
> > is not detected by EAL on initialisation, still ixgbe driver is detected.
> > here is relevant console output
>
> It is detected, but according following log ports are not enabled by
> proper application (not eal) command line argument:
> "   Cause: All available ports are disabled. Please set portmask."
>
> ./examples/l2fwd/build/l2fwd [EAL options] -- -p PORTMASK [-q NQ]
>   -p PORTMASK: hexadecimal bitmask of ports to configure
>   -q NQ: number of queue (=ports) per lcore (default is 1)
>   -T PERIOD: statistics will be refreshed each PERIOD seconds (0 to
> disable, 10 default, 86400 maximum)
>   --[no-]mac-updating: Enable or disable MAC addresses updating (enabled
> by default)
>       When enabled:
>        - The source MAC address is replaced by the TX port MAC address
>        - The destination MAC address is replaced by
> 02:00:00:00:00:TX_PORT_ID
>
> <...>
>
> >
> >
> >
> > *2) vfio-pci :* Configured vfio permissions using setup.sh, bind script
> > reports error in this case as under
>
> Not sure about this one, below looks correct. Perhaps dmesg can tell more.
>
> <...>
>
> >
> > Thanks
> > Shyam
> >
>
>