[dpdk-dev] Issue with igb_uio in Fedora 24

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Ferruh Yigit
> Sent: Friday, July 01, 2016 11:53 AM
> To: Mcnamara, John; dev at dpdk.org
> Cc: De Lara Guarch, Pablo; Stephen Hemminger
> Subject: Re: [dpdk-dev] Issue with igb_uio in Fedora 24
> 
> On 7/1/2016 11:21 AM, Mcnamara, John wrote:
> > Hi,
> >
> > We have seen an issue when using the igb_uio module in Fedora
> > 24. However, it relates to kernel 4.5+ so it could occur in other
> > distros/oses.
> >
> > The issue occurs after binding a nic to igb_uio:
> >
> >     # ./tools/dpdk_nic_bind.py -b igb_uio 0000:03:00.0
> >     # ./x86_64-native-linuxapp-gcc/app/test
> >
> >     EAL: Detected 8 lcore(s)
> >     EAL: Probing VFIO support...
> >     PMD: bnxt_rte_pmd_init() called for (null)
> >     EAL: PCI device 0000:03:00.0 on NUMA socket -1
> >     EAL:   probe driver: 8086:1533 rte_igb_pmd
> >     EAL: pci_map_resource():
> >          cannot mmap(39, 0x7f1c51800000, 0x100000, 0x0):
> >          Invalid argument (0xffffffffffffffff)
> >     EAL: Error - exiting with code: 1
> >          Cause: Requested device 0000:03:00.0 cannot be used
> >
> > The issue is exposed when the kernel is compiled with option
> > CONFIG_IO_STRICT_DEVMEM turned on.
> >
> > This option is new from kernel 4.5 and is on by default in Fedora 24:
> >
> >     config IO_STRICT_DEVMEM
> >         bool "Filter I/O access to /dev/mem"
> >         depends on STRICT_DEVMEM
> >         ---help---
> >
> >           If this option is disabled, you allow userspace (root) access
> >           to all io-memory regardless of whether a driver is actively
> >           using that range.  Accidental access to this is obviously
> >           disastrous, but specific access can be used by people
> >           debugging kernel drivers.
> >
> >           If this option is switched on, the /dev/mem file only allows
> >           userspace access to *idle* io-memory ranges (see /proc/iomem)
> >           This may break traditional users of /dev/mem (dosemu, legacy
> >           X, etc...)  if the driver using a given range cannot be
> >           disabled.
> >
> >           If in doubt, say Y.
> >
> > This issue doesn't occur with uio_pci_generic.
> >
> > John
> >
> 
> 
> Thanks for Pablo figuring out the details, and related config option,
> otherwise I was digging same issue without success.
> 
> Disabling following code in igb_uio makes it work again:
> "pci_request_regions(dev, "igb_uio");"
> 
> This call adds device bars as active resources, and with above config
> option mmap fails for those addresses.
> 
> uio_pci_generic already doesn't have this call, and I can send a patch
> to remove this from igb_uio.
> 
> BUT, I don't know why that call added at first place and concerned about
> any corner case that requires this call.
> Anybody out has/knows a reason to keep this call?
> 
> Thanks,
> ferruh
> 
> 

Another workaround to make this work is setting "iomem=relaxed" in the kernel parameters,
to avoid recompiling the kernel with IO_STRICT_DEVMEM disabled.

Thanks,
Pablo