[dpdk-dev] [PATCH] eal: map io resources for non x86 architectures

[Santosh Shukla]

On Tue, Dec 29, 2015 at 3:26 PM, Burakov, Anatoly
<anatoly.burakov at intel.com> wrote:
> Hi Santosh,
>
>> On Fri, Dec 18, 2015 at 6:25 PM, Santosh Shukla <sshukla at mvista.com>
>> wrote:
>> > On Fri, Dec 18, 2015 at 1:51 PM, Yuanhan Liu
>> > <yuanhan.liu at linux.intel.com> wrote:
>> >> On Fri, Dec 18, 2015 at 01:24:41PM +0530, Santosh Shukla wrote:
>> >>> >> I guess we have done enough evaluation / investigation that
>> >>> >> suggest - so to map iopci region to userspace in arch
>> >>> >> agnostic-way -
>> >>> >>
>> >>> >> # either we need to modify kernel
>> >>> >>                - Make sure all the non-x86 arch to support
>> >>> >> mapping for iopci region (i.e. pci_mmap_page_range). I don;t
>> >>> >> think its a correct approach though.
>> >>> >>             or
>> >>> >>                - include /dev/ioport char-mem device file who
>> >>> >> could do more than byte operation, Note that this implementation
>> >>> >> does not exist in kernel.  I could send an RFC to lkml.
>> >>> >
>> >>> > Maybe you could propose the two to lkml, to get some feedbacks
>> >>> > from those kernel/ARM gurus? Please cc me if you do so.
>> >>> >
>> >>>
>> >>> The latter one I already shared old lkml thread, Pl. revisit my v1
>> >>> 0/6 patch [1] and in that refer [2].
>> >>
>> >> Oops, sorry, a bit busy, that I didn't look at it carefully. My bad,
>> >> anyway.
>> >>
>> >>> Josh has already proposed to lkml but for some reason thread didn't
>> >>> went far. I can restart that discussion giving dpdk use-case as an
>> >>> example/ requirement.
>> >>
>> >> I had a quick go through of the discussion. Both hpa and Arnd seem to
>> >> be fine with the ioctl interface on /dev/port. Have you tried that?
>> >> And if you want to restart it, ioctl might be a better option than
>> >> /dev/ioport, judging from the discussion.
>> >>
>> >
>> > I tried legacy patch and re-writing with ioctl-way; doing changes in
>> > dpdk port as well in kernel, had to test on quite other hw not only
>> > arm64 though! so it will take time for me, I am travelling tomorrow so
>> > bit delayed, We'll post patch to lkml and share dpdk-virtio feedback
>> > perhaps by Monday.
>> >
>>
>> I posted a query about /dev/ioports approach in lkml thread [1], and Arnd
>> suggested to use vfio framework but it looks like vfio too does not map
>> ioresource_io region. Same communicated to Arnd and waiting for his reply.
>>
>> In mean time I like to ask general question;
>> - Has anyone tried vfio/non-iommu approach for virtio pmd driver? If not
>> then Is there any plan? Someone planning to take up.
>> [1] https://lkml.org/lkml/2015/12/23/145
>
> I have submitted a patch to support no-iommu mode, but I'm not aware of anyone trying VFIO-noiommu at all. That's probably expected since it's Christmas/New Year in a lot of places, and everyone is on a break.
>
> That said, I'm not sure I completely understand what is it that you're asking about. The code you're referring to (in vfio_pci.c, line 854 as of kernel 4.3) is checking if a PCI BAR is available for IO (hence checking if the IORESOURCE_MEM


Thanks for reply! You comment might help to move this discuss to next level.

Look at kernel/resource.c, it exports two symbol ioport_resource and
iomem_resource and sets appropriate flag type i.e.. IORESOURCE_IO and
IORESOURCE_MEM. In virtio-net case; it creates both pci region i.e..
_io bar and _mem bar. dpdk virtio pmd driver (<= 0.95 virtio spec)
uses pci _io bar region for device initialization as virtio headers
are locate at pci _io bar region. Since it uses pci _iobar region so
likely it update pci_resource.[index].flag = IORESOURCE_IO.  and vfio
mmap function wont handle ioresource_io (i guess). And that is why I
asked same to lkml thread.


bit is set). There isn't any "ioresource_mem" region as far as VFIO is
concerned, there are only BARs, ROM, VGA and PCI
config regions (linux/vfio.h, line 314 as of kernel 4.3). So if you're
missing some PCI regions for VFIO to map, they would first need to be
added to VFIO kernel implementation before they can be used by DPDK.
That is, unless I'm misunderstanding something :)
>

Agree. As mentioned above, I guess ioresource_io pci bar
implementation missing in vfio kernel? but before adding code support
in kernel I would like to hear from experts example, You, Alex!
(looping alex)

> Thanks,
> Anatoly