[dpdk-dev] VFIO no-iommu

[Alex Williamson]

On Tue, 2015-12-15 at 13:43 +0000, O'Driscoll, Tim wrote:
> > -----Original Message-----
> > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Alex
> > Williamson
> > Sent: Friday, December 11, 2015 11:03 PM
> > To: Vincent JARDIN; dev at dpdk.org
> > Subject: Re: [dpdk-dev] VFIO no-iommu
> > 
> > On Fri, 2015-12-11 at 23:12 +0100, Vincent JARDIN wrote:
> > > Thanks Thomas for putting back this topic.
> > > 
> > > Alex,
> > > 
> > > I'd like to hear more about the impacts of "unsupported":
> > > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/c
> > > ommi
> > > t/?id=033291eccbdb1b70ffc02641edae19ac825dc75d
> > >    Use of this mode, specifically binding a device without a
> > > native
> > >    IOMMU group to a VFIO bus driver will taint the kernel and
> > > should
> > >    therefore not be considered supported.
> > > 
> > > It means that we get ride of uio; so it is a nice code cleanup:
> > > but
> > > why
> > > would VFIO/NO IOMMU be better if the bottomline is "unsupported"?
> > 
> > How supportable do you think the uio method is?  Fundamentally we
> > have
> > a userspace driver doing unrestricted DMA; it can access and modify
> > any
> > memory in the system.  This is the reason uio won't provide a
> > mechanism
> > to enable MSI and if you ask the uio maintainer, they don't support
> > DMA
> > at all, it's only intended as a programmed IO interface to the
> > device.
> >  Unless we can sandbox a user owned device within an IOMMU
> > protected
> > container, it's not supportable.  The VFIO no-iommu mode can simply
> > provide you that unsupported mode more easily since it leverages
> > code
> > from the supported mode, which is IOMMU protected.  Thanks,
> 
> Thanks for clarifying.
> 
> This does seem like it would be useful for DPDK. We're doing some
> further investigation to see if it works out of the box with DPDK or
> if we need to make any changes to support it.

The iommu model is different, there's no type1 interface available when
using this mode since we have no ability to provide translation.  The
no-iommu iommu model really does nothing, which is a possible issue for
userspace.  Is it sufficient?  We stopped short of creating a page
pinning interface through the no-iommu model because it requires code
and adding piles of new code for an interface we claim is unsupported
doesn't make a lot of sense.  The device interface should be identical
to existing vfio support.

> Thomas highlighted that your original commit for this had been
> reverted. What specifically would you need from us in order to re-
> submit the VFIO No-IOMMU support?

No API changes should ever go into the kernel without being validated
by a user.  Without that we're risking that the kernel interface is
broken and we're stuck supporting it.  In this case I tried to make
sure we had a working user before it went it, gambled that it was close
enough to put in anyway, then paid the price when development went
silent on the user side.  To get it back in, I'm going to need a
working use first.  You can re-apply 033291eccbdb or re-
revert ae5515d66362 for development of that.  I need to see that it
works and that there's some consensus from the dpdk community that it's
a worthwhile path forward for cases without an iommu.  There's no point
in merging it if it only becomes a userspace proof of concept.  Thanks,

Alex