[dpdk-dev] Having troubles binding an SR-IOV VF to uio_pci_generic on Amazon instance

[Michael S. Tsirkin]

On Fri, Oct 02, 2015 at 03:07:24PM +0100, Bruce Richardson wrote:
> On Fri, Oct 02, 2015 at 05:00:14PM +0300, Michael S. Tsirkin wrote:
> > On Thu, Oct 01, 2015 at 02:02:24PM -0700, Alexander Duyck wrote:
> > > validation and translation would add 10s if not 100s of nanoseconds to the
> > > time needed to process each packet.  In addition we are talking about doing
> > > this in kernel space which means we wouldn't really be able to take
> > > advantage of things like SSE or AVX instructions.
> > 
> > Yes. But the nice thing is that it's rearming so it can happen on
> > a separate core, in parallel with packet processing.
> > It does not need to add to latency.
> > 
> > You will burn up more CPU, but again, all this for boxes/hypervisors
> > without an IOMMU.
> > 
> > I'm sure people can come up with even better approaches, once enough
> > people get it that kernel absolutely needs to be protected from
> > userspace.
> > 
> > Long term, the right thing to do is to focus on IOMMU support. This
> > gives you hardware-based memory protection without need to burn up CPU
> > cycles.
> > 
> > -- 
> > MST
> 
> Running it on another will have it's own problems. The main one that springs to
> mind for me is the performance impact of having all those cache lines shared
> between the two cores.
> 
> /Bruce

The cache line is currently invalidated by the device write
packet processing core -> device -> packet processing core
We are adding another stage
packet processing core -> rearming core -> device -> packet processing core
but the amount of sharing per core isn't increased.

This is something that can be tried immediately without kernel changes.
Who knows, maybe you will actually be able to push more pps this way.


Further, rearming is not doing a lot besides moving bits around in
memory, and it's in kernel so using very limited resources - maybe we
can efficiently use an HT logical core for this task.
This remains to be seen.

-- 
MST