[dpdk-dev] [PATCH v5 00/29] Support VFD and DPDK PF + kernel VF on i40e

John Fastabend john.fastabend at gmail.com
Sat Jan 14 00:04:03 CET 2017

On 17-01-11 07:47 PM, Chen, Jing D wrote:
> Hi, Vincent,
>> -----Original Message-----
>> From: Vincent JARDIN [mailto:vincent.jardin at 6wind.com]
>> Sent: Tuesday, January 10, 2017 9:30 PM
>> To: Scott Daniels <daniels at research.att.com>; dev at dpdk.org
>> Cc: kaustubh at research.att.com; az5157 at att.com; Chen, Jing D
>> <jing.d.chen at intel.com>
>> Subject: Re: [dpdk-dev] [PATCH v5 00/29] Support VFD and DPDK PF + kernel VF on
>> i40e
>> Hi Scott,
>> Le 04/01/2017 à 22:09, Scott Daniels a écrit :
>>>  With holidays we are a bit late with our thoughts, but would like to
>>>  toss them into the mix.
>> Same, I hope I am not missing emails. I do appreciate your arguments, it
>> provides lot of light. See below,
>>>  The original NAK is understandable, however having the ability to
>>>  configure the PF via DPDK is advantageous for several reasons:
>>>  1) While some functions may be duplicated and/or available from the kernel
>>>  driver, it is often not possible to introduce new kernel drivers into
>>>  production without a large amount of additional testing of the entire
>>>  platform which can cause a significant delay when introducing a DPDK based
>>>  product.  If the PF control is a part of the DPDK environment, then only
>>>  the application needs to pass the operational testing before deployment; a
>>>  much more simple task.
>> So we agree: you confirm that your foresee the benefits of using DPDK to
>> *bypass the role of the Kernel being the PF* of reference for the
>> hypervisor.
>>>  2) If the driver changes are upstreamed into the kernel proper, the
>>>  difficulty of operational readiness testing increases as a new kernel is
>>>  introduced, and further undermines the ability to quickly and easily
>>>  release a DPDK based application into production.  While the application
>>>  may eventually fall back on driver and/or kernel support, this could be
>>>  years away.
>> I do agree with the benefits of the agilities and the upsides it brings.
>> But they are other options to get the same agility without creating a
>> fragmentation of PFs.
>> For example, you do not have to update the whole kernel, you can just
>> update the PF kernel module that can be upgraded with the latest needed
>> features.
>>>  3) As DPDK is being used to configure the NIC, it just seems to make
>>>  sense, for consistency, that the configuration capabilities should include
>>>  the ability to configure the PF as is proposed.
>>  From this perspective, the kernel modules are fine for the PF: most
>> kernels of hypervisors support it without the need to upgrade their kernels.
>> To summarize, I understand that you need a flexible way to upgrade PF
>> features without touching/changing the kernel. So let's check the kernel
>> module option? VFD brings some interesting capabilities, could it be a
>> way to push and stimulate the i40e features instead of using DPDK?
>>    https://sourceforge.net/projects/e1000/files/i40e%20stable/
>> for instance could be better stimulated.

I figured I would weight in with my $0.02.

> May I ask what if DPDK VF need a new extension function from PF?
> Then, we'll have to build kernel community expertise and submit
> patch to Linux PF and wait for merge.

Correct, but it is the same story on DPDK repository as well. Either
way patch must be generated, tested, submitted and wait until it is
merged. At this point I think even the code is very similar sans
style and naming conventions for these types of features.

> Your proposal indicates DPDK community submitters will have to
> ask Linux community to authorize if we'll have any requirements
> in DPDK VF.

I don't think this is really much of a problem in most cases the same
company maintains both DPDK and Linux kernel drivers so hopefully
everyone is aligned.

> Comparing fragmentation, the extra dependency worry me most.
> Can you imagine how long it will be for any VF features gets
> ready?  

The Linux kernel releases every 6-8 weeks roughly depending on the
release. In networking stack land features must be submitted for
that kernel release before the first RC release.

So although there may be benefits to pushing these features into
DPDK vs Linux kernel I don't think actually getting the code in
the kernel is one of them.

To the point above about out of tree drivers. Most vendors do keep
out of tree drivers with the latest features and fixes that can be
supported on older kernels. Unfortunately, many of the interfaces
like the type mentioned here would require kernel updates outside
the drivers to add message handlers and driver callbacks for the
new config messages. So it may not be viable at least unless we
restructure some of the kernel to use out of tree drivers in the
short term.

So I'm sympathetic to argument (1) here. If _not_ updating the
kernel is a hard requirement then we are a bit stuck from using
existing/future Linux features. Further this is sort of a rolling
problem as well. Today we are missing feature X and we add it and
tomorrow we are missing a new feature Y. So if the feature list
is not stable with deployment of new kernels its a bit hard to
see how the Linux kernel PF model works without above mentioned
restructuring.  FWIW traditionally the Linux vendors handled this
backporting for its customers and provide kernel updates using
apt-get/dnf/whatever along with all the sanity testing to ensure
they didn't break something along the way. We are slowly getting
over this with work in BPF and other subsystems but the VF
configuration is still very much hard coded for the moment. I might
try to fix this though as a result of this thread.

Open question for me on this though is the PF in these SRIOV cases
ever being used by DPDK datapath or is it just being leveraged for
configuration with primarily control traffic?

So my take away from this is arguments about its not available in Linux
kernel are not really good ones because its easy enough to add and
get in stable kernels. But if kernel updates are a challenge then
we either need a more robust kernel backporting and out of tree
driver effort _or_ push them directly into PF driver of DPDK. I think
longterm we can fix this in the kernel with some restructuring and
better module updates but in the short term DPDK PMD updates seem
to be the only option.

I do think though if this gets pushed into DPDK PMD we shouldn't let
the PF Linux driver simply not support the new mailbox messages. This
will create one more dependency then I care to worry about. VF<->PF
mailbox should work with any combination of VF and PF as long as
version requirements are met. I think vendor maintainers can
manage this.

I'll comment on the specific features in another email because it
is helpful to have a list in public mailing list with features folks
are interested in. Any chance we could get a page up with a list
of all these and where/when we expect support for them?


john.r.fastabend at intel.com

More information about the dev mailing list