[dpdk-dev] [PATCH v2 0/7] Removal of PCI bus ABIs

[Walker, Benjamin]

> From: dev <dev-bounces at dpdk.org> On Behalf Of Thomas Monjalon
> 
> 12/10/2021 02:35, Harris, James R:
> > On 10/11/21, 5:55 AM, "Thomas Monjalon" <thomas at monjalon.net> wrote:
> >
> >     The meson option enable_driver_sdk is described as "Install headers to build
> drivers."
> >     Standard development packages should provide headers to build an
> application.
> >     This option is for projects extending DPDK drivers out of the tree.
> >     The preferred option is to develop drivers inside DPDK.
> >
> >     If a project needs the special option enable_driver_sdk,
> >     1/ it is not following the recommended approach,
> >     2/ it has to manage the burden of driver compatibility with DPDK,
> >     3/ it can compile DPDK itself.
> >
> >     So I think we neither need to make it a default, nor force distros to enable it.
> >     Am I missing something?
> >
> > Hi Thomas,
> >
> > This preference to develop PCI drivers inside of DPDK seems to be a very
> recent preference.  enable_driver_sdk was just added in DPDK 21.05, and for
> building out-of-tree ethdev drivers. But DPDK has always enabled building out-
> of-tree PCI drivers with its default build configuration - SPDK has relied on these
> APIs since its inception.
> 
> Yes DPDK allows out-of-tree drivers, but it has never been recommended.

For networking drivers, maybe. But certainly years and years ago when SPDK was started no one recommended putting an nvme driver into DPDK.

> We have introduced enable_driver_sdk option recently to keep allowing out-of-
> tree drivers.
> 
> > We have always viewed DPDK as being a very useful toolkit for building
> userspace drivers (especially storage drivers!) that aren't part of DPDK itself.  We
> hope that continues to be the case.
> 
> Yes, there is no plan to stop that, but also no plan to make it easier.

To be clear, this change actively makes it harder. DPDK has changed the longstanding status quo.

> 
> > All of that being said, SPDK already compiles DPDK itself as the default
> configuration. We maintain a DPDK fork for patches that have not yet hit DPDK
> upstream. If this gets merged we can document that users building DPDK
> themselves must set enable_driver_sdk. We can also document to our users that
> SPDK may not build against distro DPDK packages, once distros pick up these
> changes.
> 
> Yes I think that's the right thing to do.
> 

This means that a distro-packaged SPDK cannot exist, because it cannot use a distro-packaged DPDK as a dependency. While using a distro-packaged SPDK is not the common case (people just build it themselves), my personal view is that we need to be able to support this and this change from DPDK is unacceptable.

> Note: I don't remember the reason to keep your drivers out of DPDK?

SPDK uses DPDK as a framework for writing user space drivers only - scanning the PCI bus, allocating DMA-safe memory, etc. This functionality is hidden behind an abstraction layer that can be reimplemented by our users to remove the DPDK dependency entirely, and real production users have elected to do this. The reasons they do this are varied, but the shortest way to say it is that DPDK is a framework that requires their application to buy-in across the board, whereas SPDK is a set of libraries that integrates into their existing application more easily.

SPDK simply uses DPDK as the default implementation for this functionality. We cannot port our drivers into DPDK or it would break this use case. 

Thanks,
Ben