[dpdk-dev] [RFC 00/14] mlx5: support SubFunction

[Parav Pandit]

> From: Xia, Chenbo <chenbo.xia at intel.com>
> Sent: Tuesday, June 15, 2021 11:03 AM
> 
> Hi Parav,
> 
> > -----Original Message-----
> > From: Parav Pandit <parav at nvidia.com>
> > Sent: Tuesday, June 15, 2021 12:05 PM
> > To: Xia, Chenbo <chenbo.xia at intel.com>; NBU-Contact-Thomas Monjalon
> > <thomas at monjalon.net>; Yigit, Ferruh <ferruh.yigit at intel.com>
> > Cc: Xueming(Steven) Li <xuemingl at nvidia.com>; Slava Ovsiienko
> > <viacheslavo at nvidia.com>; dev at dpdk.org; Jason Gunthorpe
> > <jgg at nvidia.com>
> > Subject: RE: [dpdk-dev] [RFC 00/14] mlx5: support SubFunction
> >
> > Hi Chenbo,
> >
> > > From: Xia, Chenbo <chenbo.xia at intel.com>
> > > Sent: Tuesday, June 15, 2021 7:41 AM
> > >
> > > Hi Thomas,
> > >
> > > > From: Thomas Monjalon <thomas at monjalon.net>
> > > > Sent: Friday, June 11, 2021 3:54 PM
> > [..]
> >
> > >
> > > Yes. In our term it's called Assignable Device Interface (ADI)
> > > introduced in Intel Scalable IOV
> > > (https://01.org/blogs/2019/assignable-interfaces-intel-
> > > scalable-i/o-virtualization-linux)
> > >
> > > And vfio-mdev is chosen to be the software framework for it. I start
> > > to
> > realize
> > > there is difference between SF and ADI: SF considers multi-function
> > > devices which may include net/regex/vdpa/...
> > Yes. net, rdma, vdpa, regex ++.
> > And eventually vfio_device to map to VM too.
> >
> > Non mdev framework is chosen so that all the use cases of kernel only,
> > or user only or mix modes can be supported.
> 
> OK. Got it.
> 
> >
> > > But ADI only focuses on the
> > > virtualization of the devices and splitting devices to logic parts
> > > and
> > providing
> > > huge number of interfaces to host APP. I think SF also considers
> > > this but is mainly used for multi-function devices (like DPU in your term?
> > > Correct me if I'm wrong).
> > >
> > SF also supports DPU mode too but it is in addition to above use cases.
> > SF will expose mdev (or a vfio_device) to map to a VM.
> 
> So your SW actually supports vfio-mdev? I suppose the device-specific mdev
> Kernel module is out-of-tree?
> 
mlx5 driver doesn't support vfio_device for SFs. 
Kernel plumbing for PASID assignment to SF is WIP currently kernel community.
We do not have any out-of-tree kernel module.

> Just FYI:
> 
> We are introducing a new mdev bus for DPDK:
> http://patchwork.dpdk.org/project/dpdk/cover/20210601030644.3318-1-
> chenbo.xia at intel.com/
> 
I am yet to read about it. But I am not sure what value does it add.
A user can open a vfio device using vfio subsystem and operate on it. 
A vfio device can be a create as a result of binding PCI VF/PF to vfio-pci driver or a SF by binding SF to vfio_foo driver.
There is kernel work in progress to use vfio core as library.
So we do not anticipate to use add mdev layer and uuid to create a vfio device for a SF.

For Intel, ADI will never has any netdevs or rdma dev?