[dpdk-dev] bifurcated driver

[Zhou, Danny]

Hi Thomas,

Thanks for sharing the links to ibverbs, I will take a close look at it and compare it to bifurcated driver. My take
after a rough review is that idea is very much similar, but bifurcated driver implementation is generic for any 
Ethernet device based on existing af_packet mechanism, with extension of exchanging the messages between 
user space and kernel space driver.

I have an internal document to summary the pros and cons of below solutions, except for ibvers, but 
will be adding it shortly.

- igb_uio
- uio_pci_generic
- VFIO
- bifurcated driver

Short answers to your questions:
> 	- upstream status
Adding IOMMU based memory protection and generic descriptor description support now, into version 2 
kernel patches.

> 	- usable with kernel netdev
af_packet based, and relevant patchset will be submitted to netdev for sure.

> 	- usable in a vm
No, it does no coexist with SRIOV for number of reasons. but if you pass-through a PF to a VM, it works perfect.

> 	- usable for Ethernet
It could work with all Ethernet NICs, as flow director is available and NIC driver support new net_ops to split off 
queue pairs for user space.

> 	- hardware requirements
No specific hardware requirements. All mainstream NICs have multiple qpairs and flow director support. 

> 	- security protection
Leverage IOMMU to provide memory protection on Intel platform. Other archs provide similar memory protection
mechanism, so we only use arch-agnostic DMA memory allocation APIs in kernel to support memory protection.

> 	- performance
DPDK native performance on user space queues, as long as drop_en is enabled to avoid head-of-line blocking.

-Danny

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
> Sent: Wednesday, November 05, 2014 9:01 PM
> To: Zhou, Danny
> Cc: dev at dpdk.org; Fastabend, John R
> Subject: Re: [dpdk-dev] bifurcated driver
> 
> Hi Danny,
> 
> 2014-10-31 17:36, O'driscoll, Tim:
> > Bifurcated Driver (Danny.Zhou at intel.com)
> 
> Thanks for the presentation of bifurcated driver during the community call.
> I asked if you looked at ibverbs and you wanted a link to check.
> The kernel module is here:
> 	http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/infiniband/core
> The userspace library:
> 	http://git.kernel.org/cgit/libs/infiniband/libibverbs.git
> 
> Extract from Kconfig:
> "
> config INFINIBAND_USER_ACCESS
> 	tristate "InfiniBand userspace access (verbs and CM)"
> 	select ANON_INODES
> 	---help---
> 	  Userspace InfiniBand access support.  This enables the
> 	  kernel side of userspace verbs and the userspace
> 	  communication manager (CM).  This allows userspace processes
> 	  to set up connections and directly access InfiniBand
> 	  hardware for fast-path operations.  You will also need
> 	  libibverbs, libibcm and a hardware driver library from
> 	  <http://www.openfabrics.org/git/>.
> "
> 
> It seems to be close to the bifurcated driver needs.
> Not sure if it can solve the security issues if there is no dedicated MMU
> in the NIC.
> 
> I feel we should sum up pros and cons of
> 	- igb_uio
> 	- uio_pci_generic
> 	- VFIO
> 	- ibverbs
> 	- bifurcated driver
> I suggest to consider these criterias:
> 	- upstream status
> 	- usable with kernel netdev
> 	- usable in a vm
> 	- usable for ethernet
> 	- hardware requirements
> 	- security protection
> 	- performance
> 
> --
> Thomas