[dpdk-dev] proposal: raw packet send and receive API for PMD driver

[Venkatesan, Venky]

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Wiles, Keith
> Sent: Wednesday, May 27, 2015 7:51 AM
> To: Lin XU; dev at dpdk.org
> Subject: Re: [dpdk-dev] proposal: raw packet send and receive API for PMD
> driver
> 
> 
> 
> On 5/26/15, 11:18 PM, "Lin XU" <lxu at astri.org> wrote:
> 
> >I think it is very important to decouple PMD driver with DPDK framework.
> >   (1) Currently, the rte_mbuf struct is too simple and hard to support
> >complex application such as IPSEC, flow control etc. This key struct
> >should be extendable to support customer defined management header
> and
> >hardware offloading feature.
> 
> I was wondering if adding something like M_EXT support for external storage
> to DPDK MBUF would be more reasonable.
> 
> IMO decoupling PMDs from DPDK will possible impact performance and I
> would prefer not to let this happen. The drivers are written for performance,
> but they did start out as normal FreeBSD/Linux drivers. Most of the core
> code to the Intel drivers are shared between other systems.
> 
This was an explicit design choice to keep the mbuf simple, yet sufficient to service volume NIC controllers and the limited offloads that they have. I would prefer not to have rte_mbuf burdened with all that a protocol stack needs - that will simply increase the size of the structure and penalize applications that need a lean structure (like security applications). Extending the mbuf to 128 bytes itself caused a regression in some performance apps. 

That said, extensibility or for that matter a custom defined header is possible with in at least two ways. 
a) the userdata pointer field can be set to point to a data structure that contains more information
b) you could simply embed the custom structure (like the pipeline code does) behind the rte_mbuf

These can be used to pass through any information from NICs that support hardware offloads as well as carrying areas for protocol stack specific information (e.g. complete IPSEC offload). 

> >   (2) To support more NICs.
> >So, I thinks it time to add new API for PMD(a no radical way), and
> >developer can add initial callback function in PMD for various upper
> >layer protocol procedures.
> 
> We have one callback now I think, but what callbacks do you need?
> 
> The only callback I can think of is for a stack to know when it can release its
> hold on the data as it has been transmitted for retry needs.
> >
> >

The one place that I do think we need to change is the memory allocation framework - allowing external memory allocators (buf alloc/free) so that the driver could be run within a completely different memory allocator system. It can be done with the system we have in place today with specific overrides,  but it isn't simple. I think there was another request along similar lines on the list. This would pretty much allow a TCP stack for example to allocate and manage memory (as long as the driver interface via an MBUF can be maintained).  If this is something valuable, we could look at pursuing this for the next release. 

-Venky