[dpdk-dev] [RFC] [PATCH] eventdev: abstract ethdev HW capability to inject packets to eventdev

[Van Haaren, Harry]

Some comments inline with [HvH] prefix

> -----Original Message-----
> From: Jerin Jacob [mailto:jerin.jacob at caviumnetworks.com]
> Sent: Tuesday, May 2, 2017 5:01 PM
> To: Eads, Gage <gage.eads at intel.com>
> Cc: dev at dpdk.org; Richardson, Bruce <bruce.richardson at intel.com>; Van Haaren, Harry
> <harry.van.haaren at intel.com>; hemant.agrawal at nxp.com; nipun.gupta at nxp.com; Vangati, Narender
> <narender.vangati at intel.com>
> Subject: Re: [RFC] [dpdk-dev] [PATCH] eventdev: abstract ethdev HW capability to inject packets
> to eventdev
> 
> -----Original Message-----
> > Date: Fri, 21 Apr 2017 22:31:52 +0000
> > From: "Eads, Gage" <gage.eads at intel.com>
> > To: Jerin Jacob <jerin.jacob at caviumnetworks.com>, "dev at dpdk.org"
> >  <dev at dpdk.org>
> > CC: "Richardson, Bruce" <bruce.richardson at intel.com>, "Van Haaren, Harry"
> >  <harry.van.haaren at intel.com>, "hemant.agrawal at nxp.com"
> >  <hemant.agrawal at nxp.com>, "nipun.gupta at nxp.com" <nipun.gupta at nxp.com>,
> >  "Vangati, Narender" <narender.vangati at intel.com>
> > Subject: RE: [RFC] [dpdk-dev] [PATCH] eventdev: abstract ethdev HW
> >  capability to inject packets to eventdev
> >
> > Hi Jerin,
> 
> Hi Gage,
> 
> >
> > Thanks for getting this ball rolling, and I agree that we need a solution that covers the
> three cases you described.
> 
> OK. Half problem is solved if we agree on problem statement :-)

[HvH]
+2 :)


> > We've also been thinking about an environment where devices (NIC Rx (or even Tx), crypto, or
> a timer "device" that uses librte_timer to inject events) can plug in eventdev -- whether
> through a direct connection to the event scheduler (case #3) or using software to bridge the
> gap -- such that application software can have a consistent view of device interfacing on
> different platforms.
> 
> Make sense. Yes, The NPUs can produce events from NIC Rx, NIC Tx, crypto, timer device
> sources without SW service functions.
> 
> >
> > Some initial thoughts on your proposal:
> >
> > 1. I imagine that deploying these service functions at the granularity of a core can be
> excessive on devices with few (<= 8) cores. For example, if the crypto traffic rate is low then
> a cryptodev service function could be co-scheduled with other service functions and/or
> application work. I think we'll need a more flexible deployment of these service functions.
> 
> I agree.
> 
> >
> > 2. Knowing which device type a service function is for would be useful -- without it, it's
> not possible to assign the function to the NUMA node on which the device is located.
> 
> I guess we can use rte_eth_dev_socket_id() on requested port to get NUMA
> id.
> 
> >
> > 3. Placing the service core logic in the PMDs is nice in terms of application ease-of-use,
> but it forces PMD to write one-size-fits-all service core functions, where, for example, the
> application's control of the NIC Rx functionality is limited to the options that struct
> rte_event_queue_producer_conf exports. An application may want customized service core behavior
> such as: prioritized polling of Rx queues, using Rx queue interrupts for low traffic rate
> queues, or (for "closed system" eventdevs) control over whether/when a service core drops
> events (and a way to notify applications of event drops). For such cases, I think the
> appropriate solution is allow applications to plug in their own service core functions (when
> hardware support isn't present).
> 
> I agree. I think, we can have reusable producer code as static inline
> functions in librte_event with multiple event producing strategies and
> let application to call respective one if HW support is not present or
> not adequate.
> 
> I will work towards this theme in RFC v2.

[HvH]
Yes agree. I'd like to suggest that there might be two issues we're solving at the same time, A) How to "grab" cores for a generic software fallback purpose, and B) how we can enable the various eth/event/crypto-dev components to "play nice"

For A),  I have a header file that I'd like to share as an RFC on allowing EAL to manage this requesting of cores. The RFC does not deal with B) and configuration of eth/event/crypto devs.


> > Some of these thoughts are reflected in the eventdev_pipeline app[1] that Harry submitted
> earlier today, like flexible service function deployment. In that app, the user supplies a
> device coremask that can pin a service function to a core, multiplex multiple functions on the
> core, or even affinitize the service function to multiple cores (using cmpset-based exclusion
> to ensure it's executed by one lcore at a time).
> 
> Thanks for the sample application.I could make it work with NIC + HW
> eventdev with some tweaking. I will send the review comment on that
> email thread.

[HvH]
Great!

> One thing, I noticed with cmpset based scheme is that, at given point of
> time it can produce at most up to the events one LCORE can support.May not
> be well suited for low end cores.I think, we need multiple event
> producer strategy code as common code.
>
> > In thinking about this, Narender and I have envisioned something like a framework for
> eventdev applications in which these service functions can be registered and (in a similar
> manner to eventdev_pipeline's service functions) executed.
> 
> That will be useful. I think it will be not just restricted to eventdev
> applications, I guess, New traffic manager's SW implementation or any
> future offloads need a framework for service function registration and
> invocation.

[HvH]
Yes, the proposal for A) above would be mostly EAL based, providing a generic "service core" API to DPDK components that require an lcore for "internal" tasks (e.g.: Eventdev SW PMD).

I'll get the RFC up ASAP with you guys on CC, -Harry