[dpdk-dev] Service lcores and Application lcores

[Van Haaren, Harry]

> From: Thomas Monjalon [mailto:thomas at monjalon.net]
> Sent: Friday, June 30, 2017 10:29 AM
> To: Van Haaren, Harry <harry.van.haaren at intel.com>
> Cc: dev at dpdk.org; 'Jerin Jacob' <jerin.jacob at caviumnetworks.com>; Wiles, Keith
> <keith.wiles at intel.com>; Richardson, Bruce <bruce.richardson at intel.com>
> Subject: Re: Service lcores and Application lcores
> 
> 30/06/2017 10:52, Van Haaren, Harry:
> > From: Thomas Monjalon [mailto:thomas at monjalon.net]
> > > 29/06/2017 18:35, Van Haaren, Harry:
> > > > 3) The problem;
> > > >    If a service core runs the SW PMD schedule() function (option 2) *AND*
> > > >    the application lcore runs schedule() func (option 1), the result is that
> > > >    two threads are concurrently running a multi-thread unsafe function.
> > >
> > > Which function is multi-thread unsafe?
> >
> > With the current design, the service-callback does not have to be multi-thread safe.
> > For example, the eventdev SW PMD is not multi-thread safe.
> >
> > The service library handles serializing access to the service-callback if multiple cores
> > are mapped to that service. This keeps the atomic complexity in one place, and keeps
> > services as light-weight to implement as possible.
> >
> > (We could consider forcing all service-callbacks to be multi-thread safe by using
> atomics,
> > but we would not be able to optimize away the atomic cmpset if it is not required. This
> > feels heavy handed, and would cause useless atomic ops to execute.)
> 
> OK thank you for the detailed explanation.
> 
> > > Why the same function would be run by the service and by the scheduler?
> >
> > The same function can be run concurrently by the application, and a service core.
> > The root cause that this could happen is that an application can *think* it is the
> > only one running threads, but in reality one or more service-cores may be running
> > in the background.
> >
> > The service lcores and application lcores existence without knowledge of the others
> > behavior is the cause of concurrent running of the multi-thread unsafe service function.
> 
> That's the part I still don't understand.
> Why an application would run a function on its own core if it is already
> run as a service? Can we just have a check that the service API exists
> and that the service is running?


The point is that really it is an application / service core mis-match.
The application should never run a PMD that it knows also has a service core running it.

However, porting applications to the service-core API has an over-lap time where an
application on 17.05 will be required to call eg: rte_eventdev_schedule() itself, and
depending on startup EAL flags for service-cores, it may-or-may-not have to call schedule() manually.

This is pretty error prone, and mis-configuration would cause A) deadlock due to no CPU cycles, B) segfault due to two cores.

As per the other thread re service-lcores[1], removing rte_eventdev_schedule() from the API would force apps to use the service-core infrastructure for eventdev instead of the possibility of mis-match.


[1] http://dpdk.org/ml/archives/dev/2017-June/069492.html