[EXTERNAL] Re: [dpdk-dev] [PATCH v16 2/9] eal: add thread attributes

[Tyler Retzlaff]

-----Original Message-----
From: Stephen Hemminger <stephen at networkplumber.org> 
Sent: Tuesday, November 9, 2021 8:00 PM
To: Narcisa Ana Maria Vasile <navasile at linux.microsoft.com>
Cc: thomas <thomas at monjalon.net>; dev at dpdk.org; Dmitry Kozlyuk <dmitry.kozliuk at gmail.com>; Khoa To <khot at microsoft.com>; Dmitry Malloy <dmitrym at microsoft.com>; Tyler Retzlaff <roretzla at microsoft.com>; talshn <talshn at mellanox.com>; Omar Cardona <ocardona at microsoft.com>; bruce.richardson at intel.com; david.marchand at redhat.com; Kadam, Pallavi <pallavi.kadam at intel.com>
Subject: [EXTERNAL] Re: [dpdk-dev] [PATCH v16 2/9] eal: add thread attributes

On Tue, 9 Nov 2021 19:04:17 -0800
Narcisa Ana Maria Vasile <navasile at linux.microsoft.com> wrote:

> > > >   I'll send a new version with a better commit message.
> > > >   Thread priorities on both Linux-based and Windows platforms are similarly
> > > >   constructed from a class/policy + priority value. Currently in DPDK, most threads
> > > >   operate at the OS-default priority level but there are cases when increasing the
> > > >   priority is useful. For example, the Mellanox data path acceleration driver requires
> > > >   realtime thread priority. Similarly, some Windows applications will require elevated
> > > >   priority.  
> > > 
> > >  It should not. We should not use realtime priority.  
> >
> > Thomas, can you join the community sync tomorrow? I'll bring this up to discuss.
> >
> > High performance applications benefit from an option to raise the 
> > priority of their threads to avoid being preemted by other threads on 
> > the system. If there are issues with realtime priority on some of the 
> > platforms, maybe we can add a warning for the user to make them aware 
> > of possible crashes as Stephen H. suggested some time ago. Note that 
> > this patch doesn't change the priority of EAL threads, enabling the higher priority will be done through a command line option when starting the application.
> > Maybe we can explore raising the priority but not to the realtime level.

> Let me put it more succulently.
> Almost all DPDK applications have threads that are a 100% CPU doing polling.
> Putting those thread as real-time thread breaks Linux badly because the kernel can and will try and run work on those CPU's and the system is broken/unstable/dead at that point.

The suggestion is that when running on Windows it should be possible for the application to be configured to use real-time threads, there is no implication that it will force real-time priority on Linux.  If it doesn't make sense for it to be configured on Linux then don't configure it.  But saying it shall not be configurable for any platform just because one platform can't make use of the configuration and those platforms have to run compromised makes little sense.  Linux administrators are (or should be) knowledgeable enough to know what configuration to use.

> > This is a case of different definitions and expectations of real-time in Linux and Windows.  Linux definition of real-time priority is for well behaved and time critical applications. It expects RT applications to run, then sleep.