[dpdk-dev] [PATCH v13 09/10] eal: add EAL argument for setting thread priority

[Bruce Richardson]

On Wed, Aug 18, 2021 at 02:28:33PM -0700, Stephen Hemminger wrote:
> On Tue,  3 Aug 2021 12:01:30 -0700 Narcisa Ana Maria Vasile
> <navasile at linux.microsoft.com> wrote:
> 
> > +static int +eal_parse_thread_priority(const char *arg) +{ +
> > struct internal_config *internal_conf = +
> > eal_get_internal_configuration(); +	enum rte_thread_priority priority;
> > + +	if (!strncmp("normal", arg, sizeof("normal"))) +
> > priority = RTE_THREAD_PRIORITY_NORMAL; +	else if
> > (!strncmp("realtime", arg, sizeof("realtime"))) +		priority =
> > RTE_THREAD_PRIORITY_REALTIME_CRITICAL; +	else +		return -1;
> > + +	internal_conf->thread_priority = priority; +	return 0; +} +
> 
> In my experience using real time priority is dangerous and risks
> starvation and deadlock. The problem is that DPDK applications are
> typically 100% CPU poll mode with no system calls; but the kernel has a
> number of worker threads that can be required on those CPUs.
> 
> The typical failure is a disk completion interrupt happens on a CPU that
> is being used by DPDK lcore thread. With RT priority, the kernel thread
> to process that I/O completion never runs because the RT user thread has
> higher priority than the kernel I/O completion thread.
> 
> It maybe possible to workaround this with lots of hand crafting through
> sysfs to reassign threads and irq's. Also, later kernels with full RT
> might also help.
> 
> Before putting this in as part of DPDK in EAL, a full set of testing and
> documentation of how to configure these kind of applications and systems
> is needed.
>
I would tend to agree caution here, based on my experience of having locked
up a number of systems in the past when testing running DPDK apps with RT
priority!