[dpdk-dev] Question on DPDK multi-process support

[Richardson, Bruce]

If you disable mempool caches, you may be able to run more processes than you have cores, but other issues with duplicate core id's might arise, and your application performance will suffer as the memory pools will be very significantly slower.
However, there is no requirement that the lcore_id actually matches the physical core on which the thread is running, so for threads where the core ids will overlap, you could assign a new core id value for some of the them. This can be done in your code by just writing to the global variable _lcore_id, which is defined in eal_thread.c. The core id value must be unique to each thread, and must be between 0 and RTE_MAX_LCORE.
In general, though, it is recommended that for packet processing you run only a single thread pinned to each logical core on a system.

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Mauro
> Annarumma
> Sent: Tuesday, February 18, 2014 12:46 PM
> To: dev at dpdk.org
> Subject: Re: [dpdk-dev] Question on DPDK multi-process support
> 
> Hi Bruce,
>    just for dispel
>  any doubt: if there is one primary process and N secondary processes, and
> each process uses only one logical core (they have a coremask equal to
> 0x01, 0x02, 0x04, ....), I can run at the same time at most "my CPU logical
> cores - 1" secondary processes. Is it right?
> Is it true also if I disable the mempool per-core cache?
> 
> Thanks for your help, regards.
> 
> Mauro
> 
> > From: bruce.richardson at intel.com
> > To: mauroannarumma at hotmail.it; dev at dpdk.org
> > Subject: RE: [dpdk-dev] Question on DPDK multi-process support
> > Date: Mon, 17 Feb 2014 16:44:55 +0000
> >
> > > Hi,
> > > I have some doubts on the DPDK multi-process support:
> > >
> > > 1) According to the Programmers Guide (pag. 109 - 20.3 Multi-process
> > > Limitations), one of the multi-process limitations is:
> > >
> > > "All Intel(r) DPDK processes running as a single application and
> > > using shared memory must have distinct coremask arguments. It is not
> > > possible to have a primary and secondary instance, or two secondary
> > > instances, using any of the same logical cores. Attempting to do so
> > > can cause corruption of memory pool caches, among other issues."
> > >
> > > Does it mean that I can run a number of DPDK processes at most equal
> > > to the number of my CPU logical cores? In the case I run 2
> > > mono-lcore DPDK processes with the same coremask, what are the
> possible issues?
> > >
> > The potential issues are caused by a dependence on the lcore_id internally
> by Intel DPDK data structures, especially mempools. If two processes use
> the same lcore they will have the same lcore_id value, and will try and
> access the same mempool cache which is not thread-safe. This will cause
> mempool corruption.
> > NOTE: this applies only to co-operating processes, i.e. those run as
> primary and secondary processes. There are no mempool issues with
> running completely independent Intel DPDK processes on the same cores,
> i.e. processes run using different "--file-prefix=" parameters, since those do
> not share any memory and data structures.
> >
> > >
> > > 2) Can a secondary process use the rte_eth_rx_burst over a queue of
> > > a NIC port configured in the primary process?
> > >
> > Yes. NIC ports and their data structures are shared between co-operating
> processes. Multiple processes doing RX on the same port is shown in the
> "examples/multi_process/symmetric_mp" example application.
>