[dpdk-dev] Multi-process on the same host

[Richardson, Bruce]

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Stephen
> Hemminger
> Sent: Friday, October 04, 2013 5:39 PM
> To: Walter de Donato
> Cc: dev at dpdk.org
> Subject: Re: [dpdk-dev] Multi-process on the same host
> 
> On Fri, 4 Oct 2013 13:47:02 +0200
> Walter de Donato <walter.dedonato at unina.it> wrote:
> 
> > Hello,
> >
> > I've been using DPDK for a while and now I encountered the following
> issue:
> > when I try to run two primary processes on the same host (with
> > --no-shconf option enabled) respectively sending packets on one port
> > and receiving them on a different port (the two ports are directly
> > connected with a CAT-6 cable), I get this error on the receiving process:
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > 0x00000000004158a0 in rte_eth_rx_burst (port_id=0 '\000',
> queue_id=0,
> > rx_pkts=0x7ffff5baa8f0, nb_pkts=128) at
> > /home/devel/dpdk/build/include/rte_ethdev.h:1658
> > 1658            return (*dev->rx_pkt_burst)(dev->data-
> >rx_queues[queue_id],
> > rx_pkts, nb_pkts);
> >
> > To give some more details:
> > - the options given to the two processes:
> >   ./receiver -c 0x3 -n 2 -m 200 --no-shconf -- -p 0x1
> >   ./sender -c 0xc -n 2 -m 200 --no-shconf -- -p 0x2
> >   where the -p option is the binary mask to select the ports to enable.
> > - the network card is a dualport Intel X540:
> >   port 0: Intel Corporation Ethernet Controller 10 Gigabit X540-AT2 (rev
> 01)
> >   port 1: Intel Corporation Ethernet Controller 10 Gigabit X540-AT2
> > (rev 01)
> > - this is the hugeadm --pool-list output:
> >       Size  Minimum  Current  Maximum  Default
> >   1073741824        2        2        2        *
> >
> > My first question is: should it be possible to let separate primary
> > processes coexist if they use different resources (cores, ports,
> > memory pools)?
> >
> > A second question is: there is any other workaround to let this
> > scenario work without merging the two processes into two lcores of the
> same process?
> >
> > Thanks in advance,
> > -Walter
> 
> The problem is that huge TLB filesystem is a shared resource.
> Because of that the memory pools of the two processes overlap, and
> memory pools are used for packet buffers, malloc, etc.
> 
> You might be able to use no-huge, but then other things would probably
> break.

The way to run two primary processes side by side is documented in the document "Intel(r) Data Plane Development Kit (Intel(r) DPDK): Programmer's Guide" available at: http://www.intel.com/content/www/us/en/intelligent-systems/intel-technology/intel-dpdk-programmers-guide.html and is covered in section 17.2.3. You need to pass the "--file-prefix" flag when running your application to force the processes to use different hugepage files so they are not shared among the two processes.

Regards,
/Bruce
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.