[dpdk-dev] Huge mapping secondary process linux

Jonas Pfefferle1 JPF at zurich.ibm.com
Fri Oct 27 16:58:01 CEST 2017


"Burakov, Anatoly" <anatoly.burakov at intel.com> wrote on 10/27/2017 04:44:52
PM:

> From: "Burakov, Anatoly" <anatoly.burakov at intel.com>
> To: Jonas Pfefferle1 <JPF at zurich.ibm.com>
> Cc: bruce.richardson at intel.com, chaozhu at linux.vnet.ibm.com, dev at dpdk.org
> Date: 10/27/2017 04:45 PM
> Subject: Re: [dpdk-dev] Huge mapping secondary process linux
>
> On 27-Oct-17 3:28 PM, Jonas Pfefferle1 wrote:
> > "Burakov, Anatoly" <anatoly.burakov at intel.com> wrote on 10/27/2017
> > 04:06:44 PM:
> >
> >  > From: "Burakov, Anatoly" <anatoly.burakov at intel.com>
> >  > To: Jonas Pfefferle1 <JPF at zurich.ibm.com>, dev at dpdk.org
> >  > Cc: chaozhu at linux.vnet.ibm.com, bruce.richardson at intel.com
> >  > Date: 10/27/2017 04:06 PM
> >  > Subject: Re: [dpdk-dev] Huge mapping secondary process linux
> >  >
> >  > On 27-Oct-17 1:43 PM, Jonas Pfefferle1 wrote:
> >  > >
> >  > >
> >  > > Hi @all,
> >  > >
> >  > > I'm trying to make sense of the hugepage memory mappings in
> >  > > librte_eal/linuxapp/eal/eal_memory.c:
> >  > > * In rte_eal_hugepage_attach (line 1347) when we try to do a
private
> >  > > mapping on /dev/zero (line 1393) why do we not use MAP_FIXED if we

> > need the
> >  > > addresses to be identical with the primary process?
> >  > > * On POWER we have this weird business going on where we use
> > MAP_HUGETLB
> >  > > because according to this commit:
> >  > >
> >  > > commit 284ae3e9ff9a92575c28c858efd2c85c8de6d440
> >  > > Author: Chao Zhu <chaozhu at linux.vnet.ibm.com>
> >  > > Date:   Thu Apr 6 15:36:09 2017 +0530
> >  > >
> >  > >      eal/ppc: fix mmap for memory initialization
> >  > >
> >  > >      On IBM POWER platform, when mapping /dev/zero file to
hugepage
> > memory
> >  > >      space, mmap will not respect the requested address hint.This
will
> >  > > cause
> >  > >      the memory initialization for the second process fails. This
> > patch adds
> >  > >      the required mmap flags to make it work. Beside this, users
> > need to set
> >  > >      the nr_overcommit_hugepages to expand the VA range. When
> >  > >      doing the initialization, users need to set both nr_hugepages
and
> >  > >      nr_overcommit_hugepages to the same value, like 64, 128, etc.
> >  > >
> >  > > mmap address hints are not respected. Looking at the mmap code in
the
> >  > > kernel this is not true entirely however under some circumstances
> > the hint
> >  > > can be ignored (
> >  > > https://urldefense.proofpoint.com/v2/url?
> >  >
> >
>
u=http-3A__elixir.free-2Delectrons.com_linux_latest_source_arch_powerpc_mm_mmap.c-23L103&d=DwICaQ&c=jf_iaSHvJObTbx-

> >  > siA1ZOg&r=rOdXhRsgn8Iur7bDE0vgwvo6TC8OpoDN-
> >  > pXjigIjRW0&m=cttQcHlAYixhsYS3lz-
> >  >
BAdEeg4dpbwGdPnj2R3I8Do0&s=Gp0TIjUtIed05Jgb7XnlocpCYZdFXZXiH0LqIWiNMhA&e=
> >  > > ). However I believe we can remove the extra case for PPC if we
use
> >  > > MAP_FIXED when doing the secondary process mappings because we
need
> > them to
> >  > > be identical anyway. We could also use MAP_FIXED when doing the
primary
> >  > > process mappings resp. get_virtual_area if we want to have any
> > guarantees
> >  > > when specifying a base address. Any thoughts?
> >  > >
> >  > > Thanks,
> >  > > Jonas
> >  > >
> >  > hi Jonas,
> >  >
> >  > MAP_FIXED is not used because it's dangerous, it unmaps anything
that is
> >  > already mapped into that space. We would rather know that we can't
map
> >  > something than unwittingly unmap something that was mapped before.
> >
> > Ok, I see. Maybe we can add a check to the primary process's memory
> > mappings whether the hint has been respected or not? At least warn if
it
> > hasn't.
>
> Hi Jonas,
>
> I'm unfamiliar with POWER platform, so i'm afraid you'd have to explain
> a bit more what you mean by "hint has been respected" :)

Hi Anatoly,

What I meant was the mmap address hint:

"If addr is not NULL, then the kernel takes it as a hint
 about where to place the mapping; on Linux, the mapping will be
 created at a nearby page boundary."

This is actually not true on POWER. It can happen that the address hint is
ignored and you get any address back that fits your mapping.

Thanks,
Jonas

>
>
> --
> Thanks,
> Anatoly
>


More information about the dev mailing list