[dpdk-dev] Reg 6Wind fastpath DPDK 18.11.1 virtual memory allocation ( 54.5GB of anonymous mapping)

[Bruce Richardson]

On Wed, Jun 09, 2021 at 02:28:04PM +0200, David Marchand wrote:
> On Wed, Jun 9, 2021 at 2:02 PM Shanmugasundaram M <shansirtt at gmail.com> wrote:
> > We are using  6Wind fastpath(fp-rte) which is using DPDK version 18.11.1.
> > We are seeing  65.5G(preallocated during startup) of virtual memory for
> > fp-rte on a 32GB machine, 10G of that was huge pages (5000 mappings of 2M.)
> >
> > We found 4 strange anon (4K size) segments totaling 54.5G (16+16+16+6.5).
> > Note: we changed / customized memory CONFIG_RTE_MAX_MEMSEG_PER_LIST=24576
> > and default value of  CONFIG_RTE_MAX_MEMSEG_LISTS=64 used.
> > Kindly let us know
> > 1. Why are allocating  54.5 GB anonymous mapping
> > 2. When will we use it ?
> 
> This is likely pre-reserved virtual memory from the DPDK allocator.
> This virtual memory won't probably never be used.
> 

There may not be actually any memory backing this, since DPDK will reserve
huge amounts of virtual address space without actually using it. So those
GB's you see in the virtual memory may be completely unused, the addresses
are just reserved to ensure that if memory does need to be mapped into the
process, we have an appropriate contiguous address block to put it in.

> But you'd better ask 6WIND guys as they maintain their own DPDK downstream.
> 
> Copying a few guys.
> 
> -- 
> David Marchand
>