Hello, I am currently using DPDK 19.11. I notice the process virtual size is huge. I tried running the test PMD. It takes 64G VSZ and if I use the '--in-memory' option it goes up to 188G. Is there anyway to disable allocation of such huge VSZ in DPDK ? Also I found that version 18.02.2 doesn't have the problem, but 18.05.1 and further releases have it. Would really appreciate if someone can help, if there is a patch to get over this issue in the DPDK code ? This is becoming a huge practical issue as huge core files are getting generated. On multi NUMA setup, the VSZ goes above 400G and I can't get core files to debug crashes in my app. Regards, Siddarth
This is "by design". In order to support dynamic memory allocation for secondary processes, we need to guarantee that the memory space we'll be allocating *into* is available at all times. This is why this memory is reserved at startup, and the amount of memory being reserved is pretty huge (and depends on your system configuration - the more page sizes and NUMA nodes you have, the bigger the reserved amount) because we don't know in advance how much memory an application would need during its lifetime. This is the price we have to pay to support multiprocess *and* dynamic memory scaling. I do have a patch in the works that makes that amount configurable, but it is not possible to avoid that entirely without seriously restructuring the memory subsystem and adding a yet-another path to its already overcomplicated inner workings, so i'm not sure how much is there to fix. Even my patch provides relatively little control over the allocation process, because the EAL command-line is already complex-enough and i don't want to add fifteen different options to tune the allocator behavior. (plus, it also creates a maintenance and support burden) The reason this didn't happen in 18.02 is because the 18.05 release was the first one to feature the new memory subsystem.
Siddarth, Does the explanation by Anatoly clarify your understanding? Thanks