[dpdk-dev] Setting up hugepage memory failed when there is a java process also using hugepages

[Carlos Franco]

Hello
I am having some rare problem with hugepages. I am running DPDK 1.5.1 in CenOS 6.4. Everything is running smothly and we have done quite good progress integrating DPDK into our solution. But sometimes, there is an error with the init of the huge pages. The problem is when there is another process started that also used hugepages, in this case, a java 1.7 process (jdk 1.7.0_45 64-bits).
The machine only has one socket with 1 Xeon CPU with 4 cores and 4 GB. I have configured 256 pages of 2MB. Everything is running in 64 bits. 
grep Huge /proc/meminfo
AnonHugePages:    253952 kB
HugePages_Total:     256
HugePages_Free:      256
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
When I start the DPDK process (with –c 01 –n 4 –m 32 options), this is the trace:
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Setting up hugepage memory...
EAL: Ask a virtual area of 0x394264576 bytes
EAL: Virtual area found at 0x7fc0c9000000 (size = 0x17800000)
EAL: Ask a virtual area of 0x4194304 bytes
EAL: Virtual area found at 0x7fc0c8a00000 (size = 0x400000)
EAL: Ask a virtual area of 0x132120576 bytes
EAL: Virtual area found at 0x7fc0c0a00000 (size = 0x7e00000)
EAL: Ask a virtual area of 0x4194304 bytes
EAL: Virtual area found at 0x7fc0c0400000 (size = 0x400000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7fc0c0000000 (size = 0x200000)
EAL: Requesting 16 pages of size 2MB from socket 0
.......
Everything goes well. After the start, this is the hugepages information:
AnonHugePages:    280576 kB
HugePages_Total:     256
HugePages_Free:      220
HugePages_Rsvd:       35
HugePages_Surp:        0
Hugepagesize:       2048 kB

If I started then the java process, it starts ok and this is the hugepages information:

HugePages_Total:     256
HugePages_Free:      204
HugePages_Rsvd:       74
HugePages_Surp:        0
Hugepagesize:       2048 kB

If I stop and start the DPDK process, it fails with the following trace:

EAL: Detected lcore 0 as core 0 on socket 0
EAL: Setting up hugepage memory...
EAL: map_all_hugepages(): mmap failed: Cannot allocate memory
EAL: Failed to mmap 2 MB hugepages
PANIC in rte_eal_init():
Cannot init memory

And after the fail, this is the hugepage information:

AnonHugePages:    251904 kB
HugePages_Total:     256
HugePages_Free:       40
HugePages_Rsvd:       40
HugePages_Surp:        0
Hugepagesize:       2048 kB

It seems that it tries to map all the hugepages even if it should be limited with –m 32. After the fail, there are 201 rtemap_X files in /mnt/huge.

When I stop the java process, the DPDK process starts without any problem. I don’t know if this happens also with any other process different from java and that uses hugepages.

Am I doing something wrong? 

Thanks a lot

Carlos