[dpdk-dev] EAL reports "ot enough memory available on socket 0!"

[Bruce Richardson]

On Tue, Feb 06, 2018 at 09:09:26PM +0000, Wiles, Keith wrote:
> 
> 
> > On Feb 6, 2018, at 12:38 PM, Huertas García, Víctor <vhuertas at indra.es> wrote:
> > 
> > Hi all,
> > 
> > 
> > This is my first post in this mailing list. I am glad to become a member of this community.
> > 
> > 
> > I am a newbie using DPDK and I have tried to compile and execute an example app called "l3fwd-acl".
> > 
> > 
> > The DPDK command is DPDK_Test -c 0xf0 -n 2 --socket-mem 64
> 
> ran this command as root or used sudo correct?
> 
> > 
> > but the application tells me the following:
> > 
> > 
> > EAL: Detected 8 lcore(s)
> > EAL: Probing VFIO support...
> > EAL: VFIO support initialized
> > EAL: Cannot obtain physical addresses: Success. Only vfio will function.
> > EAL: Not enough memory available on socket 0! Requested: 64MB, available: 0MB
> > EAL: FATAL: Cannot init memory
> > 
> > EAL: Cannot init memory
> > 
> > EAL: Error - exiting with code: 1
> >  Cause: Invalid EAL parameters
> > 
> > 
> > 
> > I don't know what else I can do to make this app run as I have reserved hugepages as the user guide tells as well as mounted the /mnt/huge folder.
> > 
> > 
> > cat /proc/meminfo | grep Huge
> > AnonHugePages:    520192 kB
> > HugePages_Total:      64
> > HugePages_Free:       64
> > HugePages_Rsvd:        0
> > HugePages_Surp:        0
> > Hugepagesize:       2048 kB
> 
> I assume you added an entry into /etc/sysctl.conf and rebooted, just to make sure the huge pages are allocated early in the boot process.
> 
> I use 1G huge pages and I assume you are using 2M huge pages
> 
> vm.nr_hugepages=32  # 32x2M is 64M
> 
> I also disable randomize_va_space, which is not required for those tests.
> 
> kernel.randomize_va_space=1
> 
> 
> > 
> > 
> > My machine is has this topology (only one Numa node is detected):
> > 
> > 
> > udo usertools/cpu_layout.py
> > [sudo] password for root:
> > ======================================================================
> > Core and Socket Information (as reported by '/sys/devices/system/cpu')
> > ======================================================================
> > 
> > cores =  [0, 1, 2, 3]
> > sockets =  [0, 1]
> > 
> >       Socket 0    Socket 1
> >       --------    --------
> > Core 0 [0]         [4]
> > Core 1 [1]         [5]
> > Core 2 [2]         [6]
> > Core 3 [3]         [7]
> 
> The CPU layout shows two sockets and normally that means 2 numa node.
> 
> I would change the command above —-socket-mem 32,32  (note: two ‘-‘ in front) this command gives 32M socket 0 and 32M to socket 1
> 
> I have seen some funny memory configs, with huge pages if you do not use /etc/sysctl.conf and reboot. Sometimes memory will be allocated to one socket.
> 
> I hope that helps.
> 
I would also suggest increasing the amount of memory reserved. DPDK can
sometimes be quite memory hungry, so perhaps up the amount of hugepages
reserved in the kernel. Also, if you are ok with DPDK using all the
hugepages you have statically reserved, you can omit the --socket-mem
parameter.

/Bruce