[dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA nodes

[Ruifeng Wang]

Hi Juraj,

> -----Original Message-----
> From: dev <dev-bounces at dpdk.org> On Behalf Of Bruce Richardson
> Sent: Friday, August 28, 2020 9:10 PM
> To: Juraj Linkeš <juraj.linkes at pantheon.tech>
> Cc: dev at dpdk.org; yskoh at mellanox.com; Honnappa Nagarahalli
> <Honnappa.Nagarahalli at arm.com>
> Subject: Re: [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all
> NUMA nodes
> 
> On Fri, Aug 28, 2020 at 11:11:36AM +0000, Juraj Linkeš wrote:
> >
> >
> > > -----Original Message----- From: Bruce Richardson
> > > <bruce.richardson at intel.com> Sent: Friday, August 28, 2020 12:30 PM To:
> > > Juraj Linkeš <juraj.linkes at pantheon.tech> Cc: dev at dpdk.org Subject: Re:
> > > [dpdk-dev] DPDK 20.08 meson build testpmd not detecting all NUMA
> > > nodes
> > >
> > > On Fri, Aug 28, 2020 at 11:20:02AM +0100, Bruce Richardson wrote:
> > > > On Fri, Aug 28, 2020 at 09:26:12AM +0000, Juraj Linkeš wrote:
> > > > > Hello DPDK devs,
> > > > >
> > > > > I've meson built (using just meson build && ninja -C build)
> > > > > dpdk-20.08
> > > (downloaded from the site) on a Taishan arm server and dpdk-testpmd
> > > is incorrectly detecting NUMA nodes on the system:
> > > > > EAL: Detected 16 lcore(s) EAL: Detected 1 NUMA nodes
> > > > >
> > > > > lscpu report 4 NUMA nodes: NUMA node0 CPU(s):   0-15 NUMA
> node1
> > > > > CPU(s):   16-31 NUMA node2 CPU(s):   32-47 NUMA node3 CPU(s):
> > > > > 48-63
> > > > >
> > > > > When built with make T=arm64-armv8a-linuxapp-gcc testpmd is
> > > > > detecting
> > > NUMA nodes correctly:
> > > > > EAL: Detected 64 lcore(s) EAL: Detected 4 NUMA nodes
> > > > >
> > > > > Any idea what's the difference between the builds? I've looked
> > > > > at NUMA
> > > config options but those seem ok to me:
> > > > > grep NUMA arm64-armv8a-linuxapp-gcc/.config
> > > > > build/rte_build_config.h
> > > > > arm64-armv8a-linuxapp-
> gcc/.config:CONFIG_RTE_MAX_NUMA_NODES=8
> > > > > arm64-armv8a-linuxapp-gcc/.config:# a "type" is a combination of
> > > > > page size and NUMA node. total number of memseg
> > > > > arm64-armv8a-linuxapp-
> > > gcc/.config:CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGE
> > > > > S=y arm64-armv8a-linuxapp-
> > > gcc/.config:CONFIG_RTE_LIBRTE_VHOST_NUMA=y
> > > > > build/rte_build_config.h:#define
> RTE_EAL_NUMA_AWARE_HUGEPAGES 1
> > > > > build/rte_build_config.h:#define RTE_HAS_LIBNUMA 1
> > > > > build/rte_build_config.h:#define RTE_LIBRTE_VHOST_NUMA 1
> > > > > build/rte_build_config.h:#define RTE_MAX_NUMA_NODES 4
> > > > >
> > > > > Any pointers are welcome.
> > > > >
> > > > What is the max lcores value in your build config? Do you know
> > > > what settings are being chosen for your platform in
> > > > config/arm/meson.build?
> > > >
> > > By way of follow-up, the reason to look at the cores is that the
> > > number of numa nodes is detected simply by iterating through all the
> > > cores discovered and finding out what NUMA node they are on. If only
> > > 16 cores are allowed, then only one NUMA node will be found because
> > > all cores discovered will report that as their NUMA location.
> > >
> > > /Bruce
> >
> > Thanks Bruce, RTE_MAX_LCORE is being set only to 16 (line 39 in
> > config/arm/meson.build), so that's the cause. Why is this used in
> > native builds? Shouldn't this be discovered?

There is a patch set from Dharmik that detects lcore count for native builds:
http://patches.dpdk.org/project/dpdk/list/?series=11784
It will be good if you can try and test it.

Thanks.
/Ruifeng
> >
> > I any case, setting it statically for implementer 0x41 seems wrong, as
> > there's a big variety different processors under it:
> > https://github.com/bp0/armids/blob/master/arm.ids
> >
> Sorry, no idea about this, since I don't generally work on/with the
> arm/meson.build file. Looking at git, that particular value seems to come
> from commit d97108a33231 ("config: change defaults of armv8"), so perhaps
> check with the author and reviewer of that patch. [Adding them on CC]
> 
> Regards,
> /Bruce