[dpdk-dev] [PATCH v5 0/2] Balanced allocation of hugepages

[Ilya Maximets]

On 21.06.2017 11:25, Sergio Gonzalez Monroy wrote:
> On 21/06/2017 09:14, Hemant Agrawal wrote:
>> On 6/20/2017 9:11 PM, Jerin Jacob wrote:
>>> -----Original Message-----
>>>> Date: Tue, 20 Jun 2017 15:58:50 +0100
>>>> From: Sergio Gonzalez Monroy <sergio.gonzalez.monroy at intel.com>
>>>> To: Thomas Monjalon <thomas at monjalon.net>, Ilya Maximets
>>>>  <i.maximets at samsung.com>
>>>> CC: dev at dpdk.org, Hemant Agrawal <hemant.agrawal at nxp.com>, Bruce Richardson
>>>>  <bruce.richardson at intel.com>, David Marchand <david.marchand at 6wind.com>,
>>>>  Heetae Ahn <heetae82.ahn at samsung.com>, Yuanhan Liu <yliu at fridaylinux.org>,
>>>>  Jianfeng Tan <jianfeng.tan at intel.com>, Neil Horman
>>>>  <nhorman at tuxdriver.com>, Yulong Pei <yulong.pei at intel.com>
>>>> Subject: Re: [dpdk-dev] [PATCH v5 0/2] Balanced allocation of hugepages
>>>> User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101
>>>>  Thunderbird/45.1.1
>>>>
>>>> On 20/06/2017 15:35, Thomas Monjalon wrote:
>>>>> 20/06/2017 15:58, Ilya Maximets:
>>>>>> On 20.06.2017 16:07, Thomas Monjalon wrote:
>>>>>>> 19/06/2017 13:10, Hemant Agrawal:
>>>>>>>>>>> On Thu, Jun 08, 2017 at 02:21:58PM +0300, Ilya Maximets wrote:
>>>>>>>>>>>> So, there are 2 option:
>>>>>>>>>>>>
>>>>>>>>>>>>      1. Return back config option RTE_LIBRTE_EAL_NUMA_AWARE_HUGEPAGES
>>>>>>>>>>>>         from the first version of the patch and disable it by default.
>>>>>>>>>>>>
>>>>>>>>>>>>      2. Keep patch as it is now and make everyone install libnuma
>>>>>>>>>>>>         for successful build.
>>>>>>>> +1 for option 1
>>>>>>>> It will be a issue and undesired dependency for SoCs, not supporting
>>>>>>>> NUMA architecture.
>>>>>>>>
>>>>>>>> It can be added to the config, who desired to use it by default.
>>>>>>> Yes I agree, it cannot be a dependency for architectures which
>>>>>>> do not support NUMA.
>>>>>>> Please can we rework the patch so that only one node is assumed
>>>>>>> if NUMA is disabled for the architecture?
>>>>
>>>> Ilya, I missed that libnuma is not supported on ARM.
>>>
>>> It is supported on arm64 and arm64 has NUMA machines(thunderx, thunderx2) too.
>>>
>>> [dpdk.org] $ dpkg-query -L libnuma-dev
>>> /.
>>> /usr
>>> /usr/lib
>>> /usr/lib/aarch64-linux-gnu
>>> /usr/lib/aarch64-linux-gnu/libnuma.a
>>> /usr/share
>>> /usr/share/man
>>> /usr/share/man/man3
>>> /usr/share/man/man3/numa.3.gz
>>> /usr/share/doc
>>> /usr/share/doc/libnuma-dev
>>> /usr/share/doc/libnuma-dev/copyright
>>> /usr/include
>>> /usr/include/numaif.h
>>> /usr/include/numa.h
>>> /usr/include/numacompat1.h
>>> /usr/lib/aarch64-linux-gnu/libnuma.so
>>>
>>
>> 1. There are many machines (arm/ppc), which do not support NUMA.
>>
>> https://wiki.linaro.org/LEG/Engineering/Kernel/NUMA
>>
> 
> I did find that link too, last modified 4 years ago.
> Despite that, I could not find any ARM references in libnuma sources, but Jerin proved that there is support for it.
> 
> http://oss.sgi.com/projects/libnuma/
> https://github.com/numactl/numactl
> 
>> 2. I could not locate it by default in Linaro toolchains.
>>
>> 3.  Since this is not a common across all platform. This option should not be added to the common_base or common configs. It can be added to any architecture configuration, which needs it.
>>
> 
> So is it thunderx the only arm64 to enable this feature by default?
> I thought the dependency was the libnuma library support itself.

ARMv7 is the only architecture without libnuma package in common distros.
So, in v6 I enabled this feature by default for x86, ppc and thunderx.
I didn't enable it for the whole ARMv8 just because thunderx is the only
platform which supports NUMA and has special defconfig in DPDK repository.

Best regards, Ilya Maximets.

>>>>
>>>>>> We're still don't have dynamic build time configuration system.
>>>>>> To make get/set_mempolicy work we need to include <numaif.h>
>>>>>> and have libnuma for successful linkage.
>>>>>> This means that the only option to not have libnuma as dependency
>>>>>> is to return back configuration option RTE_LIBRTE_EAL_NUMA_AWARE_HUGEPAGES
>>>>>> as it was in the first version of the patch.
>>>>>>
>>>>>> There is, actually, the third option (besides 2 already described):
>>>>>>
>>>>>>     3. Return back config option RTE_LIBRTE_EAL_NUMA_AWARE_HUGEPAGES
>>>>>>        from the first version of the patch and *enable* it by default.
>>>>>>        In this case anyone who doesn't want to have libnuma as dependency
>>>>>>        will be able to disable the config option manually.
>>>>>>
>>>>>> Thomas, what do you think? Bruce? Sergio?
>>>>> It should be enabled on x86 and ppc, and disabled in other
>>>>> default configurations (ARM for now).
>>>>
>>>> Agree.
>>>>
>>>>>> P.S. We're always able to implement syscall wrappers by hands without any
>>>>>>       external dependencies, but I don't think it's a good decision.
>>>>> I agree to use libnuma instead of re-inventing the wheel.
>>>>> Let's just make it optional at build time and fallback on one node
>>>>> if disabled.
>>>>
>>>> That is the simple way out.
>>>>
>>>> Sergio
>>>
>>
>>
> 
> 
> 
>