[PATCH v5] enhance NUMA affinity heuristic
Burakov, Anatoly
anatoly.burakov at intel.com
Wed Feb 15 15:22:39 CET 2023
On 2/1/2023 12:20 PM, Kaisen You wrote:
> Trying to allocate memory on the first detected numa node has less
> chance to find some memory actually available rather than on the main
> lcore numa node (especially when the DPDK application is started only
> on one numa node).
>
> Fixes: 705356f0811f ("eal: simplify control thread creation")
> Fixes: bb0bd346d5c1 ("eal: suggest using --lcores option")
> Cc: stable at dpdk.org
>
> Signed-off-by: David Marchand <david.marchand at redhat.com>
> Signed-off-by: Kaisen You <kaisenx.you at intel.com>
> ---
> Changes since v4:
> - mod the patch title,
>
> Changes since v3:
> - add the assignment of socket_id in thread initialization,
>
> Changes since v2:
> - add uncommitted local change and fix compilation,
>
> Changes since v1:
> - accomodate for configurations with main lcore running on multiples
> physical cores belonging to different numa,
> ---
> lib/eal/common/eal_common_thread.c | 1 +
> lib/eal/common/malloc_heap.c | 4 ++++
> 2 files changed, 5 insertions(+)
>
> diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c
> index 38d83a6885..21bff971f8 100644
> --- a/lib/eal/common/eal_common_thread.c
> +++ b/lib/eal/common/eal_common_thread.c
> @@ -251,6 +251,7 @@ static void *ctrl_thread_init(void *arg)
> void *routine_arg = params->arg;
>
> __rte_thread_init(rte_lcore_id(), cpuset);
> + RTE_PER_LCORE(_socket_id) = SOCKET_ID_ANY;
> params->ret = rte_thread_set_affinity_by_id(rte_thread_self(), cpuset);
> if (params->ret != 0) {
> __atomic_store_n(¶ms->ctrl_thread_status,
> diff --git a/lib/eal/common/malloc_heap.c b/lib/eal/common/malloc_heap.c
> index d7c410b786..3ee19aee15 100644
> --- a/lib/eal/common/malloc_heap.c
> +++ b/lib/eal/common/malloc_heap.c
> @@ -717,6 +717,10 @@ malloc_get_numa_socket(void)
> return socket_id;
> }
>
> + socket_id = rte_lcore_to_socket_id(rte_get_main_lcore());
> + if (socket_id != (unsigned int)SOCKET_ID_ANY)
> + return socket_id;
> +
> return rte_socket_id_by_idx(0);
> }
>
I may be lacking context, but I don't quite get the suggested change.
From what I understand, the original has to do with assigning lcore
cpusets in such a way that an lcore ends up having two socket ID's
(because it's been assigned to CPU's on different sockets). Why is this
allowed in the first place? It seems like a user error to me, as it
breaks many of the fundamental assumptions DPDK makes.
I'm fine with using main lcore socket for control threads, I just don't
think the `socket_id != SOCKET_ID_ANY` thing should be checked here,
because it apparently tries to compensate for a problem with cpuset of
the main thread, which shouldn't have happened to begin with.
--
Thanks,
Anatoly
More information about the stable
mailing list