[PATCH v7] enhance NUMA affinity heuristic

[Burakov, Anatoly]

On 5/23/2023 3:50 AM, Kaisen You wrote:
> When a DPDK application is started on only one numa node, memory is
> allocated for only one socket. When interrupt threads use memory,
> memory may not be found on the socket where the interrupt thread
> is currently located, and memory has to be reallocated on the hugepage,
> this operation will lead to performance degradation.
> 
> Fixes: 705356f0811f ("eal: simplify control thread creation")
> Fixes: 770d41bf3309 ("malloc: fix allocation with unknown socket ID")
> Cc: stable at dpdk.org
> 
> Signed-off-by: Kaisen You <kaisenx.you at intel.com>
> ---

For the record, I still think that this is a solution for a problem that 
should be fixed elsewhere, because a DPDK lcore (even main lcore!) 
having a specific NUMA node affinity is one of the most fundamental 
assumptions about DPDK, and I feel like we're inviting problems if we 
allow lcores to have multiple NUMA node affinities.

For example, if I run DPDK test app with the following command-line:

--lcores "1@(1,29),2@(30)"

The malloc autotest will fail because main lcore now returns -1 when 
we're calling `rte_socket_id()` from it. Correspondigly, any API's that 
use `rte_socket_id()` internally for various purposes (especially 
indexing arrays!) will now have to account for the fact that 
`rte_socket_id()` can just return -1 and it is not an exceptional situation.

IMO if we want to keep this behavior, EAL should at least warn the user 
that a DPDK lcore was assigned SOCKET_ID_ANY on account of multiple NUMA 
nodes being in its cpuset. So, as an unrealted change (so, i'm not 
suggesting doing it in this specific patchset), I would suggest that 
`thread_update_affinity()` should warn about DPDK lcore being assigned 
socket ID like that.

-- 
Thanks,
Anatoly