[PATCH] net/iavf:fix slow memory allocation

[David Marchand]

On Tue, Dec 20, 2022 at 7:52 AM You, KaisenX <kaisenx.you at intel.com> wrote:
> > >> As to the reason for not using rte_malloc_socket. I thought
> > >> rte_malloc_socket() could solve the problem too. And the appropriate
> > >> parameter should be the socket_id that created the memory pool for
> > >> DPDK initialization. Assuming that> the socket_id of the initially
> > >> allocated memory = 1, first let the
> > > eal_intr_thread
> > >> determine if it is on the socket_id, then record this socket_id in
> > >> the eal_intr_thread and pass it to the iavf_event_thread.  But there
> > >> seems no way to link this parameter to the iavf_dev_event_post()
> > function. That is why rte_malloc_socket is not used.
> > >>
> > >
> > > I was thinking socket id of device can be used, but that won't help if
> > > the core that interrupt handler runs is in different socket.
> > > And I also don't know if there is a way to get socket that interrupt
> > > thread is on. @David may help perhaps.
> > >
> > > So question is why interrupt thread is not running on main lcore.
> > >
> >
> > OK after some talk with David, what I am missing is 'rte_ctrl_thread_create()'
> > does NOT run on main lcore, it can run on any core except data plane cores.
> >
> > Driver "iavf-event-thread" thread (iavf_dev_event_handle()) and interrupt
> > thread (so driver interrupt callback iavf_dev_event_post()) can run on any
> > core, making it hard to manage.
> > And it seems it is not possible to control where interrupt thread to run.
> >
> > One option can be allocating hugepages for all sockets, but this requires user
> > involvement, and can't happen transparently.
> >
> > Other option can be to control where "iavf-event-thread" run, like using
> > 'rte_thread_create()' to create thread and provide attribute to run it on main
> > lcore (rte_lcore_cpuset(rte_get_main_lcore()))?
> >
> > Can you please test above option?
> >
> >
> The first option can solve this issue. but to borrow from your previous saying,
> "in a dual socket system, if all used cores are in socket 1 and the NIC is in socket 1,
>  no memory is allocated for socket 0.  This is to optimize memory consumption."
> I think it's unreasonable to do so.
>
> About other option. In " rte_eal_intr_init" function, After the thread is created,
> I set the thread affinity for eal-intr-thread, but it does not solve this issue.

Jumping in this thread.

I tried to play a bit with a E810 nic on a dual numa and I can't see
anything wrong for now.
Can you provide a simple and small reproducer of your issue?

Thanks.

-- 
David Marchand