[dpdk-dev] [PATCH] parray: introduce internal API for dynamic arrays

[Thomas Monjalon]

15/06/2021 10:44, Bruce Richardson:
> On Tue, Jun 15, 2021 at 09:53:33AM +0200, Morten Brørup wrote:
> > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Thomas Monjalon
> > > Sent: Tuesday, 15 June 2021 08.48
> > > 
> > > 14/06/2021 17:48, Morten Brørup:
> > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Thomas
> > > Monjalon
> > > > It would be much simpler to just increase RTE_MAX_ETHPORTS to
> > > something big enough to hold a sufficiently large array. And possibly
> > > add an rte_max_ethports variable to indicate the number of populated
> > > entries in the array, for use when iterating over the array.
> > > >
> > > > Can we come up with another example than RTE_MAX_ETHPORTS where this
> > > library provides a better benefit?
> > > 
> > > What is big enough?
> > > Is 640KB enough for RAM? ;)
> > 
> > Good point!
> > 
> > I think we agree that:
> > - The cost of this library is some added complexity, i.e. working with a dynamically sized array through a library instead of just indexing into a compile time fixed size array.
> > - The main benefit of this library is saving some RAM (and still allowing a potentially very high number of ports.)
> > 
> > My point was: The amount of RAM we are saving is a key parameter for the cost/benefit analysis. And since I don't think the rte_eth_devices[] array uses a significant amount of memory, I was asking for some other array using more memory, where the cost/benefit analysis would come out more advantageous to your proposed parray library.
> > 
> > > 
> > > When dealing with microservices switching, the numbers can increase
> > > very fast.
> > 
> > Yes, I strongly supported increasing the port_id type from 8 to 16 bits for this reason, when it was discussed at the DPDK Userspace a few years ago in Dublin. And with large RTE_MAX_QUEUES_PER_PORT values, the rte_eth_dev structure uses quite a lot of space for the rx/tx callback arrays. But the memory usage of rte_eth_devices[] is still relatively insignificant in a system wide context.
> > 
> > If main purpose is to optimize the rte_eth_devices[] array, I think there are better alternatives than this library. Bruce and Konstantin already threw a few ideas on the table.
> >
> 
> Yes, though I think we need to be clear on what problems we are trying to
> solve here. A generic resizable array may be a useful library for DPDK in
> its own right, but for the ethdev (and other devs) arrays I think my
> understanding of the problem is that we want:
> 
> * scalability of ethdevs list to large numbers of ports, e.g. 2k
> * while not paying a large memory footprint penalty for those apps which
>   only need a small number of ports, e.g. 2 or 4.
> 
> Is that a fair summary?

Yes.

We must take into account two related issues:
	- the app and libs could allocate some data per device,
increasing the bill.
	- per-device allocation may be more efficient
if allocated on the NUMA node of the device