[dpdk-dev] [RFC] ethdev: change queue release callback

[Xueming(Steven) Li]

> -----Original Message-----
> From: dev <dev-bounces at dpdk.org> On Behalf Of Xueming(Steven) Li
> Sent: Wednesday, August 11, 2021 8:13 PM
> To: Ferruh Yigit <ferruh.yigit at intel.com>; Singh, Aman Deep <aman.deep.singh at intel.com>; Andrew Rybchenko
> <andrew.rybchenko at oktetlabs.ru>
> Cc: dev at dpdk.org; Slava Ovsiienko <viacheslavo at nvidia.com>; NBU-Contact-Thomas Monjalon <thomas at monjalon.net>;
> jerinj at marvell.com
> Subject: Re: [dpdk-dev] [RFC] ethdev: change queue release callback
> 
> 
> 
> > -----Original Message-----
> > From: Ferruh Yigit <ferruh.yigit at intel.com>
> > Sent: Wednesday, August 11, 2021 7:58 PM
> > To: Xueming(Steven) Li <xuemingl at nvidia.com>; Singh, Aman Deep
> > <aman.deep.singh at intel.com>; Andrew Rybchenko
> > <andrew.rybchenko at oktetlabs.ru>
> > Cc: dev at dpdk.org; Slava Ovsiienko <viacheslavo at nvidia.com>;
> > NBU-Contact-Thomas Monjalon <thomas at monjalon.net>; jerinj at marvell.com
> > Subject: Re: [dpdk-dev] [RFC] ethdev: change queue release callback
> >
> > On 8/10/2021 10:07 AM, Xueming(Steven) Li wrote:
> > >
> > >
> > >> -----Original Message-----
> > >> From: Ferruh Yigit <ferruh.yigit at intel.com>
> > >> Sent: Tuesday, August 10, 2021 4:54 PM
> > >> To: Xueming(Steven) Li <xuemingl at nvidia.com>; Singh, Aman Deep
> > >> <aman.deep.singh at intel.com>; Andrew Rybchenko
> > >> <andrew.rybchenko at oktetlabs.ru>
> > >> Cc: dev at dpdk.org; Slava Ovsiienko <viacheslavo at nvidia.com>;
> > >> NBU-Contact-Thomas Monjalon <thomas at monjalon.net>
> > >> Subject: Re: [dpdk-dev] [RFC] ethdev: change queue release callback
> > >>
> > >> On 8/10/2021 9:03 AM, Xueming(Steven) Li wrote:
> > >>> Hi Singh and Ferruh,
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: Ferruh Yigit <ferruh.yigit at intel.com>
> > >>>> Sent: Monday, August 9, 2021 11:31 PM
> > >>>> To: Singh, Aman Deep <aman.deep.singh at intel.com>; Andrew
> > >>>> Rybchenko <andrew.rybchenko at oktetlabs.ru>; Xueming(Steven) Li
> > >>>> <xuemingl at nvidia.com>
> > >>>> Cc: dev at dpdk.org; Slava Ovsiienko <viacheslavo at nvidia.com>;
> > >>>> NBU-Contact-Thomas Monjalon <thomas at monjalon.net>
> > >>>> Subject: Re: [dpdk-dev] [RFC] ethdev: change queue release
> > >>>> callback
> > >>>>
> > >>>> On 8/9/2021 3:39 PM, Singh, Aman Deep wrote:
> > >>>>> Hi Xueming,
> > >>>>>
> > >>>>> On 7/28/2021 1:10 PM, Andrew Rybchenko wrote:
> > >>>>>> On 7/27/21 6:41 AM, Xueming Li wrote:
> > >>>>>>> To align with other eth device queue configuration callbacks,
> > >>>>>>> change RX and TX queue release callback API parameter from
> > >>>>>>> queue object to device and queue index.
> > >>>>>>>
> > >>>>>>> Signed-off-by: Xueming Li <xuemingl at nvidia.com>
> > >>>>>>
> > >>>>>> In fact, there is no strong reasons to do it, but I think it is
> > >>>>>> a nice cleanup to use (dev + queue index) on control path.
> > >>>>>>
> > >>>>>> Hopefully it will not result in any regressions.
> > >>>>>
> > >>>>> Combined there are 100+ API's for Rx/Tx queue_release that need
> > >>>>> to be modified for it.
> > >>>>>
> > >>>>> I believe all regression possibilities here will be caught, in
> > >>>>> compilation phase itself.
> > >>>>>
> > >>>>
> > >>>> Same here, it is a good cleanup but there is no strong reason for it.
> > >>>>
> > >>>> Since it is all internal, there is no ABI restriction on the
> > >>>> patch, and v21.11 will be full ABI break patches, to not cause conflicts with this change, what would you think to have it on
> v22.02?
> > >>>
> > >>> This patch is required by shared-rxq feature which ABI broken, target to 21.11.
> > >>
> > >> Why it is required?
> > >
> > > In rx burst function, rxq object is used in data path. For best data performance, it's shared-rxq object in case of shared rxq enabled.
> > > I think eth api defined rxq object for performance as well, specific on data plane.
> > > Hardware saves port info received packet descriptor for my case.
> > > Can't tell which device's queue with this shared rxq object, control
> > > path can't use this shared rxq anymore, have to be specific on
> > dev and queue id.
> > >
> >
> > I have seen shared Rx queue patch, but that just introduces the
> > offload and doesn't have the PMD implementation, so hard to see the dependency, can you please put the pseudocode for PMDs
> for shared-rxq?
> 
> The code is almost ready, I'll upload the PMD part soon.

Seems lots of PMD conflicts to rebase, have to hold it due to other urgent tasks. Here is the overall data structure:
Struct mlx5_rxq_ctrl {
	Bool shared;
	LIST_HEAD owners; // owner rxq(s)
	// datapath resources
}
Struct mlx5_rxq_priv { // rx queue 
	U16 queue_index;
	LIST_ENTRY owner_entry; // membership in shared rxq
	Struct mlx5_rxq_ctrl *ctrl; // save to dev->data->rx_queues[]
	// other per queue resources
}
Rxq_ctrl could be 1:1 mapping to rxq_priv in case of standard rxq, 1:N in case of shared
Shared rxq_ctrl will be released till last owner rxq_priv released.

BTW, v1 posted, please check.

> But firstly, I'll upload v1 patch for this RFC, the make PMD patches depends on this v1 patch.
> 
> > How a queue will know if it is shared or not, during release?
> 
> That's why this RFC want to change callback parameter to device and queue id.
> There is an offloading flag during rxq setup, either in device or in queue configuration.
> PMD driver saves the flag and operate accordingly.
> Ethdev api doesn't need to save this, unless a solid reason.
> 
> >
> > Btw, shared Rx doesn't mention from this dependency in the patch.
> 
> Agree, indeed a strong dependency, thanks!
> 
> >
> > >>
> > >>> I'll do it carefully, fortunately, the change is straightforward.
> > >>>
> > >