[v2,2/4] hash: add local cache for TSX region

  From: Yipeng Wang <yipeng1.wang@intel.com>

This patch adds back the local cache when TSX support is turned on.

When TSX is turned on, free key-data slot ring would be contended by
various TSX regions. The purpose of this commit is to reduce possible
memory collisions during key insertion.

Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 lib/librte_hash/rte_cuckoo_hash.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
  

> 
> From: Yipeng Wang <yipeng1.wang@intel.com>
> 
> This patch adds back the local cache when TSX support is turned on.
> 
> When TSX is turned on, free key-data slot ring would be contended by various
> TSX regions. The purpose of this commit is to reduce possible memory
> collisions during key insertion.
> 
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  lib/librte_hash/rte_cuckoo_hash.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/librte_hash/rte_cuckoo_hash.c
> b/lib/librte_hash/rte_cuckoo_hash.c
> index 76f5dc8..5d5f9f1 100644
> --- a/lib/librte_hash/rte_cuckoo_hash.c
> +++ b/lib/librte_hash/rte_cuckoo_hash.c
> @@ -180,8 +180,10 @@ struct rte_hash *
>  	}
> 
>  	/* Check extra flags field to check extra options. */
> -	if (params->extra_flag &
> RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT)
> +	if (params->extra_flag &
> RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT) {
> +		use_local_cache = 1;
Do you see the issue even in the case of single writer? Enabling this flag creates local caches on all the data plane cores. This increases the memory usage for the single writer use case. Then there is 'writers on the control plane' use case, the requirement on hash_add rate is comparatively lower when compared to 'writers on the data plane'. The writers also are not pinned to any core as well. In this use case, I am not sure how much having a local cache matters.

Enabling this flag effectively changes the free slot allocation from a ring to a stack data structure. Does it indicate that for single writer use case with TSX, the free slot (global) data structure should be a stack (rather than a ring)?
 
>  		hw_trans_mem_support = 1;
> +	}
> 
>  	if (params->extra_flag &
> RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) {
>  		use_local_cache = 1;
> --
> 1.8.5.6
  

12/11/2018 19:34, Honnappa Nagarahalli:
> > 
> > From: Yipeng Wang <yipeng1.wang@intel.com>
> > 
> > This patch adds back the local cache when TSX support is turned on.
> > 
> > When TSX is turned on, free key-data slot ring would be contended by various
> > TSX regions. The purpose of this commit is to reduce possible memory
> > collisions during key insertion.
> > 
> > Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> > Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> >  lib/librte_hash/rte_cuckoo_hash.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/lib/librte_hash/rte_cuckoo_hash.c
> > b/lib/librte_hash/rte_cuckoo_hash.c
> > index 76f5dc8..5d5f9f1 100644
> > --- a/lib/librte_hash/rte_cuckoo_hash.c
> > +++ b/lib/librte_hash/rte_cuckoo_hash.c
> > @@ -180,8 +180,10 @@ struct rte_hash *
> >  	}
> > 
> >  	/* Check extra flags field to check extra options. */
> > -	if (params->extra_flag &
> > RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT)
> > +	if (params->extra_flag &
> > RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT) {
> > +		use_local_cache = 1;
> Do you see the issue even in the case of single writer? Enabling this flag creates local caches on all the data plane cores. This increases the memory usage for the single writer use case. Then there is 'writers on the control plane' use case, the requirement on hash_add rate is comparatively lower when compared to 'writers on the data plane'. The writers also are not pinned to any core as well. In this use case, I am not sure how much having a local cache matters.
> 
> Enabling this flag effectively changes the free slot allocation from a ring to a stack data structure. Does it indicate that for single writer use case with TSX, the free slot (global) data structure should be a stack (rather than a ring)?

Is it blocking this patchset from entering in 18.11?
If I understand well, there are some fixes for 18.11.
  

> > >
> > > From: Yipeng Wang <yipeng1.wang@intel.com>
> > >
> > > This patch adds back the local cache when TSX support is turned on.
> > >
> > > When TSX is turned on, free key-data slot ring would be contended by
> > > various TSX regions. The purpose of this commit is to reduce
> > > possible memory collisions during key insertion.
> > >
> > > Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> > > Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> > > ---
> > >  lib/librte_hash/rte_cuckoo_hash.c | 4 +++-
> > >  1 file changed, 3 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/lib/librte_hash/rte_cuckoo_hash.c
> > > b/lib/librte_hash/rte_cuckoo_hash.c
> > > index 76f5dc8..5d5f9f1 100644
> > > --- a/lib/librte_hash/rte_cuckoo_hash.c
> > > +++ b/lib/librte_hash/rte_cuckoo_hash.c
> > > @@ -180,8 +180,10 @@ struct rte_hash *
> > >  	}
> > >
> > >  	/* Check extra flags field to check extra options. */
> > > -	if (params->extra_flag &
> > > RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT)
> > > +	if (params->extra_flag &
> > > RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT) {
> > > +		use_local_cache = 1;
> > Do you see the issue even in the case of single writer? Enabling this flag
> creates local caches on all the data plane cores. This increases the memory
> usage for the single writer use case. Then there is 'writers on the control
> plane' use case, the requirement on hash_add rate is comparatively lower
> when compared to 'writers on the data plane'. The writers also are not pinned
> to any core as well. In this use case, I am not sure how much having a local
> cache matters.
> >
> > Enabling this flag effectively changes the free slot allocation from a ring to a
> stack data structure. Does it indicate that for single writer use case with TSX,
> the free slot (global) data structure should be a stack (rather than a ring)?
> 
> Is it blocking this patchset from entering in 18.11?
> If I understand well, there are some fixes for 18.11.
> 
I am fine with the other fixes in this patchset
  

>-----Original Message-----
>From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
>Sent: Tuesday, November 13, 2018 9:17 AM
>To: Thomas Monjalon <thomas@monjalon.net>; Richardson, Bruce <bruce.richardson@intel.com>; Wang, Yipeng1
><yipeng1.wang@intel.com>
>Cc: stable@dpdk.org; dev@dpdk.org; nd <nd@arm.com>; nd <nd@arm.com>
>Subject: RE: [dpdk-stable] [dpdk-dev] [PATCH v2 2/4] hash: add local cache for TSX region
>> > >  	/* Check extra flags field to check extra options. */
>> > > -	if (params->extra_flag &
>> > > RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT)
>> > > +	if (params->extra_flag &
>> > > RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT) {
>> > > +		use_local_cache = 1;
>> > Do you see the issue even in the case of single writer? Enabling this flag
>> creates local caches on all the data plane cores. This increases the memory
>> usage for the single writer use case. Then there is 'writers on the control
>> plane' use case, the requirement on hash_add rate is comparatively lower
>> when compared to 'writers on the data plane'. The writers also are not pinned
>> to any core as well. In this use case, I am not sure how much having a local
>> cache matters.
>> >
>> > Enabling this flag effectively changes the free slot allocation from a ring to a
>> stack data structure. Does it indicate that for single writer use case with TSX,
>> the free slot (global) data structure should be a stack (rather than a ring)?
>>
>> Is it blocking this patchset from entering in 18.11?
>> If I understand well, there are some fixes for 18.11.

[Wang, Yipeng] Hi Thomas, please go ahead merge the other commits without this one since Honnapa's concern.
I will talk with Honnappa separately on a better way to do this.

Thanks!
>>
>I am fine with the other fixes in this patchset
  

On Tue, Nov 13, 2018 at 05:24:55PM +0000, Wang, Yipeng1 wrote:
> >-----Original Message-----
> >From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
> >Sent: Tuesday, November 13, 2018 9:17 AM
> >To: Thomas Monjalon <thomas@monjalon.net>; Richardson, Bruce <bruce.richardson@intel.com>; Wang, Yipeng1
> ><yipeng1.wang@intel.com>
> >Cc: stable@dpdk.org; dev@dpdk.org; nd <nd@arm.com>; nd <nd@arm.com>
> >Subject: RE: [dpdk-stable] [dpdk-dev] [PATCH v2 2/4] hash: add local cache for TSX region
> >> > >  	/* Check extra flags field to check extra options. */
> >> > > -	if (params->extra_flag &
> >> > > RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT)
> >> > > +	if (params->extra_flag &
> >> > > RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT) {
> >> > > +		use_local_cache = 1;
> >> > Do you see the issue even in the case of single writer? Enabling this flag
> >> creates local caches on all the data plane cores. This increases the memory
> >> usage for the single writer use case. Then there is 'writers on the control
> >> plane' use case, the requirement on hash_add rate is comparatively lower
> >> when compared to 'writers on the data plane'. The writers also are not pinned
> >> to any core as well. In this use case, I am not sure how much having a local
> >> cache matters.
> >> >
> >> > Enabling this flag effectively changes the free slot allocation from a ring to a
> >> stack data structure. Does it indicate that for single writer use case with TSX,
> >> the free slot (global) data structure should be a stack (rather than a ring)?
> >>
> >> Is it blocking this patchset from entering in 18.11?
> >> If I understand well, there are some fixes for 18.11.
> 
> [Wang, Yipeng] Hi Thomas, please go ahead merge the other commits without this one since Honnapa's concern.
> I will talk with Honnappa separately on a better way to do this.
> 
> Thanks!

No objections to that plan here.

/Bruce