[dpdk-dev,v4] mbuf: fix mbuf free performance with non atomic refcnt

  When RTE_MBUF_REFCNT_ATOMIC=n, the decrement of the mbuf reference
counter uses an atomic operation. This is not necessary and impacts
the performance (seen with TRex traffic generator).

We cannot replace rte_atomic16_add_return() by rte_mbuf_refcnt_update()
because it would add an additional check.

Solves this by introducing __rte_mbuf_refcnt_update(), which
updates the reference counter without doing anything else.

Fixes: 8f094a9ac5d7 ("mbuf: set mbuf fields while in pool")
Suggested-by: Hanoch Haim <hhaim@cisco.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---

Hanoh,

The following patch implements what was discussed in the thread.
Are you ok with it?

Thanks,
Olivier


 lib/librte_mbuf/rte_mbuf.h | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
  

Olivier,

> On Dec 8, 2017, at 7:46 PM, Olivier Matz <olivier.matz@6wind.com> wrote:
> 

> 
> lib/librte_mbuf/rte_mbuf.h | 23 ++++++++++++++++++-----
> 1 file changed, 18 insertions(+), 5 deletions(-)
> 
> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> index ce8a05ddf..dd08cb72b 100644
> --- a/lib/librte_mbuf/rte_mbuf.h
> +++ b/lib/librte_mbuf/rte_mbuf.h
> @@ -764,6 +764,13 @@ rte_mbuf_refcnt_set(struct rte_mbuf *m, uint16_t new_value)
> 	rte_atomic16_set(&m->refcnt_atomic, new_value);
> }
> 
> +/* internal */
> +static inline uint16_t
> +__rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> +{
> +	return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value));

What’s the purpose of using direct cast to uint16_t here and in other places?

> +}
> +
> /**
>  * Adds given value to an mbuf's refcnt and returns its new value.
>  * @param m
> @@ -788,19 +795,26 @@ rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> 		return 1 + value;
> 	}
> 
> -	return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value));
> +	return __rte_mbuf_refcnt_update(m, value);
> }
> 
> #else /* ! RTE_MBUF_REFCNT_ATOMIC */
> 
> +/* internal */
> +static inline uint16_t
> +__rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> +{
> +	m->refcnt = (uint16_t)(m->refcnt + value);
> +	return m->refcnt;
> +}
> +
> /**
>  * Adds given value to an mbuf's refcnt and returns its new value.
>  */
> static inline uint16_t
> rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> {
> -	m->refcnt = (uint16_t)(m->refcnt + value);
> -	return m->refcnt;
> +	return __rte_mbuf_refcnt_update(m, value);
> }
> 
> /**
> @@ -1364,8 +1378,7 @@ rte_pktmbuf_prefree_seg(struct rte_mbuf *m)
> 
> 		return m;
> 
> -       } else if (rte_atomic16_add_return(&m->refcnt_atomic, -1) == 0) {
> -
> +	} else if (__rte_mbuf_refcnt_update(m, -1) == 0) {
> 
> 		if (RTE_MBUF_INDIRECT(m))
> 			rte_pktmbuf_detach(m);
> -- 
> 2.11.0
>
  

On Fri, Dec 08, 2017 at 08:04:50PM +0400, Ilya Matveychikov wrote:
> Olivier,
> 
> > On Dec 8, 2017, at 7:46 PM, Olivier Matz <olivier.matz@6wind.com> wrote:
> > 
> 
> > 
> > lib/librte_mbuf/rte_mbuf.h | 23 ++++++++++++++++++-----
> > 1 file changed, 18 insertions(+), 5 deletions(-)
> > 
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > index ce8a05ddf..dd08cb72b 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -764,6 +764,13 @@ rte_mbuf_refcnt_set(struct rte_mbuf *m, uint16_t new_value)
> > 	rte_atomic16_set(&m->refcnt_atomic, new_value);
> > }
> > 
> > +/* internal */
> > +static inline uint16_t
> > +__rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> > +{
> > +	return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value));
> 
> What’s the purpose of using direct cast to uint16_t here and in other places?

This is just a code move.

Few years ago, I remember that icc was quite quick to trigger warnings when
doing implicit casts. I don't know it it's still true, but that may be the
reason why this was done like this initially, or not.

I agree we could remove this explicit cast, but I think it should go in
another patch, because there are several of them.

Olivier
  

On Fri,  8 Dec 2017 16:46:51 +0100
Olivier Matz <olivier.matz@6wind.com> wrote:

> +/* internal */
> +static inline uint16_t
> +__rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> +{
> +	return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value));
> +}

You don't need the cast (or paren's around rte_atomic16_addr_return)
because C has implicit cast to return value.
  

On Fri, Dec 08, 2017 at 04:46:51PM +0100, Olivier Matz wrote:
> When RTE_MBUF_REFCNT_ATOMIC=n, the decrement of the mbuf reference
> counter uses an atomic operation. This is not necessary and impacts
> the performance (seen with TRex traffic generator).
> 
> We cannot replace rte_atomic16_add_return() by rte_mbuf_refcnt_update()
> because it would add an additional check.
> 
> Solves this by introducing __rte_mbuf_refcnt_update(), which
> updates the reference counter without doing anything else.
> 
> Fixes: 8f094a9ac5d7 ("mbuf: set mbuf fields while in pool")
> Suggested-by: Hanoch Haim <hhaim@cisco.com>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>

Forgot to Cc stable@dpdk.org
  

08/12/2017 16:46, Olivier Matz:
> When RTE_MBUF_REFCNT_ATOMIC=n, the decrement of the mbuf reference
> counter uses an atomic operation. This is not necessary and impacts
> the performance (seen with TRex traffic generator).
> 
> We cannot replace rte_atomic16_add_return() by rte_mbuf_refcnt_update()
> because it would add an additional check.
> 
> Solves this by introducing __rte_mbuf_refcnt_update(), which
> updates the reference counter without doing anything else.
> 
> Fixes: 8f094a9ac5d7 ("mbuf: set mbuf fields while in pool")
> Suggested-by: Hanoch Haim <hhaim@cisco.com>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>

Applied, thanks