[dpdk-dev,v4] mbuf: fix mbuf free performance with non atomic refcnt
Checks
Commit Message
When RTE_MBUF_REFCNT_ATOMIC=n, the decrement of the mbuf reference
counter uses an atomic operation. This is not necessary and impacts
the performance (seen with TRex traffic generator).
We cannot replace rte_atomic16_add_return() by rte_mbuf_refcnt_update()
because it would add an additional check.
Solves this by introducing __rte_mbuf_refcnt_update(), which
updates the reference counter without doing anything else.
Fixes: 8f094a9ac5d7 ("mbuf: set mbuf fields while in pool")
Suggested-by: Hanoch Haim <hhaim@cisco.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
Hanoh,
The following patch implements what was discussed in the thread.
Are you ok with it?
Thanks,
Olivier
lib/librte_mbuf/rte_mbuf.h | 23 ++++++++++++++++++-----
1 file changed, 18 insertions(+), 5 deletions(-)
Comments
Olivier,
> On Dec 8, 2017, at 7:46 PM, Olivier Matz <olivier.matz@6wind.com> wrote:
>
>
> lib/librte_mbuf/rte_mbuf.h | 23 ++++++++++++++++++-----
> 1 file changed, 18 insertions(+), 5 deletions(-)
>
> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> index ce8a05ddf..dd08cb72b 100644
> --- a/lib/librte_mbuf/rte_mbuf.h
> +++ b/lib/librte_mbuf/rte_mbuf.h
> @@ -764,6 +764,13 @@ rte_mbuf_refcnt_set(struct rte_mbuf *m, uint16_t new_value)
> rte_atomic16_set(&m->refcnt_atomic, new_value);
> }
>
> +/* internal */
> +static inline uint16_t
> +__rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> +{
> + return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value));
What’s the purpose of using direct cast to uint16_t here and in other places?
> +}
> +
> /**
> * Adds given value to an mbuf's refcnt and returns its new value.
> * @param m
> @@ -788,19 +795,26 @@ rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> return 1 + value;
> }
>
> - return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value));
> + return __rte_mbuf_refcnt_update(m, value);
> }
>
> #else /* ! RTE_MBUF_REFCNT_ATOMIC */
>
> +/* internal */
> +static inline uint16_t
> +__rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> +{
> + m->refcnt = (uint16_t)(m->refcnt + value);
> + return m->refcnt;
> +}
> +
> /**
> * Adds given value to an mbuf's refcnt and returns its new value.
> */
> static inline uint16_t
> rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> {
> - m->refcnt = (uint16_t)(m->refcnt + value);
> - return m->refcnt;
> + return __rte_mbuf_refcnt_update(m, value);
> }
>
> /**
> @@ -1364,8 +1378,7 @@ rte_pktmbuf_prefree_seg(struct rte_mbuf *m)
>
> return m;
>
> - } else if (rte_atomic16_add_return(&m->refcnt_atomic, -1) == 0) {
> -
> + } else if (__rte_mbuf_refcnt_update(m, -1) == 0) {
>
> if (RTE_MBUF_INDIRECT(m))
> rte_pktmbuf_detach(m);
> --
> 2.11.0
>
On Fri, Dec 08, 2017 at 08:04:50PM +0400, Ilya Matveychikov wrote:
> Olivier,
>
> > On Dec 8, 2017, at 7:46 PM, Olivier Matz <olivier.matz@6wind.com> wrote:
> >
>
> >
> > lib/librte_mbuf/rte_mbuf.h | 23 ++++++++++++++++++-----
> > 1 file changed, 18 insertions(+), 5 deletions(-)
> >
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > index ce8a05ddf..dd08cb72b 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -764,6 +764,13 @@ rte_mbuf_refcnt_set(struct rte_mbuf *m, uint16_t new_value)
> > rte_atomic16_set(&m->refcnt_atomic, new_value);
> > }
> >
> > +/* internal */
> > +static inline uint16_t
> > +__rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> > +{
> > + return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value));
>
> What’s the purpose of using direct cast to uint16_t here and in other places?
This is just a code move.
Few years ago, I remember that icc was quite quick to trigger warnings when
doing implicit casts. I don't know it it's still true, but that may be the
reason why this was done like this initially, or not.
I agree we could remove this explicit cast, but I think it should go in
another patch, because there are several of them.
Olivier
On Fri, 8 Dec 2017 16:46:51 +0100
Olivier Matz <olivier.matz@6wind.com> wrote:
> +/* internal */
> +static inline uint16_t
> +__rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
> +{
> + return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value));
> +}
You don't need the cast (or paren's around rte_atomic16_addr_return)
because C has implicit cast to return value.
On Fri, Dec 08, 2017 at 04:46:51PM +0100, Olivier Matz wrote:
> When RTE_MBUF_REFCNT_ATOMIC=n, the decrement of the mbuf reference
> counter uses an atomic operation. This is not necessary and impacts
> the performance (seen with TRex traffic generator).
>
> We cannot replace rte_atomic16_add_return() by rte_mbuf_refcnt_update()
> because it would add an additional check.
>
> Solves this by introducing __rte_mbuf_refcnt_update(), which
> updates the reference counter without doing anything else.
>
> Fixes: 8f094a9ac5d7 ("mbuf: set mbuf fields while in pool")
> Suggested-by: Hanoch Haim <hhaim@cisco.com>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Forgot to Cc stable@dpdk.org
08/12/2017 16:46, Olivier Matz:
> When RTE_MBUF_REFCNT_ATOMIC=n, the decrement of the mbuf reference
> counter uses an atomic operation. This is not necessary and impacts
> the performance (seen with TRex traffic generator).
>
> We cannot replace rte_atomic16_add_return() by rte_mbuf_refcnt_update()
> because it would add an additional check.
>
> Solves this by introducing __rte_mbuf_refcnt_update(), which
> updates the reference counter without doing anything else.
>
> Fixes: 8f094a9ac5d7 ("mbuf: set mbuf fields while in pool")
> Suggested-by: Hanoch Haim <hhaim@cisco.com>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Applied, thanks
@@ -764,6 +764,13 @@ rte_mbuf_refcnt_set(struct rte_mbuf *m, uint16_t new_value)
rte_atomic16_set(&m->refcnt_atomic, new_value);
}
+/* internal */
+static inline uint16_t
+__rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
+{
+ return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value));
+}
+
/**
* Adds given value to an mbuf's refcnt and returns its new value.
* @param m
@@ -788,19 +795,26 @@ rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
return 1 + value;
}
- return (uint16_t)(rte_atomic16_add_return(&m->refcnt_atomic, value));
+ return __rte_mbuf_refcnt_update(m, value);
}
#else /* ! RTE_MBUF_REFCNT_ATOMIC */
+/* internal */
+static inline uint16_t
+__rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
+{
+ m->refcnt = (uint16_t)(m->refcnt + value);
+ return m->refcnt;
+}
+
/**
* Adds given value to an mbuf's refcnt and returns its new value.
*/
static inline uint16_t
rte_mbuf_refcnt_update(struct rte_mbuf *m, int16_t value)
{
- m->refcnt = (uint16_t)(m->refcnt + value);
- return m->refcnt;
+ return __rte_mbuf_refcnt_update(m, value);
}
/**
@@ -1364,8 +1378,7 @@ rte_pktmbuf_prefree_seg(struct rte_mbuf *m)
return m;
- } else if (rte_atomic16_add_return(&m->refcnt_atomic, -1) == 0) {
-
+ } else if (__rte_mbuf_refcnt_update(m, -1) == 0) {
if (RTE_MBUF_INDIRECT(m))
rte_pktmbuf_detach(m);