[dpdk-dev] [PATCH 2/5] net/mlx5: free buffers in bulk on Tx completion

[Yongseok Koh]

On Fri, Jun 30, 2017 at 02:43:21PM +0200, Nélio Laranjeiro wrote:
> On Fri, Jun 30, 2017 at 02:30:47PM +0200, Nélio Laranjeiro wrote:
> > On Wed, Jun 28, 2017 at 04:04:00PM -0700, Yongseok Koh wrote:
> > > When processing Tx completion, it is more efficient to free buffers in bulk
> > > using rte_mempool_put_bulk() if buffers are from a same mempool.
> > > 
> > > Signed-off-by: Yongseok Koh <yskoh at mellanox.com>
> > > ---
> > >  drivers/net/mlx5/mlx5_rxtx.c | 36 +++++++++++++++++++++++++++---------
> > >  1 file changed, 27 insertions(+), 9 deletions(-)
> > > 
> > > diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
> > > index 43db06ad8..d81d630f7 100644
> > > --- a/drivers/net/mlx5/mlx5_rxtx.c
> > > +++ b/drivers/net/mlx5/mlx5_rxtx.c
> > > @@ -264,6 +264,8 @@ txq_complete(struct txq *txq)
> > >  	uint16_t cq_ci = txq->cq_ci;
> > >  	volatile struct mlx5_cqe *cqe = NULL;
> > >  	volatile struct mlx5_wqe_ctrl *ctrl;
> > > +	struct rte_mbuf *m, *free[elts_n];
> > > +	unsigned int blk_n = 0;
> > >  
> > >  	do {
> > >  		volatile struct mlx5_cqe *tmp;
> > > @@ -296,21 +298,37 @@ txq_complete(struct txq *txq)
> > >  	assert((elts_tail & elts_m) < (1 << txq->wqe_n));
> > >  	/* Free buffers. */
> > >  	while (elts_free != elts_tail) {
> > > -		struct rte_mbuf *elt = (*txq->elts)[elts_free & elts_m];
> > > -		struct rte_mbuf *elt_next =
> > > -			(*txq->elts)[(elts_free + 1) & elts_m];
> > > -
> > > +		m = rte_pktmbuf_prefree_seg((*txq->elts)[elts_free++ & elts_m]);
> > > +		if (likely(m != NULL)) {
> > > +			if (blk_n) {
> > > +				if (likely(m->pool == free[0]->pool)) {
> > > +					free[blk_n++] = m;
> > > +				} else {
> > > +					rte_mempool_put_bulk(
> > > +							free[0]->pool,
> > > +							(void *)free,
> > > +							blk_n);
> > 
> > The indentation is strange here, free[0] should be on the same line as
> > rte_mempool_put_bulk.
> > 
> > > +					free[0] = m;
> > > +					blk_n = 1;
> > > +				}
> > > +			} else {
> > > +				free[0] = m;
> > > +				blk_n = 1;
> > > +			}
> > > +		}
> > 
> > This loop could be smaller, blk_n can only be equal to 0 in the first
> > iteration, otherwise is >= 1.
> > The first if statement can be merged with the second one: 
> > 
> > 	if (likely(m != NULL)) {
> > 		if (likely(blk_n && m->pool == free[0]->pool)) {
> 
> This condition is a wrong also, it should be !blk_n || (m->pool ...
> 
> Why don't you keep a pointer to the mpool (e.g. m->pool == pool)?  It
> seems to cost a little to deference two pointers to reach the pool's
> one.
Good point. The following will be the final streamlined code.
        /* Free buffers. */
        while (elts_free != elts_tail) {
                m = rte_pktmbuf_prefree_seg((*txq->elts)[elts_free++ & elts_m]);
                if (likely(m != NULL)) {
                        if (likely(m->pool == pool)) {
                                free[blk_n++] = m;
                        } else {
                                if (likely(pool != NULL))
                                        rte_mempool_put_bulk(pool,
                                                             (void *)free,
                                                             blk_n);
                                free[0] = m;
                                pool = m->pool;
                                blk_n = 1;
                        }
                }
        }
        if (blk_n)
                rte_mempool_put_bulk(pool, (void *)free, blk_n);


Thanks,
Yongseok