[dpdk-dev] [PATCH 3/5] net/mlx5: use buffer address for LKEY search

[Yongseok Koh]

On Fri, Jun 30, 2017 at 03:01:51PM +0200, Nélio Laranjeiro wrote:
> On Wed, Jun 28, 2017 at 04:04:01PM -0700, Yongseok Koh wrote:
> > When searching LKEY, if search key is mempool pointer, the 2nd cacheline
> > has to be accessed and it even requires to check whether a buffer is
> > indirect per every search. Instead, using address for search key can reduce
> > cycles taken. And caching the last hit entry is beneficial as well.
> > 
> > Signed-off-by: Yongseok Koh <yskoh at mellanox.com>
> > ---
> >  drivers/net/mlx5/mlx5_mr.c   |  2 ++
> >  drivers/net/mlx5/mlx5_rxtx.c | 37 ++++++++++++++++++++-----------------
> >  drivers/net/mlx5/mlx5_rxtx.h |  3 +++
> >  3 files changed, 25 insertions(+), 17 deletions(-)
> > 
> >[...]
> > diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
> > index f9b738b4e..a86f41abc 100644
> > --- a/drivers/net/mlx5/mlx5_rxtx.h
> > +++ b/drivers/net/mlx5/mlx5_rxtx.h
> > @@ -267,10 +267,13 @@ struct txq {
> >  	volatile uint32_t *cq_db; /* Completion queue doorbell. */
> >  	volatile void *bf_reg; /* Blueflame register. */
> >  	struct {
> > +		uintptr_t start; /* Start address of MR */
> > +		uintptr_t end;   /* End address of MR */
> 
> Extra space between the ';' and the comment.
> 
> >  		const struct rte_mempool *mp; /* Cached Memory Pool. */
> >  		struct ibv_mr *mr; /* Memory Region (for mp). */
> >  		uint32_t lkey; /* htonl(mr->lkey) */
> >  	} mp2mr[MLX5_PMD_TX_MP_CACHE]; /* MP to MR translation table. */
> > +	uint16_t mr_cache_idx; /* Index of last hit entry. */
> >  	struct rte_mbuf *(*elts)[]; /* TX elements. */
> >  	struct mlx5_txq_stats stats; /* TX queue counters. */
> >  } __rte_cache_aligned;
> > -- 
> > 2.11.0
> 
> The pointer to the mempool should be removed from this structure, it was
> useful in the dataplane to retrieve the lkey, but with this new
> implementation it becomes useless.
> This also helps to keep the memory footprint of this array.  The control
> plane can spend some cycles to retrieve the start/end addresses of the
> mempool to compare them.
Agree. Will make the change.

Thanks,
Yongseok