[dpdk-dev] [PATCH 2/2] net/mlx5: don't map doorbell register to write combining

[Shahaf Shuler]

Wednesday, August 23, 2017 2:04 PM, Ferruh Yigit:
> On 8/21/2017 8:47 AM, Sagi Grimberg wrote:
> > From: Shahaf Shuler <shahafs at mellanox.com>
> >
> > By default, Verbs maps the doorbell register to write combining.
> > Working with write combining is useful for drivers which use blue
> > flame for the doorbell write.
> >
> > Since mlx5 PMD uses only doorbells and write combining mapping
> > requires an extra memory barrier to flush the doorbell after its
> > write, setting the mapping to un-cached by default.
> >
> > Such change is reduces the max and average round trip latency
> > significantly.
> >
> > Reported-by: Alexander Solganik <solganik at gmail.com>
> > Signed-off-by: Shahaf Shuler <shahafs at mellanox.com>
> > Signed-off-by: Yongseok Koh <yskoh at mellanox.com>
> > Signed-off-by: Alexander Solganik <solganik at gmail.com>
> > Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
> > ---
> >  drivers/net/mlx5/mlx5.c | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index
> > b7e5046325c0..4c2a0b9652e3 100644
> > --- a/drivers/net/mlx5/mlx5.c
> > +++ b/drivers/net/mlx5/mlx5.c
> > @@ -920,6 +920,8 @@ rte_mlx5_pmd_init(void)
> >  	 * using this PMD, which is not supported in forked processes.
> >  	 */
> >  	setenv("RDMAV_HUGEPAGES_SAFE", "1", 1);
> > +	/* Don't map UAR to WC if BlueFlame is not used.*/
> > +	setenv("MLX5_SHUT_UP_BF", "1", 1);

Even if it is not clean, this is the only way to toggle the BF mapping mode on libmlx5/rdma-core.
We will try to propose changes on those libs to expose a proper API, in the meanwhile we have no other way.

> 
> Although technically this is possible, I wonder how good idea it is a driver
> communicating with other processes via system environment variable?
> 
> >  	ibv_fork_init();
> >  	rte_pci_register(&mlx5_driver);
> >  }
> >