[PATCH] vhost: avoid sleeping under mutex

[Xia, Chenbo]

Hi David,

> -----Original Message-----
> From: David Marchand <david.marchand at redhat.com>
> Sent: Thursday, March 23, 2023 1:05 AM
> To: dev at dpdk.org
> Cc: stable at dpdk.org; Maxime Coquelin <maxime.coquelin at redhat.com>; Xia,
> Chenbo <chenbo.xia at intel.com>; Yuanhan Liu <yliu at fridaylinux.org>
> Subject: [PATCH] vhost: avoid sleeping under mutex
> 
> Covscan reported:
> 
>  2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
> 	"pthread_mutex_lock" acquires lock "vhost_user.mutex".
> 23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
> 	"vhost_user_reconnect_init" might sleep while holding lock
> 	"vhost_user.mutex".
>  #   953|   		vsocket->reconnect =
> 	!(flags & RTE_VHOST_USER_NO_RECONNECT);
>  #   954|   		if (vsocket->reconnect && reconn_tid == 0) {
>  #   955|-> 			if (vhost_user_reconnect_init() != 0)
>  #   956|   				goto out_mutex;
>  #   957|   		}
> 
> The reason for this warning is that vhost_user_reconnect_init() creates a
> ctrl thread and calls nanosleep waiting for this thread to be ready,
> while vhost_user.mutex is taken.
> 
> Move the call to vhost_user_reconnect_init() out of this mutex.
> 
> While at it, a pthread_t value should be considered opaque.
> Instead of relying reconn_tid == 0, use an internal flag in
> vhost_user_reconnect_init().

Should we make reconn_tid not a global variable anymore then?
Maybe a static pthread_t in vhost_user_reconnect_init?

Thanks,
Chenbo

> 
> Coverity issue: 373686
> Bugzilla ID: 981
> Fixes: e623e0c6d8a5 ("vhost: add reconnect ability")
> Cc: stable at dpdk.org
> 
> Signed-off-by: David Marchand <david.marchand at redhat.com>
> ---
>  lib/vhost/socket.c | 17 ++++++++++++-----
>  1 file changed, 12 insertions(+), 5 deletions(-)
> 
> diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c
> index 669c322e12..21002848e6 100644
> --- a/lib/vhost/socket.c
> +++ b/lib/vhost/socket.c
> @@ -498,8 +498,12 @@ vhost_user_client_reconnect(void *arg __rte_unused)
>  static int
>  vhost_user_reconnect_init(void)
>  {
> +	static bool reconn_init_done;
>  	int ret;
> 
> +	if (reconn_init_done)
> +		return 0;
> +
>  	ret = pthread_mutex_init(&reconn_list.mutex, NULL);
>  	if (ret < 0) {
>  		VHOST_LOG_CONFIG("thread", ERR, "%s: failed to initialize
> mutex\n", __func__);
> @@ -515,6 +519,8 @@ vhost_user_reconnect_init(void)
>  			VHOST_LOG_CONFIG("thread", ERR,
>  				"%s: failed to destroy reconnect mutex\n",
>  				__func__);
> +	} else {
> +		reconn_init_done = true;
>  	}
> 
>  	return ret;
> @@ -866,6 +872,11 @@ rte_vhost_driver_register(const char *path, uint64_t
> flags)
>  	if (!path)
>  		return -1;
> 
> +	if ((flags & RTE_VHOST_USER_CLIENT) != 0 &&
> +			(flags & RTE_VHOST_USER_NO_RECONNECT) == 0 &&
> +			vhost_user_reconnect_init() != 0)
> +		return -1;
> +
>  	pthread_mutex_lock(&vhost_user.mutex);
> 
>  	if (vhost_user.vsocket_cnt == MAX_VHOST_SOCKET) {
> @@ -961,11 +972,7 @@ rte_vhost_driver_register(const char *path, uint64_t
> flags)
>  	}
> 
>  	if ((flags & RTE_VHOST_USER_CLIENT) != 0) {
> -		vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT);
> -		if (vsocket->reconnect && reconn_tid == 0) {
> -			if (vhost_user_reconnect_init() != 0)
> -				goto out_mutex;
> -		}
> +		vsocket->reconnect = (flags & RTE_VHOST_USER_NO_RECONNECT) ==
> 0;
>  	} else {
>  		vsocket->is_server = true;
>  	}
> --
> 2.39.2