[PATCH 2/2] ethdev: fix race condition in fast-path ops setup

[Stephen Hemminger]

On Tue, 21 Feb 2023 07:24:19 +0000
Ruifeng Wang <Ruifeng.Wang at arm.com> wrote:

> > -----Original Message-----
> > From: fengchengwen <fengchengwen at huawei.com>
> > Sent: Monday, February 20, 2023 2:58 PM
> > To: Ashok Kaladi <ashok.k.kaladi at intel.com>; jerinj at marvell.com; thomas at monjalon.net
> > Cc: dev at dpdk.org; s.v.naga.harish.k at intel.com; erik.g.carrillo at intel.com;
> > abhinandan.gujjar at intel.com; stable at dpdk.org; Ruifeng Wang <Ruifeng.Wang at arm.com>
> > Subject: Re: [PATCH 2/2] ethdev: fix race condition in fast-path ops setup
> > 
> > On 2023/2/20 14:08, Ashok Kaladi wrote:  
> > > If ethdev enqueue or dequeue function is called during
> > > eth_dev_fp_ops_setup(), it may get pre-empted after setting the
> > > function pointers, but before setting the pointer to port data.
> > > In this case the newly registered enqueue/dequeue function will use
> > > dummy port data and end up in seg fault.
> > >
> > > This patch moves the updation of each data pointers before updating
> > > corresponding function pointers.
> > >
> > > Fixes: c87d435a4d79 ("ethdev: copy fast-path API into separate
> > > structure")
> > > Cc: stable at dpdk.org

Why is something calling enqueue/dequeue when device is not fully started.
A correctly written application would not call rx/tx burst until after
ethdev start had finished.

Would something like this work better?

Note: there is another bug in current code. The check for link state interrupt
and link_ops could return -ENOTSUP and leave device in indeterminate state.
The check should be done before calling PMD.

diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index 0266cc82acb6..d6c163ed85e7 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -1582,6 +1582,14 @@ rte_eth_dev_start(uint16_t port_id)
 		return 0;
 	}
 
+	if (dev->data->dev_conf.intr_conf.lsc == 0 &&
+	    dev->dev_ops->link_update == NULL) {
+		RTE_ETHDEV_LOG(INFO,
+			       "Device with port_id=%"PRIu16" link update not supported\n",
+			       port_id);
+			return -ENOTSUP;
+	}
+
 	ret = rte_eth_dev_info_get(port_id, &dev_info);
 	if (ret != 0)
 		return ret;
@@ -1591,9 +1599,7 @@ rte_eth_dev_start(uint16_t port_id)
 		eth_dev_mac_restore(dev, &dev_info);
 
 	diag = (*dev->dev_ops->dev_start)(dev);
-	if (diag == 0)
-		dev->data->dev_started = 1;
-	else
+	if (diag != 0)
 		return eth_err(port_id, diag);
 
 	ret = eth_dev_config_restore(dev, &dev_info, port_id);
@@ -1611,16 +1617,18 @@ rte_eth_dev_start(uint16_t port_id)
 		return ret;
 	}
 
-	if (dev->data->dev_conf.intr_conf.lsc == 0) {
-		if (*dev->dev_ops->link_update == NULL)
-			return -ENOTSUP;
-		(*dev->dev_ops->link_update)(dev, 0);
-	}
-
 	/* expose selection of PMD fast-path functions */
 	eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
 
+	/* ensure state is set before marking device ready */
+	rte_smp_wmb();
+
 	rte_ethdev_trace_start(port_id);
+
+	/* Update current link state */
+	if (dev->data->dev_conf.intr_conf.lsc == 0)
+		(*dev->dev_ops->link_update)(dev, 0);
+
 	return 0;
 }