[dpdk-dev,2/2] net/virtio-user: fix multi-process issue

Message ID 1488102926-24158-3-git-send-email-amis@radware.com (mailing list archive)
State Superseded, archived
Delegated to: Yuanhan Liu
Headers

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Ami Sabo Feb. 26, 2017, 9:55 a.m. UTC
  Secondary process doesn't properly attach to the rte_eth_device
initialized by the primary process.

ccessing device from secondary process (e.g. via rte_eth_rx_burst),
causes process to crash. because rte_eth_dev_data is not properly set.

The issue was flood by
'commit 7f95f78a8aea ("ethdev: clear data when allocating device")'
which now clears rte_eth_dev_data entry.
For pci devices the struct is initialized by rte_eth_dev_pci_probe
->eth_dev_attach_secondary().
However, for virtio-user virtio_user_pmd_probe() is called instead of
rte_eth_dev_pci_probe().

The fix is to call rte_eth_dev_attach_secondary(), for secondary
process, from virtio_user_pmd_probe.

Fixes: 7f95f78a8aea ("ethdev: clear data when allocating device")

Cc: stable@dpdk.org

Signed-off-by: Ami Sabo <amis@radware.com>
---
 drivers/net/virtio/virtio_user_ethdev.c | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)
  

Comments

Yuanhan Liu Feb. 28, 2017, 6:40 a.m. UTC | #1
On Sun, Feb 26, 2017 at 11:55:26AM +0200, Ami Sabo wrote:
> Secondary process doesn't properly attach to the rte_eth_device
> initialized by the primary process.
> 
> ccessing device from secondary process (e.g. via rte_eth_rx_burst),
> causes process to crash. because rte_eth_dev_data is not properly set.
> 
> The issue was flood by
> 'commit 7f95f78a8aea ("ethdev: clear data when allocating device")'
> which now clears rte_eth_dev_data entry.
> For pci devices the struct is initialized by rte_eth_dev_pci_probe
> ->eth_dev_attach_secondary().
> However, for virtio-user virtio_user_pmd_probe() is called instead of
> rte_eth_dev_pci_probe().
> 
> The fix is to call rte_eth_dev_attach_secondary(), for secondary
> process, from virtio_user_pmd_probe.
> 
> Fixes: 7f95f78a8aea ("ethdev: clear data when allocating device")

Are you sure that's the real culprit? As I'm aware of, virtio-user
is not built with multiple process support in the beginning. That
said, it's likely that the first commit introduces virtio-user is
the "culprit" commit.

Besides that, the code looks good to me. If Thomas is fine with
your first patch, I could merge them to my tree.

	--yliu
  
Ami Sabo Feb. 28, 2017, 7:50 a.m. UTC | #2
Hi,
You are right, the commit I mentioned didn't cause the issue - it just flooded it.
The real issue is that rte_eth_dev_allocate should be called only from the primary process.
Tomas's commit flood the issue by resseting rte_eth_dev_data, so now, when the virtio-user secondary process comes up and calls rte_eth_dev_allocate
It clears the ethdev->data struct (so fields like rx_queues, mac_addrs, etc will be 0, plus this may cause race condition between the primary and secondary processes...) 

	--ami

-----Original Message-----
From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com] 
Sent: Tuesday, February 28, 2017 8:40 AM
To: Ami Sabo
Cc: dev@dpdk.org; stable@dpdk.org
Subject: Re: [PATCH 2/2] net/virtio-user: fix multi-process issue

On Sun, Feb 26, 2017 at 11:55:26AM +0200, Ami Sabo wrote:
> Secondary process doesn't properly attach to the rte_eth_device 
> initialized by the primary process.
> 
> ccessing device from secondary process (e.g. via rte_eth_rx_burst), 
> causes process to crash. because rte_eth_dev_data is not properly set.
> 
> The issue was flood by
> 'commit 7f95f78a8aea ("ethdev: clear data when allocating device")'
> which now clears rte_eth_dev_data entry.
> For pci devices the struct is initialized by rte_eth_dev_pci_probe
> ->eth_dev_attach_secondary().
> However, for virtio-user virtio_user_pmd_probe() is called instead of 
> rte_eth_dev_pci_probe().
> 
> The fix is to call rte_eth_dev_attach_secondary(), for secondary 
> process, from virtio_user_pmd_probe.
> 
> Fixes: 7f95f78a8aea ("ethdev: clear data when allocating device")

Are you sure that's the real culprit? As I'm aware of, virtio-user is not built with multiple process support in the beginning. That said, it's likely that the first commit introduces virtio-user is the "culprit" commit.

Besides that, the code looks good to me. If Thomas is fine with your first patch, I could merge them to my tree.

	--yliu
  

Patch

diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 0b226ac..6033908 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -418,18 +418,24 @@  virtio_user_pmd_probe(const char *name, const char *params)
 		goto end;
 	}
 
-	eth_dev = virtio_user_eth_dev_alloc(name);
-	if (!eth_dev) {
-		PMD_INIT_LOG(ERR, "virtio_user fails to alloc device");
-		goto end;
-	}
+	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+		eth_dev = virtio_user_eth_dev_alloc(name);
+		if (!eth_dev) {
+			PMD_INIT_LOG(ERR, "virtio_user fails to alloc device");
+			goto end;
+		}
 
-	hw = eth_dev->data->dev_private;
-	if (virtio_user_dev_init(hw->virtio_user_dev, path, queues, cq,
+		hw = eth_dev->data->dev_private;
+		if (virtio_user_dev_init(hw->virtio_user_dev, path, queues, cq,
 				 queue_size, mac_addr) < 0) {
-		PMD_INIT_LOG(ERR, "virtio_user_dev_init fails");
-		virtio_user_eth_dev_free(eth_dev);
-		goto end;
+			PMD_INIT_LOG(ERR, "virtio_user_dev_init fails");
+			virtio_user_eth_dev_free(eth_dev);
+			goto end;
+		}
+	} else {
+		eth_dev = rte_eth_dev_attach_secondary(name);
+		if (!eth_dev)
+			goto end;
 	}
 
 	/* previously called by rte_eal_pci_probe() for physical dev */