[dpdk-dev] [PATCH v2] net/tap: fix device removal when no queues exist

[Ophir Munk]

Hi Keith,
Please find comments inline

> -----Original Message-----
> From: Wiles, Keith [mailto:keith.wiles at intel.com]
> Sent: Thursday, May 17, 2018 4:00 PM
> To: Ophir Munk <ophirmu at mellanox.com>
> Cc: dev at dpdk.org; Pascal Mazon <pascal.mazon at 6wind.com>; Thomas
> Monjalon <thomas at monjalon.net>; Olga Shern <olgas at mellanox.com>;
> Shahaf Shuler <shahafs at mellanox.com>; stable at dpdk.org
> Subject: Re: [PATCH v2] net/tap: fix device removal when no queues exist
> 
> 
> 
> > On May 17, 2018, at 2:47 AM, Ophir Munk <ophirmu at mellanox.com>
> wrote:
> >
> > TAP device is created following its first queue creation. Multiple
> > queues can be added or removed over time. In Linux terminology those
> > are file descriptors which are opened or closed over time. As long as
> > the number of opened file descriptors is positive - TAP device will
> > appear as a Linux device. In case all queues are released (the
> > equivalent of all file descriptors being closed) the TAP device will
> > be removed. This can lead to abnormalities in different scenarios
> > where the TAP device should exist even if all its queues are released.
> > In order to make TAP existence independent of its number of queues -
> > an extra file descriptor is opened on TAP creation and is closed on
> > TAP closure. It's only purpose is to serve as a keep-alive mechanism
> > for the TAP device.
> >
> > Fixes: bf7b7f437b49 ("net/tap: create netdevice during probing")
> > Cc: stable at dpdk.org
> >
> > Signed-off-by: Ophir Munk <ophirmu at mellanox.com>
> > ---
> > v1:
> > Initial release
> > v2:
> > Reword commit message (a fixing patch)
> >
> > drivers/net/tap/rte_eth_tap.c | 31 ++++++++++++++++++++++++-------
> > drivers/net/tap/rte_eth_tap.h |  1 +
> > 2 files changed, 25 insertions(+), 7 deletions(-)
> 
> I did not see where ka_fd is set to -1 at startup, just in case we fail before the
> first open attempt and possible hit the close code in the remove routine. I did
> not look at the complete driver, but I think it maybe reasonable to add that
> initial variable setup.
> 

Your concern is in place. However, please note that ka_fd is allocated and initialized to 0 as part of private device structure initialization during the call to:
dev = rte_eth_vdev_allocate(vdev, sizeof(*pmd));

Therefore it is guaranteed to be 0 in case we fail before the first open attempt and if we hit the close code we will not be closing anything wrong (closing a 0 fd has no effect). 

While reading the code again I have noticed that tun_alloc() always return -1 so no need to reassign ka_fd to -1 after calling tun_alloc. Also I have fixed a typo in the commit message: It's --> Its.
I have sent v3 with the above small changes.

> >
> > diff --git a/drivers/net/tap/rte_eth_tap.c
> > b/drivers/net/tap/rte_eth_tap.c index c006d07..6901edc 100644
> > --- a/drivers/net/tap/rte_eth_tap.c
> > +++ b/drivers/net/tap/rte_eth_tap.c
> > @@ -929,6 +929,15 @@ tap_dev_close(struct rte_eth_dev *dev)
> > 		ioctl(internals->ioctl_sock, SIOCSIFFLAGS,
> > 				&internals->remote_initial_flags);
> > 	}
> > +
> > +	if (internals->ka_fd != -1) {
> > +		close(internals->ka_fd);
> > +		internals->ka_fd = -1;
> > +	}
> > +	/*
> > +	 * Since TUN device has no more opened file descriptors
> > +	 * it will be removed from kernel
> > +	 */
> > }
> >
> > static void
> > @@ -1561,13 +1570,18 @@ eth_dev_tap_create(struct rte_vdev_device
> *vdev, char *tap_name,
> > 			rte_memcpy(&pmd->eth_addr, mac_addr,
> sizeof(*mac_addr));
> > 	}
> >
> > -	/* Immediately create the netdevice (this will create the 1st queue).
> */
> > -	/* rx queue */
> > -	if (tap_setup_queue(dev, pmd, 0, 1) == -1)
> > -		goto error_exit;
> > -	/* tx queue */
> > -	if (tap_setup_queue(dev, pmd, 0, 0) == -1)
> > +	/*
> > +	 * Allocate a TUN device keep-alive file descriptor that will only be
> > +	 * closed when the TUN device itself is closed or removed.
> > +	 * This keep-alive file descriptor will guarantee that the TUN device
> > +	 * exists even when all of its queues are closed
> > +	 */
> > +	pmd->ka_fd = tun_alloc(pmd);
> > +	if (pmd->ka_fd < 0) {
> > +		pmd->ka_fd = -1;
> > +		TAP_LOG(ERR, "Unable to create %s interface",
> tuntap_name);
> > 		goto error_exit;
> > +	}
> >
> > 	ifr.ifr_mtu = dev->data->mtu;
> > 	if (tap_ioctl(pmd, SIOCSIFMTU, &ifr, 1, LOCAL_AND_REMOTE) < 0)
> @@
> > -1961,9 +1975,12 @@ rte_pmd_tap_remove(struct rte_vdev_device *dev)
> >
> > 	close(internals->ioctl_sock);
> > 	rte_free(eth_dev->data->dev_private);
> > -
> > 	rte_eth_dev_release_port(eth_dev);
> >
> > +	if (internals->ka_fd != -1) {
> > +		close(internals->ka_fd);
> > +		internals->ka_fd = -1;
> > +	}
> > 	return 0;
> > }
> >
> > diff --git a/drivers/net/tap/rte_eth_tap.h
> > b/drivers/net/tap/rte_eth_tap.h index babe42d..575dce4 100644
> > --- a/drivers/net/tap/rte_eth_tap.h
> > +++ b/drivers/net/tap/rte_eth_tap.h
> > @@ -81,6 +81,7 @@ struct pmd_internals {
> > 	struct rx_queue rxq[RTE_PMD_TAP_MAX_QUEUES]; /* List of RX
> queues */
> > 	struct tx_queue txq[RTE_PMD_TAP_MAX_QUEUES]; /* List of TX
> queues */
> > 	struct rte_intr_handle intr_handle;          /* LSC interrupt handle. */
> > +	int ka_fd;                        /* keep-alive file descriptor */
> > };
> >
> > /* tap_intr.c */
> > --
> > 2.7.4
> >
> 
> Regards,
> Keith