[dpdk-dev,v4,4/4] vhost: destroy unused virtqueues when multiqueue not negotiated
Checks
Commit Message
QEMU sends VHOST_USER_SET_VRING_CALL requests for all queues
declared in QEMU command line before the guest is started.
It has the effect in DPDK vhost-user backend to allocate vrings
for all queues declared by QEMU.
If the first driver being used does not support multiqueue,
the device never changes to VIRTIO_DEV_RUNNING state as only
the first queue pair is initialized. One driver impacted by
this bug is virtio-net's iPXE driver which does not support
VIRTIO_NET_F_MQ feature.
It is safe to destroy unused virtqueues in SET_FEATURES request
handler, as it is ensured the device is not in running state
at this stage, so virtqueues aren't being processed.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
lib/librte_vhost/vhost_user.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
Comments
On Mon, Dec 11, 2017 at 04:15:03PM +0100, Maxime Coquelin wrote:
> QEMU sends VHOST_USER_SET_VRING_CALL requests for all queues
> declared in QEMU command line before the guest is started.
> It has the effect in DPDK vhost-user backend to allocate vrings
> for all queues declared by QEMU.
>
> If the first driver being used does not support multiqueue,
> the device never changes to VIRTIO_DEV_RUNNING state as only
> the first queue pair is initialized. One driver impacted by
> this bug is virtio-net's iPXE driver which does not support
> VIRTIO_NET_F_MQ feature.
>
> It is safe to destroy unused virtqueues in SET_FEATURES request
> handler, as it is ensured the device is not in running state
> at this stage, so virtqueues aren't being processed.
>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> lib/librte_vhost/vhost_user.c | 18 ++++++++++++++++++
> 1 file changed, 18 insertions(+)
>
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 471b1612c..d5ca1ac90 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -216,6 +216,24 @@ vhost_user_set_features(struct virtio_net *dev, uint64_t features)
> (dev->features & (1 << VIRTIO_NET_F_MRG_RXBUF)) ? "on" : "off",
> (dev->features & (1ULL << VIRTIO_F_VERSION_1)) ? "on" : "off");
>
> + if (!(dev->features & (1ULL << VIRTIO_NET_F_MQ))) {
> + /*
> + * Remove all but first queue pair if MQ hasn't been
> + * negotiated. This is safe because the device is not
> + * running at this stage.
> + */
> + while (dev->nr_vring > 2) {
> + struct vhost_virtqueue *vq;
> +
> + vq = dev->virtqueue[--dev->nr_vring];
> + if (!vq)
> + continue;
> +
> + cleanup_vq(vq, 1);
> + free_vq(vq);
Hi,
Sorry, I didn't look into this patch in last version.
The freed dev->virtqueue[$idx] also needs to be zeroed.
Otherwise, it won't be allocated in the future due to the
below check in vhost_user_check_and_alloc_queue_pair(),
and the freed memory will be used again.
/*
* Allocate a queue pair if it hasn't been allocated yet
*/
static int
vhost_user_check_and_alloc_queue_pair(struct virtio_net *dev, VhostUserMsg *msg)
{
........
if (dev->virtqueue[vring_idx])
return 0;
return alloc_vring_queue(dev, vring_idx);
}
Best regards,
Tiwei Bie
> + }
> + }
> +
> return 0;
> }
>
> --
> 2.14.3
>
On 12/13/2017 04:16 AM, Tiwei Bie wrote:
> On Mon, Dec 11, 2017 at 04:15:03PM +0100, Maxime Coquelin wrote:
>> QEMU sends VHOST_USER_SET_VRING_CALL requests for all queues
>> declared in QEMU command line before the guest is started.
>> It has the effect in DPDK vhost-user backend to allocate vrings
>> for all queues declared by QEMU.
>>
>> If the first driver being used does not support multiqueue,
>> the device never changes to VIRTIO_DEV_RUNNING state as only
>> the first queue pair is initialized. One driver impacted by
>> this bug is virtio-net's iPXE driver which does not support
>> VIRTIO_NET_F_MQ feature.
>>
>> It is safe to destroy unused virtqueues in SET_FEATURES request
>> handler, as it is ensured the device is not in running state
>> at this stage, so virtqueues aren't being processed.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>> lib/librte_vhost/vhost_user.c | 18 ++++++++++++++++++
>> 1 file changed, 18 insertions(+)
>>
>> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
>> index 471b1612c..d5ca1ac90 100644
>> --- a/lib/librte_vhost/vhost_user.c
>> +++ b/lib/librte_vhost/vhost_user.c
>> @@ -216,6 +216,24 @@ vhost_user_set_features(struct virtio_net *dev, uint64_t features)
>> (dev->features & (1 << VIRTIO_NET_F_MRG_RXBUF)) ? "on" : "off",
>> (dev->features & (1ULL << VIRTIO_F_VERSION_1)) ? "on" : "off");
>>
>> + if (!(dev->features & (1ULL << VIRTIO_NET_F_MQ))) {
>> + /*
>> + * Remove all but first queue pair if MQ hasn't been
>> + * negotiated. This is safe because the device is not
>> + * running at this stage.
>> + */
>> + while (dev->nr_vring > 2) {
>> + struct vhost_virtqueue *vq;
>> +
>> + vq = dev->virtqueue[--dev->nr_vring];
>> + if (!vq)
>> + continue;
>> +
>> + cleanup_vq(vq, 1);
>> + free_vq(vq);
>
> Hi,
>
> Sorry, I didn't look into this patch in last version.
Don't be sorry, thanks for catching this bug.
> The freed dev->virtqueue[$idx] also needs to be zeroed.
> Otherwise, it won't be allocated in the future due to the
> below check in vhost_user_check_and_alloc_queue_pair(),
> and the freed memory will be used again.
>
> /*
> * Allocate a queue pair if it hasn't been allocated yet
> */
> static int
> vhost_user_check_and_alloc_queue_pair(struct virtio_net *dev, VhostUserMsg *msg)
> {
> ........
>
> if (dev->virtqueue[vring_idx])
> return 0;
>
> return alloc_vring_queue(dev, vring_idx);
> }
You are right, I'll post v5 setting dev->virtqueue[$idx] to NULL after
free_vq() call.
Thanks for the review,
Maxime
> Best regards,
> Tiwei Bie
>
>> + }
>> + }
>> +
>> return 0;
>> }
>>
>> --
>> 2.14.3
>>
@@ -216,6 +216,24 @@ vhost_user_set_features(struct virtio_net *dev, uint64_t features)
(dev->features & (1 << VIRTIO_NET_F_MRG_RXBUF)) ? "on" : "off",
(dev->features & (1ULL << VIRTIO_F_VERSION_1)) ? "on" : "off");
+ if (!(dev->features & (1ULL << VIRTIO_NET_F_MQ))) {
+ /*
+ * Remove all but first queue pair if MQ hasn't been
+ * negotiated. This is safe because the device is not
+ * running at this stage.
+ */
+ while (dev->nr_vring > 2) {
+ struct vhost_virtqueue *vq;
+
+ vq = dev->virtqueue[--dev->nr_vring];
+ if (!vq)
+ continue;
+
+ cleanup_vq(vq, 1);
+ free_vq(vq);
+ }
+ }
+
return 0;
}