[dpdk-dev] [PATCH v3 00/10] rxq interrupt mode for virtio PMD

Yao, Lei A lei.a.yao at intel.com
Tue Jan 17 03:14:51 CET 2017


Tested-by: Lei Yao <lei.a.yao at intel.com>
Apply patch to dpdk_next_virtio branch.
Qemu version: 2.5.0
Kernel version in VM: 4.8.1 

Following TCs are tested and passed:
Test Case1: Basic Virtio Interrupt test
Test Case2: Interrupted received in VM with different Virtio version(0.95 and 1.0)
Test Case3: Interrupted by packet data on all queue
Test Case4: Interrupted by packet data on unique queue
Test Case5: Stop packet transmit, related cores will be back to sleep


> -----Original Message-----
> From: Tan, Jianfeng
> Sent: Monday, January 16, 2017 10:47 PM
> To: dev at dpdk.org
> Cc: yuanhan.liu at linux.intel.com; stephen at networkplumber.org; Yao, Lei A
> <lei.a.yao at intel.com>; Tan, Jianfeng <jianfeng.tan at intel.com>
> Subject: [PATCH v3 00/10] rxq interrupt mode for virtio PMD
> 
> v3:
>   - Update documents:
>     * doc/guides/nics/features/virtio.ini
>     * doc/guides/nics/features/virtio_vec.ini
>     * doc/guides/nics/virtio.rst
>   - Use hw->max_queue_pairs instead of dev->data->nb_rx_queues to
>     allocate intr_vec array.
>   - Fix v2 not working on legacy virtio devices by moving msix enabling
>     before queue/irq binding.
>   - Reword cover letter to give an overview of this series.
>   - Remove wrapper to call vtpci->set_config_irq and vtpci->set_queue_irq.
>   - Rebase on the new code, and fix a bug after changes by the commit
>     bb30369dc10("eal: allow passing const interrupt handle"). Basically,
>     it changes the way to get max interrupts. And we need to re-register
>     callback to update intr_handle->max_intr.
>   - In l3fwd-power ptype fix, use rte_eth_dev_get_supported_ptypes() to
>     query if PMD provides needed ptypes.
> 
> v2:
>   - Add PCI queue/irq config ops.
>   - Move rxq interrupt settings before sending DRIVER OK.
> 
> Historically, virtio PMD can only be binded to igb_uio or
> uio_pci_generic, and not for vfio-pci (iommu group cannot be created as
> vIOMMU is not enabled in QEMU yet).  Besides, quote from
> http://dpdk.org/doc/guides-16.11/rel_notes/release_2_1.html:
>   "Per queue RX interrupt events are only allowed in VFIO
>    which supports multiple MSI-X vectors."
> 
> Linux starts to support VFIO NO-IOMMU mode since 4.8.0. It cannot put
> devices into groups for separation as normal VFIO does. So it does not
> require QEMU to support vIOMMU. But it does inherit other benefits from
> VFIO framework, like better interrupts supports (than UIO). It gives a
> good chance to enable rxq interrupt for virtio PMD.
> 
> To implement it,
> a. Firstly, we need to enable msix. This should be done before DRIVER_OK
> setting and also before queue/irq binding in step b.
> b. Bind queue/irq through portio (legacy devices) or mmio (modern devices).
>    So far, we hard-code 1:1 queue/irq mapping (each rx queue has one
>    exclusive interrupt), like this:
>       vec 0 -> config irq
>       vec 1 -> rxq0
>       vec 2 -> rxq1
>       ...
> 
>     which means, the "vectors" option of QEMU should be configured with
>     a value >= N+1 (N is the number of the queue pairs).
> c. To enable/disable interrupt notification, flags on virtqueues are used
>    to control devices either sending interrupt or not.
> d. Encap above behaviors into callbacks in ether_dev_ops, like
>    rx_queue_intr_enable/rx_queue_intr_disable/rx_descriptor_done etc.
> 
> 
> How to test:
> 
> Step 1, prepare a VM image with kernel version >= 4.8.0, and make sure
> the kernel is compiled with CONFIG_VFIO_NOIOMMU=y.
> 
> Step 2, on the host, start a testpmd with a vhost port:
>   $ testpmd -c 0x7 -m 1024 --vdev 'eth_vhost0,iface=/tmp/sock0,queues=2' \
> 	--no-pci -- -i --rxq=2 --txq=2 --nb-cores=2
> 
> Step 3, boot the VM:
>   $ qemu ... -chardev socket,id=chr1,path=/tmp/sock0 \
> 	-netdev vhost-user,id=net1,chardev=chr1,vhostforce,queues=2 \
> 	-device virtio-net-pci,netdev=net1,mq=on,vectors=5 ...
> 
> Step 4, insert kernel modules
>   $ modprobe vfio enable_unsafe_noiommu_mode=1
>   $ modprobe vfio-pci
> 
> Step 5, start l3fwd-power in VM:
>   $ l3fwd-power -c 0x3 -n 4 -- -p 1 -P --config="(0,0,1),(0,1,1)" \
> 						 --no-numa --parse-ptype
> 
> Step 6, send packets from testpmd on the host:
>   $ start tx_first
> 
> Then l3fwd-power outputs:
>   L3FWD_POWER: lcore 1 is waked up from rx interrupt on port 0 queue 0
>   L3FWD_POWER: lcore 1 is waked up from rx interrupt on port 0 queue 1
> 
> Signed-off-by: Jianfeng Tan <jianfeng.tan at intel.com>
> 
> Jianfeng Tan (10):
>   net/virtio: fix rewriting LSC flag
>   net/virtio: clean up wrapper of set_config_irq
>   net/virtio: add Rx descriptor check
>   net/virtio: add PCI ops for queue/irq binding
>   net/virtio: add Rx queue intr enable/disable functions
>   net/virtio: setup rxq interrupts
>   net/virtio: unbind intr/eventfd when stop device
>   net/virtio: unmapping queue/irq when close device
>   examples/l3fwd-power: add parse-ptype option
>   examples/l3fwd-power: fix not stop and close device
> 
>  drivers/net/virtio/virtio_ethdev.c | 161
> +++++++++++++++++++++++++++++++++++--
>  drivers/net/virtio/virtio_ethdev.h |   3 +
>  drivers/net/virtio/virtio_pci.c    |  31 +++++--
>  drivers/net/virtio/virtio_pci.h    |   5 +-
>  drivers/net/virtio/virtio_rxtx.c   |   9 +++
>  drivers/net/virtio/virtqueue.c     |  11 ---
>  drivers/net/virtio/virtqueue.h     |  16 +++-
>  examples/l3fwd-power/main.c        | 120 ++++++++++++++++++++++++++-
>  8 files changed, 325 insertions(+), 31 deletions(-)
> 
> --
> 2.7.4



More information about the dev mailing list