[dpdk-dev] [PATCH] virtio: fix the vq size issue

[Xu, Qian Q]

Tested-by: Qian Xu <qian.q.xu at intel.com>
- Test Commit: c55e94f560ef5c9fcee4584952de1d0bd414aece
- OS: Fedora 21
- GCC: gcc (GCC) 4.9.2 20141101 (Red Hat 4.9.2-1)
- CPU: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
- NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
- Target: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
- Total 2 cases(Test Case7 and 8), 2 passed, 0 failed. 

Test Case 1:  test_perf_virtio_one_vm_dpdk_fwd_vhost-cuse_jumboframe
====================================================================

On host:

1. Start up vhost-switch, mergeable 1 means the jubmo frame feature is enabled. vm2vm 0 means only one vm without vm to vm communication::

    taskset -c 1-3 <dpdk_folder>/examples/vhost/build/vhost-switch -c 0xf -n 4 --huge-dir /mnt/huge --socket-mem 1024,1024 -- -p 1 --mergeable 1 --zero-copy 0 --vm2vm 0
   

2. Start VM with vhost cuse as backend::

    taskset -c 4-6  /home/qxu10/qemu-2.2.0/x86_64-softmmu/qemu-system-x86_64 -object memory-backend-file, id=mem,size=2048M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc \
    -enable-kvm -m 2048 -smp 4 -cpu host -name dpdk1-vm1 \
    -drive file=/home/img/dpdk1-vm1.img \
    -netdev tap,id=vhost3,ifname=tap_vhost3,vhost=on,script=no \
    -device virtio-net pci,netdev=vhost3,mac=52:54:00:00:00:01,id=net3,csum=off,gso=off,guest_csum=off,guest_tso4=off,guest_tso6=off,guest_ecn=off \
    -netdev tap,id=vhost4,ifname=tap_vhost4,vhost=on,script=no \
    -device virtio-net-pci,netdev=vhost4,mac=52:54:00:00:00:02,id=net4,csum=off,gso=off,guest_csum=off,guest_tso4=off,guest_tso6=off,guest_ecn=off \
    -netdev tap,id=ipvm1,ifname=tap3,script=/etc/qemu-ifup -device rtl8139,netdev=ipvm1,id=net0,mac=00:00:00:00:00:01 \
    -localtime -nographic

On guest:

3. ensure the dpdk folder copied to the guest with the same config file and build process as host. Then bind 2 virtio devices to igb_uio and start testpmd, below is the step for reference::

    ./<dpdk_folder>/tools/dpdk_nic_bind.py --bind igb_uio 00:03.0 00:04.0

    ./<dpdk_folder>/x86_64-native-linuxapp-gcc/app/test-pmd/testpmd -c f -n 4 -- -i --txqflags 0x0f00 --max-pkt-len 9000 
    
    $ >set fwd mac
    
    $ >start tx_first

4. After typing start tx_first in testpmd, user can see there would be 2 virtio device with MAC and vlan id registered in vhost-user, the log would be shown in host's vhost-sample output.

5. Send traffic(30second) to virtio1 and virtio2, and set the packet size from 64 to 1518 as well as the jumbo frame 3000. Check the performance in Mpps. The traffic sent to virtio1 should have the DEST MAC of Virtio1's MAC, Vlan id of Virtio1. The traffic sent to virtio2 should have the DEST MAC of Virtio2's MAC, Vlan id of Virtio2. As to the functionality criteria, The received rate should not be zero. As to the performance criteria, need check it with developer or design doc/PRD.

Test Case 7:  test_perf_virtio_one_vm_dpdk_fwd_vhost-user_jumboframe
====================================================================

This case is similar to TestCase1, just change the backend from vhost cuse to vhost-user, so need rebuild the dpdk in vhost-user on host, other steps are same as TestCase1. The command to launch vm is different, see below as example:: 

    <qemu-2.2.0_folder>/x86_64-softmmu/qemu-system-x86_64 -name us-vhost-vm1 -cpu host -enable-kvm -m 2048 -object memory-backend-file,id=mem,size=2048M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -smp 2 -drive file=/home/img/dpdk1-vm1.img -chardev socket,id=char0,path=<dpdk/vhost-net -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce -device virtio-net-pci,mac=52:54:00:00:00:01,netdev=mynet1 -chardev socket,id=char1,path=/home/qxu10/dpdk/vhost-net -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce -device virtio-net-pci,mac=52:54:00:00:00:02,netdev=mynet2 -netdev tap,id=ipvm1,ifname=tap3,script=/etc/qemu-ifup -device rtl8139,netdev=ipvm1,id=net0,mac=00:00:00:00:00:09 -nographic

On the guest, need add one parameter at the end of testpmd command line: --disable-hw-vlan-filter.

Test Case 8:  test_perf_virtio_one_vm_dpdk_fwd_vhost-user
=========================================================

This case is similar to TestCase7, just set mergeable=0(disable jumbo frame) when launch the vhost sample, and send the packet size from 64B to 1518B to check the performance and basic functions.

Thanks
Qian


-----Original Message-----
From: Ouyang, Changchun 
Sent: Wednesday, July 01, 2015 3:49 PM
To: dev at dpdk.org
Cc: Cao, Waterman; Xu, Qian Q; Ouyang, Changchun
Subject: [PATCH] virtio: fix the vq size issue

This commit breaks virtio basic packets rx functionality:
  d78deadae4dca240e85054bf2d604a801676becc

The QEMU use 256 as default vring size, also use this default value to calculate the virtio avail ring base address and used ring base address, and vhost in the backend use the ring base address to do packet IO.

Virtio spec also says the queue size in PCI configuration is read-only, so virtio front end can't change it. just need use the read-only value to allocate space for vring and calculate the avail and used ring base address. Otherwise, the avail and used ring base address will be different between host and guest, accordingly, packet IO can't work normally.

Signed-off-by: Changchun Ouyang <changchun.ouyang at intel.com>
---
 drivers/net/virtio/virtio_ethdev.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index fe5f9a1..d84de13 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -263,8 +263,6 @@ int virtio_dev_queue_setup(struct rte_eth_dev *dev,
 	 */
 	vq_size = VIRTIO_READ_REG_2(hw, VIRTIO_PCI_QUEUE_NUM);
 	PMD_INIT_LOG(DEBUG, "vq_size: %d nb_desc:%d", vq_size, nb_desc);
-	if (nb_desc == 0)
-		nb_desc = vq_size;
 	if (vq_size == 0) {
 		PMD_INIT_LOG(ERR, "%s: virtqueue does not exist", __func__);
 		return -EINVAL;
@@ -275,15 +273,9 @@ int virtio_dev_queue_setup(struct rte_eth_dev *dev,
 		return -EINVAL;
 	}
 
-	if (nb_desc < vq_size) {
-		if (!rte_is_power_of_2(nb_desc)) {
-			PMD_INIT_LOG(ERR,
-				     "nb_desc(%u) size is not powerof 2",
-				     nb_desc);
-			return -EINVAL;
-		}
-		vq_size = nb_desc;
-	}
+	if (nb_desc != vq_size)
+		PMD_INIT_LOG(ERR, "Warning: nb_desc(%d) is not equal to vq size (%d), fall to vq size",
+			nb_desc, vq_size);
 
 	if (queue_type == VTNET_RQ) {
 		snprintf(vq_name, sizeof(vq_name), "port%d_rvq%d",
--
1.8.4.2