[dpdk-dev] [Bug 29] pktgen hangs when it tries to send packets through libvirt driver, works for all other drivers

[bugzilla at dpdk.org]

https://dpdk.org/tracker/show_bug.cgi?id=29

            Bug ID: 29
           Summary: pktgen hangs when it tries to send packets through
                    libvirt driver, works for all other drivers
           Product: DPDK
           Version: 18.02
          Hardware: x86
                OS: Linux
            Status: CONFIRMED
          Severity: normal
          Priority: Normal
         Component: ethdev
          Assignee: dev at dpdk.org
          Reporter: gregory.shimansky at intel.com
  Target Milestone: ---

Created attachment 6
  --> https://dpdk.org/tracker/attachment.cgi?id=6&action=edit
Vagrantfile and scripts to create test VMs

This is a bug we encounter in NFF-Go project when we try to use pktgen in two
VMs connected directly through internal network. On Qemu/KVM we use UDP tunnel
for internal networks connection, on VirtualBox we use VB internal network.

I attached Vagrantfile and scripts.sh which we use in our project to setup VMs.
You need vagrant version 2.0.x to use this Vagrantfile and a reload plugin for
it. Install it with command "vagrant plugin install vagrant-reload". If you
connect to internet through a proxy, also install a vagrant-proxyconf plugin
with "vagrant plugin install vagrant-proxyconf".

To use libvirt provisioner it is also important to install vagrant-libvirt
plugin ("vagrant plugin install vagrant-libvirt") because otherwise VirtualBox
is used to create VMs. For libvirt we also use "images" storage pool because
default location is often too small to hold all of the VM images. If you don't
need it, comment line 37 in Vagrantfile. Otherwise it is necessary to create it
with the following commands

virsh pool-define-as images dir --target /localdisk/libvirt
virsh pool-start images

After setting up you can created VMs with command "vagrant up
--provider=libvirt". It creates and provisions two VMs, downloads and builds
NFF-Go project which also includes DPDK 18.02 and pktgen 3.4.9. If you used
VirtualBox it is necessary to manually change type of NICs 2 and 3 from Intel
to virtio.

When VMs are created, use "vagrant ssh" to access both of them. There is a
predefined bash function "bindports" which binds two ports which are connected
to each other on these VMs to DPDK igb_uio driver. There is also a function
"runpktgen" bash function to start pktgen for ports 0 and 1.

When pktgen is started, try to use command "start 0" and "start 1". It starts
sending packets but only for a very brief period of time. Quickly it stops
sending packets and only restarting the program allows it to send some more
packets and hang again. This behavior happens only for virtio driver and pktgen
author claims that the cause of the problem is in it
https://github.com/pktgen/Pktgen-DPDK/issues/148#issuecomment-380972000

If you don't want to deal with vagrant, here are two Qemu command lines that
vagrant starts for these two VMs:

libvirt+ 28402     1  4 12:44 ?        00:06:03 /usr/bin/qemu-system-x86_64
-name guest=tests_fedora-1,debug-threads=on -S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-61-tests_fedora-1/master-key.aes
-machine pc-i440fx-bionic,accel=kvm,usb=off,dump-guest-core=off -cpu
Haswell-noTSX,vme=on,ss=on,vmx=on,f16c=on,rdrand=on,hypervisor=on,arat=on,tsc_adjust=on,xsaveopt=on,pdpe1gb=on,abm=on
-m 4096 -realtime mlock=off -smp 8,sockets=8,cores=1,threads=1 -uuid
9cb78faf-9894-4fef-8833-8416689163cb -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-61-tests_fedora-1/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/localdisk/libvirt/tests_fedora-1.img,format=qcow2,if=none,id=drive-virtio-disk0
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-netdev tap,fd=33,id=hostnet0,vhost=on,vhostfd=35 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:b8:95:7f,bus=pci.0,addr=0x5
-netdev socket,udp=127.0.0.1:44403,localaddr=127.0.0.1:44409,id=hostnet1
-device
virtio-net-pci,netdev=hostnet1,id=net1,mac=3c:fd:fe:a4:dd:f0,bus=pci.0,addr=0x6
-netdev socket,udp=127.0.0.1:44404,localaddr=127.0.0.1:44410,id=hostnet2
-device
virtio-net-pci,netdev=hostnet2,id=net2,mac=52:54:00:3b:d5:7d,bus=pci.0,addr=0x7
-chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0
-vnc 127.0.0.1:7 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on

libvirt+ 36753     1  4 12:47 ?        00:05:39 /usr/bin/qemu-system-x86_64
-name guest=tests_fedora-0,debug-threads=on -S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-62-tests_fedora-0/master-key.aes
-machine pc-i440fx-bionic,accel=kvm,usb=off,dump-guest-core=off -cpu
Haswell-noTSX,vme=on,ss=on,vmx=on,f16c=on,rdrand=on,hypervisor=on,arat=on,tsc_adjust=on,xsaveopt=on,pdpe1gb=on,abm=on
-m 4096 -realtime mlock=off -smp 8,sockets=8,cores=1,threads=1 -uuid
1e2c2002-d1e3-4985-a403-254762fbf30d -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-62-tests_fedora-0/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/localdisk/libvirt/tests_fedora-0.img,format=qcow2,if=none,id=drive-virtio-disk0
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=35 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:26:6a:fb,bus=pci.0,addr=0x5
-netdev socket,udp=127.0.0.1:44409,localaddr=127.0.0.1:44403,id=hostnet1
-device
virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:d5:31:ca,bus=pci.0,addr=0x6
-netdev socket,udp=127.0.0.1:44410,localaddr=127.0.0.1:44404,id=hostnet2
-device
virtio-net-pci,netdev=hostnet2,id=net2,mac=52:54:00:f7:b9:48,bus=pci.0,addr=0x7
-chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0
-vnc 127.0.0.1:6 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on

-- 
You are receiving this mail because:
You are the assignee for the bug.