Bug 448 - Post-copy-Live migration with 8 vhost queues succeeds with warning from QEMU
Summary: Post-copy-Live migration with 8 vhost queues succeeds with warning from QEMU
Status: UNCONFIRMED
Alias: None
Product: DPDK
Classification: Unclassified
Component: vhost/virtio (show other bugs)
Version: 20.02
Hardware: x86 Linux
: Normal minor
Target Milestone: ---
Assignee: Maxime Coquelin
URL:
Depends on:
Blocks:
 
Reported: 2020-04-13 11:29 CEST by Sunil Pai G
Modified: 2020-04-25 02:49 CEST (History)
2 users (show)



Attachments

Description Sunil Pai G 2020-04-13 11:29:19 CEST
Overview:
Even though Post-copy Live migration succeeds, QEMU reports the following warning with 8 vhost queues:
qemu-system-x86_64: vhost_user_set_mem_table_postcopy: Received unexpected msg type.Expected 5 received 30
qemu-system-x86_64: vhost_set_mem_table failed: Resource temporarily unavailable (11)
qemu-system-x86_64: unable to start vhost net: 11: falling back on userspace virtio

Steps to Reproduce:
1.download and build dpdk for x86_64-native-linuxapp-gcc with CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=y on both source and destination host.

2.Set 8192 2MB huge pages on both destination and source host

3.Run DPDK's Testpmd application on source:
./testpmd -m 512 --file-prefix=src -l 0,2 -n 4 --vdev 'net_vhost0,iface=/tmp/vu-src,queues=8' -- --portmask=1 -i --rxq=8 --txq=8 --nb-cores=1 --eth-peer=0,52:54:00:11:22:12 --no-mlockall 

4.Run DPDK's Testpmd application on destination:
./testpmd -m 512 --file-prefix=dst -l 0,2 -n 4 --vdev 'net_vhost0,iface=/tmp/vu-dst,queues=8,postcopy-support=1' -- --portmask=1 -i --rxq=8 --txq=8 --nb-cores=1 --eth-peer=0,52:54:00:11:22:12 --no-mlockall

5.Run QEMU on src host:
export VM_NAME=vhost-vm
export GUEST_MEM=4096M
export VM_PATH=~/Downloads/grpc-2.qcow2
export qemu_queues=8
export qemu_vectors=18
export QEMU=~/qemu-latest/qemu/x86_64-softmmu/qemu-system-x86_64

taskset -c 5-8 $QEMU -name $vhost-vm -cpu host -enable-kvm \
   -m $GUEST_MEM -drive file=$VM_PATH --nographic \
   -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=4 \
   -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \
   -chardev socket,id=char0,path=/tmp/vu-src \
   -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce,queues=$qemu_queues \
   -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mq=on,mrg_rxbuf=on,rx_queue_size=1024,tx_queue_size=1024,vectors=$qemu_vectors \
   -monitor telnet::1234,server,nowait \
   -net user,hostfwd=tcp::10023-:22 -net nic &

6.Run QEMU on dst host:
export VM_NAME=vhost-vm
export GUEST_MEM=4096M
export VM_PATH=~/Downloads/grpc-2.qcow2
export qemu_queues=8
export qemu_vectors=18
export QEMU=~/qemu-latest/qemu/x86_64-softmmu/qemu-system-x86_64

taskset -c 5-8 $QEMU -name $VM_NAME -cpu host -enable-kvm \
   -m $GUEST_MEM -drive file=$VM_PATH --nographic \
   -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=4 \
   -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \
   -chardev socket,id=char0,path=/tmp/vu-src \
   -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce,queues=$qemu_queues \
   -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mq=on,mrg_rxbuf=on,rx_queue_size=1024,tx_queue_size=1024,vectors=$qemu_vectors \
   -monitor telnet::1234,server,nowait \
   -incoming tcp:0:4444 \
   -net user,hostfwd=tcp::10023-:22 -net nic &

7.In both testpmd prompts, start flooding the virtio-net device:
testpmd> set fwd txonly
testpmd> start

8.In destination's Qemu monitor, enable postcopy:
(qemu) migrate_set_capability postcopy-ram on

9.In source's Qemu monitor, enable postcopy and launch migration:
(qemu) migrate_set_capability postcopy-ram on
(qemu) migrate -d tcp:<dst host ip>:4444
(qemu) migrate_start_postcopy


Actual Results:
Even though migration succeeds, on dst host following message is seen on console:
qemu-system-x86_64: vhost_user_set_mem_table_postcopy: Received unexpected msg type.Expected 5 received 30
qemu-system-x86_64: vhost_set_mem_table failed: Resource temporarily unavailable (11)
qemu-system-x86_64: unable to start vhost net: 11: falling back on userspace virtio

Expected Results:
No Warning or error messages from QEMU

Version information:
QEMU-version: master branch [v5.0.0-rc1]
DPDK-version: master branch [v20.02]

Environment Information:
  Operating System: Ubuntu 18.04.4 LTS
            Kernel: Linux 4.15.0-74-generic
      Architecture: x86-64

Additional information:
This error is not seen when number of vhost queues is set to 1
Comment 1 Ajit Khaparde 2020-04-25 02:49:25 CEST
Maxime, Can you please take a look? Thanks

Note You need to log in before you can comment on or make changes to this bug.