Bug 241 - QEMU (vIOMMU+virtio) crashes when DPDK exits
Summary: QEMU (vIOMMU+virtio) crashes when DPDK exits
Status: CONFIRMED
Alias: None
Product: DPDK
Classification: Unclassified
Component: vhost/virtio (show other bugs)
Version: 18.11
Hardware: All All
: Normal major
Target Milestone: ---
Assignee: Jianfeng Tan
URL:
Depends on:
Blocks:
 
Reported: 2019-04-08 08:06 CEST by Jianfeng Tan
Modified: 2020-09-16 23:45 CEST (History)
2 users (show)



Attachments

Description Jianfeng Tan 2019-04-08 08:06:57 CEST
This could be a QEMU bug, I record it here as it's convenient to reproduce using DPDK.

QEMU version: v2.10.2/v2.11.2/v2.12.1
DPDK version: v18.11 (the other versions could also have this issue, which I did not test)

The way to start QEMU:

  iommu="-M q35,accel=kvm,kernel-irqchip=split -device intel-iommu,device-iotlb=on,intremap=on,eim=on"

  VIRTIO0="-chardev socket,id=char0,path=/tmp/sock0 -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce -device virtio-net-pci,netdev=netdev0,disable-legacy=on,iommu_platform=on,ats=on"

  qemu ... $iommu $VIRTIO0 ...

Inside VM, we bind virtio to vfio-pci, and start testpmd:
  testpmd -c 3 --in-memory -- -i

And we forcely kill testpmd by:
  kill -9 `pidof testpmd`


QEMU crashes with "Bad ram offset ..."

(gdb) where
#0  0x0000555c004a5648 in qemu_get_ram_block (addr=146033025026) at qemu/exec.c:1114
#1  0x0000555c004a8427 in qemu_map_ram_ptr (ram_block=0x0, addr=146033025026) at qemu/exec.c:2288
#2  0x0000555c004ac9b8 in address_space_lduw_internal_cached (cache=0x7feeb41cf9d0, addr=2, attrs=..., result=0x0, endian=DEVICE_LITTLE_ENDIAN)
    at qemu/memory_ldst.inc.c:281
#3  0x0000555c004acaaf in address_space_lduw_le_cached (cache=0x7feeb41cf9d0, addr=2, attrs=..., result=0x0) at qemu/memory_ldst.inc.c:315
#4  0x0000555c004acb5b in lduw_le_phys_cached (cache=0x7feeb41cf9d0, addr=2) at qemu/memory_ldst.inc.c:334
#5  0x0000555c005844ea in virtio_lduw_phys_cached (vdev=0x555c03ebb180, cache=0x7feeb41cf9d0, pa=2)
    at qemu/include/hw/virtio/virtio-access.h:166
#6  0x0000555c00584d71 in vring_used_idx (vq=0x7ff2c04a4010) at qemu/hw/virtio/virtio.c:262
#7  0x0000555c00589edc in virtio_queue_update_used_idx (vdev=0x555c03ebb180, n=0) at qemu/hw/virtio/virtio.c:2335
#8  0x0000555c0058ff9f in vhost_virtqueue_stop (dev=0x555c02d940c0, vdev=0x555c03ebb180, vq=0x555c02d942e8, idx=0)
    at qemu/hw/virtio/vhost.c:1075
#9  0x0000555c005916ba in vhost_dev_stop (hdev=0x555c02d940c0, vdev=0x555c03ebb180) at qemu/hw/virtio/vhost.c:1557
#10 0x0000555c00560fff in vhost_net_stop_one (net=0x555c02d940c0, dev=0x555c03ebb180) at qemu/hw/net/vhost_net.c:289
#11 0x0000555c00561434 in vhost_net_stop (dev=0x555c03ebb180, ncs=0x555c03ec9870, total_queues=1) at qemu/hw/net/vhost_net.c:368
#12 0x0000555c0055b615 in virtio_net_vhost_status (n=0x555c03ebb180, status=11 '\v') at qemu/hw/net/virtio-net.c:185
#13 0x0000555c0055b8a6 in virtio_net_set_status (vdev=0x555c03ebb180, status=11 '\v') at qemu/hw/net/virtio-net.c:259
#14 0x0000555c00586f0b in virtio_set_status (vdev=0x555c03ebb180, val=11 '\v') at qemu/hw/virtio/virtio.c:1144
#15 0x0000555c0084f2e2 in virtio_write_config (pci_dev=0x555c03eb3010, address=4, val=1283, len=2) at hw/virtio/virtio-pci.c:610
#16 0x0000555c007c1291 in pci_host_config_write_common (pci_dev=0x555c03eb3010, addr=4, limit=256, val=1283, len=2) at hw/pci/pci_host.c:66
#17 0x0000555c007c13b9 in pci_data_write (s=0x555c03092d00, addr=2147489796, val=1283, len=2) at hw/pci/pci_host.c:100
#18 0x0000555c007c14e5 in pci_host_data_write (opaque=0x555c030547a0, addr=0, val=1283, len=2) at hw/pci/pci_host.c:153
#19 0x0000555c00506f01 in memory_region_write_accessor (mr=0x555c03054ba0, addr=0, value=0x7ff2c1fe3838, size=2, shift=0, mask=65535, attrs=...)
    at qemu/memory.c:530
#20 0x0000555c00507119 in access_with_adjusted_size (addr=0, value=0x7ff2c1fe3838, size=2, access_size_min=1, access_size_max=4, access_fn=
    0x555c00506e17 <memory_region_write_accessor>, mr=0x555c03054ba0, attrs=...) at qemu/memory.c:597
#21 0x0000555c00509da2 in memory_region_dispatch_write (mr=0x555c03054ba0, addr=0, data=1283, size=2, attrs=...) at qemu/memory.c:1474
#22 0x0000555c004a9bcb in flatview_write_continue (fv=0x7fee9c3be520, addr=3324, attrs=..., buf=0x7ff2d1ede000 "\003\005", len=2, addr1=0, l=2, mr=0x555c03054ba0) at qemu/exec.c:3094
#23 0x0000555c004a9d47 in flatview_write (fv=0x7fee9c3be520, addr=3324, attrs=..., buf=0x7ff2d1ede000 "\003\005", len=2)
    at qemu/exec.c:3144
#24 0x0000555c004aa125 in address_space_write (as=0x555c0137efe0 <address_space_io>, addr=3324, attrs=..., buf=0x7ff2d1ede000 "\003\005", len=2)
    at qemu/exec.c:3260
#25 0x0000555c004aa176 in address_space_rw (as=0x555c0137efe0 <address_space_io>, addr=3324, attrs=..., buf=0x7ff2d1ede000 "\003\005", len=2, is_write=true)
    at qemu/exec.c:3271
#26 0x0000555c0051fce6 in kvm_handle_io (port=3324, attrs=..., data=0x7ff2d1ede000, direction=1, size=2, count=1)
    at qemu/accel/kvm/kvm-all.c:1730
#27 0x0000555c0052042a in kvm_cpu_exec (cpu=0x555c02e2c6d0) at qemu/accel/kvm/kvm-all.c:1970
#28 0x0000555c004ed00e in qemu_kvm_cpu_thread_fn (arg=0x555c02e2c6d0) at qemu/cpus.c:1215
#29 0x00007ff2ccc7d6ca in start_thread () at /lib64/libpthread.so.0
#30 0x00007ff2cc9b7edf in clone () at /lib64/libc.so.6
Comment 1 Sarosh Arif 2020-03-16 07:07:15 CET
Can you please specify which commands did you run after running testpmd?
Comment 2 Ajit Khaparde 2020-09-16 23:45:55 CEST
Can you respond to Sarosh's questions? Thanks

Note You need to log in before you can comment on or make changes to this bug.