[PATCH] net/virtio: fix the setting of the vector for link state interrupt

[Ma, WenwuX]

Hi Maxime,

> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin at redhat.com>
> Sent: Friday, October 20, 2023 5:01 PM
> To: Ma, WenwuX <wenwux.ma at intel.com>; dev at dpdk.org
> Cc: chenbo.xia at intel.com; Ling, WeiX <weix.ling at intel.com>;
> stable at dpdk.org
> Subject: Re: [PATCH] net/virtio: fix the setting of the vector for link state
> interrupt
> 
> Hi Wenwu,
> 
> Please reword the commit title to something:
> net/virtio: fix link state interrupt vector setting
> 
> On 8/7/23 05:15, Wenwu Ma wrote:
> > The settings of the vector for link state interrupts should be done
> > before the initialization of the device is completed.
> >
> > Fixes: ee85024cf5f7 ("net/virtio: complete init stage at the right
> > place")
> > Cc: stable at dpdk.org
> >
> > Signed-off-by: Wenwu Ma <wenwux.ma at intel.com>
> > ---
> >   drivers/net/virtio/virtio_ethdev.c | 16 ++++++++--------
> >   1 file changed, 8 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/net/virtio/virtio_ethdev.c
> > b/drivers/net/virtio/virtio_ethdev.c
> > index 2c23f1c00e..1801b0ae47 100644
> > --- a/drivers/net/virtio/virtio_ethdev.c
> > +++ b/drivers/net/virtio/virtio_ethdev.c
> > @@ -1912,6 +1912,14 @@ virtio_init_device(struct rte_eth_dev *eth_dev,
> uint64_t req_features)
> >   		}
> >   	}
> >
> > +	if (eth_dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC)
> > +		/* Enable vector (0) for Link State Interrupt */
> > +		if (VIRTIO_OPS(hw)->set_config_irq(hw, 0) ==
> > +				VIRTIO_MSI_NO_VECTOR) {
> > +			PMD_DRV_LOG(ERR, "failed to set config vector");
> > +			return -EBUSY;
> > +		}
> > +
> >   	virtio_reinit_complete(hw);
> >
> >   	return 0;
> > @@ -2237,14 +2245,6 @@ virtio_dev_configure(struct rte_eth_dev *dev)
> >   	hw->has_tx_offload = tx_offload_enabled(hw);
> >   	hw->has_rx_offload = rx_offload_enabled(hw);
> >
> > -	if (dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC)
> > -		/* Enable vector (0) for Link State Interrupt */
> > -		if (VIRTIO_OPS(hw)->set_config_irq(hw, 0) ==
> > -				VIRTIO_MSI_NO_VECTOR) {
> > -			PMD_DRV_LOG(ERR, "failed to set config vector");
> > -			return -EBUSY;
> > -		}
> > -
> >   	if (virtio_with_packed_queue(hw)) {
> >   #if defined(RTE_ARCH_X86_64) && defined(CC_AVX512_SUPPORT)
> >   		if ((hw->use_vec_rx || hw->use_vec_tx) &&
> 
> It looks good to me, so I can change the title myself while applying if Ok for
> you.
> 
I will submit a new patch with your reworded title.

> Reviewed-by: Maxime Coquelin <maxime.coquelin at redhat.com>
> 
> By the way, can you tell me with which backends have you tested it with?
> Only Virtio-PCI? Or also Virtio-user?
> 
Test step:

1.Bind 1 NIC port to vfio-pci driver:

dpdk-devbind.py --force --bind=vfio-pci 0000:4b:00.0

2.Start dpdk-testpmd as back-end:

x86_64-native-linuxapp-gcc/app/dpdk-testpmd  -l 1-5 -n 8 -a 0000:4b:00.0  \
--vdev net_vhost0,iface=/root/dpdk/vhost-net,queues=4    \
-- -i --nb-cores=4 --rxq=4 --txq=4 --rss-ip
testpmd>start

3.Start VM with QEMU-8.0.0 as front-end:

taskset -c 20,21,22,23,24,25,26,27 /home/QEMU/qemu-8.0.0/bin/qemu-system-x86_64  -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
-daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait -netdev user,id=nttsip1,hostfwd=tcp:10.239.252.245:6000-:22 -device e1000,netdev=nttsip1  \
-cpu host -smp 4 -m 8192 -object memory-backend-file,id=mem,size=8192M,mem-path=/dev/hugepages,share=on -numa node,memdev=mem -mem-prealloc \
-chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 \
-drive file=/home/image/ubuntu2004.img -chardev socket,id=char0,path=/root/dpdk/vhost-net -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce,queues=4 \
-device virtio-net-pci,netdev=netdev0,mac=00:11:22:33:44:55,disable-modern=true,mrg_rxbuf=on,csum=on,mq=on,vectors=10

4.SSH connect VM and build dpdk-l3fwd-power APP, and then start dpdk-l3fwd-power:

CC=gcc meson -Denable_kmods=True -Dlibdir=lib  --default-library=static x86_64-native-linuxapp-gcc
ninja -C x86_64-native-linuxapp-gcc
meson configure -Dexamples=l3fwd-power x86_64-native-linuxapp-gcc
ninja -C x86_64-native-linuxapp-gcc

echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
dpdk-devbind.py -b vfio-pci 0000:00:05.0

./x86_64-native-linuxapp-gcc/examples/dpdk-l3fwd-power  -c 0xf -n 4 --log-level='user1,7' -- -p 1 -P --config '(0,0,0),(0,1,1),(0,2,2),(0,3,3)' --no-numa  --parse-ptype --interrupt-only

The VM will crash, when start dpdk-l3fwd-power APP in VM with QEMU-8.0.0, and it works well when start VM with other QEMU version less than QEMU-8.0.0.

> Thanks,
> Maxime