[dpdk-dev] [PATCH v2 00/12] net/virtio: add offload support
Yuanhan Liu
yuanhan.liu at linux.intel.com
Tue Oct 11 13:35:55 CEST 2016
Hi,
Firstly, apologize for so late review. It's been forgotten :(
BTW, please feel free to ping me in future if I made no response
in one or two weeks!
I haven't reviewed it carefully yet (something I will do tomorrow).
Before that, few quick questions.
Firstly, would you write down some test steps? Honestly, I'm not
quite sure how that works without the TCP/IP stack.
On Mon, Oct 03, 2016 at 11:00:11AM +0200, Olivier Matz wrote:
> This patchset, targetted for 16.11, introduces the support of rx and tx
> offload in virtio pmd. To achieve this, some new mbuf flags must be
> introduced, as discussed in [1].
>
> It applies on top of:
> - software packet type [2]
> - testpmd enhancements [3]
I didn't do the search. Have the two got merged?
--yliu
>
> The new mbuf checksum flags are backward compatible for current
> applications that assume that unknown_csum = good_cum (since there
> was only a bad_csum flag). But it the patchset is integrated, we
> should consider updating the PMDs to match the new API for 16.11.
>
> [1] http://dpdk.org/ml/archives/dev/2016-May/039920.html
> [2] http://dpdk.org/ml/archives/dev/2016-October/048073.html
> [3] http://dpdk.org/ml/archives/dev/2016-September/046443.html
>
> changes v1 -> v2
> - change mbuf checksum calculation static inline
> - fix checksum calculation for protocol where csum=0 means no csum
> - move mbuf checksum calculation in librte_net
> - use RTE_MIN() to set max rx/tx queue
> - rebase on top of head
>
> Olivier Matz (12):
> virtio: move device initialization in a function
> virtio: setup and start cq in configure callback
> virtio: reinitialize the device in configure callback
> net: add function to calculate a checksum in a mbuf
> mbuf: add new Rx checksum mbuf flags
> app/testpmd: fix checksum stats in csum engine
> mbuf: new flag for LRO
> app/testpmd: display lro segment size
> virtio: add Rx checksum offload support
> virtio: add Tx checksum offload support
> virtio: add Lro support
> virtio: add Tso support
>
> app/test-pmd/csumonly.c | 8 +-
> doc/guides/rel_notes/release_16_11.rst | 16 ++
> drivers/net/virtio/virtio_ethdev.c | 182 +++++++++++++---------
> drivers/net/virtio/virtio_ethdev.h | 18 +--
> drivers/net/virtio/virtio_pci.h | 4 +-
> drivers/net/virtio/virtio_rxtx.c | 270 ++++++++++++++++++++++++++++++---
> drivers/net/virtio/virtqueue.h | 1 +
> lib/librte_mbuf/rte_mbuf.c | 18 ++-
> lib/librte_mbuf/rte_mbuf.h | 58 ++++++-
> lib/librte_net/rte_ip.h | 60 ++++++++
> 10 files changed, 526 insertions(+), 109 deletions(-)
>
> Test plan
> =========
>
> (not fully replayed on v2, but no major change)
>
> Platform description
> --------------------
>
> guest (dpdk)
> +----------------+
> | |
> | |
> | port0 +-----<---+
> | ixgbe / | |
> | directio | |
> | | |
> | port1 | ^ flow1
> +----------------+ | (flow2 is the reverse)
> | |
> | virtio |
> v |
> +----------------+ |
> | tap0 / | |
> |1.1.1.1 / | |
> |ns-tap / | |
> | / | |
> | / ixgbe2 +------>--+
> | / 1.1.1.2 |
> |/ ns-ixgbe |
> +----------------+
> host (linux, vhost-net)
>
>
> flow1:
> host -(ixgbe)-> guest -(virtio)-> host
> 1.1.1.2 -> 1.1.1.1
>
> flow2:
> host -(virtio)-> guest -(ixgbe)-> host
> 1.1.1.2 -> 1.1.1.1
>
> Host configuration
> ------------------
>
> Start qemu with:
>
> - a ne2k management interface to avoi any conflict with dpdk
> - 2 ixgbe interfaces given to with vm through vfio
> - a virtio net device, connected to a tap interface through vhost-net
>
> /usr/bin/qemu-system-x86_64 -k fr -daemonize --enable-kvm -m 1G -cpu host \
> -smp 3 -serial telnet::40564,server,nowait -serial null \
> -qmp tcp::44340,server,nowait -monitor telnet::49229,server,nowait \
> -device ne2k_pci,mac=de:ad:de:01:02:03,netdev=user.0,addr=03 \
> -netdev user,id=user.0,hostfwd=tcp::34965-:22 \
> -device vfio-pci,host=0000:04:00.0 -device vfio-pci,host=0000:04:00.1 \
> -netdev type=tap,id=vhostnet0,script=no,vhost=on,queues=8 \
> -device virtio-net-pci,netdev=vhostnet0,ioeventfd=on,mq=on,vectors=17 \
> -hda "/path/to/ubuntu-14.04-template.qcow2" \
> -snapshot -vga none -display none
>
> Move the tap interface in a netns, and configure it:
>
> ip netns add ns-tap
> ip netns exec ns-tap ip l set lo up
> ip link set tap0 netns ns-tap
> ip netns exec ns-tap ip l set tap0 down
> ip netns exec ns-tap ip l set addr 02:00:00:00:00:01 dev tap0
> ip netns exec ns-tap ip l set tap0 up
> ip netns exec ns-tap ip a a 1.1.1.1/24 dev tap0
> ip netns exec ns-tap arp -s 1.1.1.2 02:00:00:00:00:00
> ip netns exec ns-tap ip a
>
> Move the ixgbe interface in a netns, and configure it:
>
> IXGBE=ixgbe2
> ip netns add ns-ixgbe
> ip netns exec ns-ixgbe ip l set lo up
> ip link set ${IXGBE} netns ns-ixgbe
> ip netns exec ns-ixgbe ip l set ${IXGBE} down
> ip netns exec ns-ixgbe ip l set addr 02:00:00:00:00:00 dev ${IXGBE}
> ip netns exec ns-ixgbe ip l set ${IXGBE} up
> ip netns exec ns-ixgbe ip a a 1.1.1.2/24 dev ${IXGBE}
> ip netns exec ns-ixgbe arp -s 1.1.1.1 02:00:00:00:00:01
> ip netns exec ns-ixgbe ip a
>
> Guest configuration
> -------------------
>
> List of pci devices:
>
> 00:02.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
> 00:03.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL-8029(AS) [10ec:8029]
> 00:04.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
> 00:05.0 Ethernet controller [0200]: Red Hat, Inc Virtio network device [1af4:1000]
>
> Compile dpdk:
>
> cd dpdk.org
> make config T=x86_64-native-linuxapp-gcc
> make -j4
>
> Prepare environment:
>
> mkdir -p /mnt/huge
> mount -t hugetlbfs nodev /mnt/huge
> echo 256 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
> modprobe uio_pci_generic
> python tools/dpdk_nic_bind.py -b uio_pci_generic 0000:00:02.0
> python tools/dpdk_nic_bind.py -b uio_pci_generic 0000:00:05.0
>
> Run test
> ========
>
> The test uses iperf to validate connectivity between the 2 netns of the
> host and trough the guest.
>
> Iperf is run with:
>
> # flow1: host -(ixgbe)-> guest -(virtio)-> host
> ip netns exec ns-tap iperf -s
> ip netns exec ns-ixgbe iperf -c 1.1.1.1 -t 10
>
> # flow2: host -(virtio)-> guest -(ixgbe)-> host
> ip netns exec ns-ixgbe iperf -s
> ip netns exec ns-tap iperf -c 1.1.1.2 -t 10
>
> The guest runs testpmd with csum forward engine, its configuration
> depends on the test case.
>
> test1: large packets (lro/tso)
> ------------------------------
>
> Configuration of testpmd:
>
> ./build/app/testpmd -l 0,1 --log-level 8 -- --total-num-mbufs=16384 \
> -i --port-topology=chained --disable-hw-vlan-filter \
> --disable-hw-vlan-strip --enable-rx-cksum --enable-lro \
> --crc-strip --txqflags=0
>
> set fwd csum
> tso set 1440 0
> csum set ip hw 0
> csum set tcp hw 0
> tso set 1440 1
> #csum set ip hw 1 # not supported by virtio
> csum set tcp hw 1
> start
>
> Iperf log:
>
> root at ubuntu1404:~# ip netns exec ns-ixgbe iperf -c 1.1.1.1 -t 10
> ------------------------------------------------------------
> Client connecting to 1.1.1.1, TCP port 5001
> TCP window size: 85.0 KByte (default)
> ------------------------------------------------------------
> [ 3] local 1.1.1.2 port 54460 connected with 1.1.1.1 port 5001
> [ ID] Interval Transfer Bandwidth
> [ 3] 0.0-10.0 sec 6.14 GBytes 5.27 Gbits/sec
> root at ubuntu1404:~# ip netns exec ns-tap iperf -c 1.1.1.2 -t 10
> ------------------------------------------------------------
> Client connecting to 1.1.1.2, TCP port 5001
> TCP window size: 85.0 KByte (default)
> ------------------------------------------------------------
> [ 3] local 1.1.1.1 port 58312 connected with 1.1.1.2 port 5001
> [ ID] Interval Transfer Bandwidth
> [ 3] 0.0-10.0 sec 6.70 GBytes 5.76 Gbits/sec
>
> Example of what we see with "set verbose 1" in testpmd:
>
> -- flow1: ixgbe2 -> port0 (ixgbe) -> testpmd -> port1 (virtio) <-> tap0
> port=0, mbuf=0x7f968ad9fdc0, pkt_len=24682, nb_segs=13:
> rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_UNKNOWN
> tx: m->l2_len=14 m->l3_len=20 m->l4_len=32
> tx: m->tso_segsz=1440
> tx: flags=PKT_TX_IP_CKSUM PKT_TX_L4_NO_CKSUM PKT_TX_TCP_SEG PKT_TX_IPV4
>
> -- flow2: tap0 -> port1 (virtio)-> testpmd -> port0 (ixgbe) -> ixgbe2
> port=1, mbuf=0x7f968acc9f40, pkt_len=42058, nb_segs=21:
> rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_NONE PKT_RX_IP_CKSUM_UNKNOWN PKT_RX_LRO
> rx: m->lro_segsz=1440
> tx: m->l2_len=14 m->l3_len=20 m->l4_len=32
> tx: m->tso_segsz=1440
> tx: flags=PKT_TX_IP_CKSUM PKT_TX_L4_NO_CKSUM PKT_TX_TCP_SEG PKT_TX_IPV4
>
> test2: hardware checksum only
> -----------------------------
>
> Configuration of testpmd:
>
> ./build/app/testpmd -l 0,1 --log-level 8 -- --total-num-mbufs=16384 \
> -i --port-topology=chained --disable-hw-vlan-filter \
> --disable-hw-vlan-strip --enable-rx-cksum --crc-strip --txqflags=0
>
> set fwd csum
> csum set ip hw 0
> csum set tcp hw 0
> csum set tcp hw 1
> start
>
> Iperf log:
>
> root at ubuntu1404:~# ip netns exec ns-ixgbe iperf -c 1.1.1.1 -t 10
> ------------------------------------------------------------
> Client connecting to 1.1.1.1, TCP port 5001
> TCP window size: 85.0 KByte (default)
> ------------------------------------------------------------
> [ 3] local 1.1.1.2 port 54462 connected with 1.1.1.1 port 5001
> [ ID] Interval Transfer Bandwidth
> [ 3] 0.0-10.0 sec 4.49 GBytes 3.86 Gbits/sec
> root at ubuntu1404:~# ip netns exec ns-tap iperf -c 1.1.1.2 -t 10
> ------------------------------------------------------------
> Client connecting to 1.1.1.2, TCP port 5001
> TCP window size: 85.0 KByte (default)
> ------------------------------------------------------------
> [ 3] local 1.1.1.1 port 58314 connected with 1.1.1.2 port 5001
> [ ID] Interval Transfer Bandwidth
> [ 3] 0.0-10.0 sec 6.66 GBytes 5.72 Gbits/sec
>
> Example of what we see with "set verbose 1" in testpmd:
>
> -- flow1: ixgbe2 -> port0 (ixgbe) -> testpmd -> port1 (virtio) <-> tap0
> port=0, mbuf=0x7f0adca89b40, pkt_len=1514, nb_segs=1:
> rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_UNKNOWN
> tx: m->l2_len=14 m->l3_len=20 m->l4_len=32
> tx: flags=PKT_TX_TCP_CKSUM PKT_TX_IPV4
>
> -- flow2: tap0 -> port1 (virtio)-> testpmd -> port0 (ixgbe) -> ixgbe2
> port=1, mbuf=0x7f0adcb98d80, pkt_len=1514, nb_segs=1:
> rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_NONE PKT_RX_IP_CKSUM_UNKNOWN
> tx: m->l2_len=14 m->l3_len=20 m->l4_len=32
> tx: flags=PKT_TX_IP_CKSUM PKT_TX_TCP_CKSUM PKT_TX_IPV4
>
> test3: no offload
> -----------------
>
> Configuration of testpmd:
>
> ./build/app/testpmd -l 0,1 --log-level 8 -- --total-num-mbufs=16384 \
> -i --port-topology=chained --disable-hw-vlan-filter --disable-hw-vlan-strip
>
> set fwd csum
> start
>
> Iperf log:
>
> root at ubuntu1404:~# ip netns exec ns-ixgbe iperf -c 1.1.1.1 -t 10
> ------------------------------------------------------------
> Client connecting to 1.1.1.1, TCP port 5001
> TCP window size: 85.0 KByte (default)
> ------------------------------------------------------------
> [ 3] local 1.1.1.2 port 54466 connected with 1.1.1.1 port 5001
> [ ID] Interval Transfer Bandwidth
> [ 3] 0.0-10.0 sec 4.29 GBytes 3.68 Gbits/sec
> root at ubuntu1404:~# ip netns exec ns-tap iperf -c 1.1.1.2 -t 10
> ------------------------------------------------------------
> Client connecting to 1.1.1.2, TCP port 5001
> TCP window size: 85.0 KByte (default)
> ------------------------------------------------------------
> [ 3] local 1.1.1.1 port 58316 connected with 1.1.1.2 port 5001
> [ ID] Interval Transfer Bandwidth
> [ 3] 0.0-10.0 sec 6.66 GBytes 5.72 Gbits/sec
>
> Example of what we see with "set verbose 1" in testpmd:
>
> -- flow1: ixgbe2 -> port0 (ixgbe) -> testpmd -> port1 (virtio) <-> tap0
> port=0, mbuf=0x7faf38b3e700, pkt_len=1514, nb_segs=1:
> rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_UNKNOWN
> tx: flags=PKT_TX_L4_NO_CKSUM PKT_TX_IPV4
>
> -- flow2: tap0 -> port1 (virtio)-> testpmd -> port0 (ixgbe) -> ixgbe2
> port=1, mbuf=0x7faf38b71500, pkt_len=1514, nb_segs=1:
> rx: l2_len=14 ethertype=800 l3_len=20 l4_proto=6 l4_len=32 flags=PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_UNKNOWN
> tx: flags=PKT_TX_L4_NO_CKSUM PKT_TX_IPV4
>
> --
> 2.8.1
More information about the dev
mailing list