[dts] [DTS]Patch v1 2/2: test plan for avf

[Liu, Yong]

Hi Fei,
Overall you need add more information in your test plan. You need to describe what is testing and which is need to check in the test plan.
And overall format need align to our rule. Please reference to other plans like vlan_ethertype_config_test_plan.

Thanks,
Marvin

> -----Original Message-----
> From: dts [mailto:dts-bounces at dpdk.org] On Behalf Of Wang Fei
> Sent: Monday, June 25, 2018 9:47 AM
> To: dts at dpdk.org
> Cc: Wang, FeiX Y <feix.y.wang at intel.com>
> Subject: [dts] [DTS]Patch v1 2/2: test plan for avf
> 
> Signed-off-by: Wang Fei <feix.y.wang at intel.com>
> ---
>  test_plans/avf_test_plan.rst | 518
> +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 518 insertions(+)
>  create mode 100644 test_plans/avf_test_plan.rst
> 
> diff --git a/test_plans/avf_test_plan.rst b/test_plans/avf_test_plan.rst
> new file mode 100644
> index 0000000..781f97d
> --- /dev/null
> +++ b/test_plans/avf_test_plan.rst
> @@ -0,0 +1,518 @@
> +# BSD LICENSE
> +#
> +# Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
> +# All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without
> +# modification, are permitted provided that the following conditions
> +# are met:
> +#
> +#   * Redistributions of source code must retain the above copyright
> +#     notice, this list of conditions and the following disclaimer.
> +#   * Redistributions in binary form must reproduce the above copyright
> +#     notice, this list of conditions and the following disclaimer in
> +#     the documentation and/or other materials provided with the
> +#     distribution.
> +#   * Neither the name of Intel Corporation nor the names of its
> +#     contributors may be used to endorse or promote products derived
> +#     from this software without specific prior written permission.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +
> +==============
> +AVF test plan:
> +==============
> +
> +Description
> +===========
> +
> +Prerequisites
> +=============
> +
> +Hardwares:
> +----------
> +i40e (Fortville XXV710)

Not sure "i40e" meaning here, you can just describe like "Fortville family Ethernet card."

> +
> +1. apply below patch to dpdk source code
> +
> +diff --git a/drivers/net/avf/base/avf_devids.h
> b/drivers/net/avf/base/avf_devids.h
> +index 7d9fed2..3eff68c 100644
> +--- a/drivers/net/avf/base/avf_devids.h
> ++++ b/drivers/net/avf/base/avf_devids.h
> +@@ -38,6 +38,6 @@ POSSIBILITY OF SUCH DAMAGE.
> + #define AVF_INTEL_VENDOR_ID            0x8086
> +
> + /* Device IDs */
> +-#define AVF_DEV_ID_ADAPTIVE_VF         0x1889
> ++#define AVF_DEV_ID_ADAPTIVE_VF         0x154c
> +
> + #endif /* _AVF_DEVIDS_H_ */
> +diff --git a/drivers/net/i40e/base/i40e_devids.h
> b/drivers/net/i40e/base/i40e_devids.h
> +index 66ff1cc..a473c89 100644
> +--- a/drivers/net/i40e/base/i40e_devids.h
> ++++ b/drivers/net/i40e/base/i40e_devids.h
> +@@ -52,7 +52,7 @@ POSSIBILITY OF SUCH DAMAGE.
> + #define I40E_DEV_ID_25G_B              0x158A
> + #define I40E_DEV_ID_25G_SFP28          0x158B
> + #if defined(INTEGRATED_VF) || defined(VF_DRIVER) ||
> defined(I40E_NDIS_SUPPORT)
> +-#define I40E_DEV_ID_VF                 0x154C
> ++#define I40E_DEV_ID_VF                 0x164C
> + #define I40E_DEV_ID_VF_HV              0x1571
> + #define I40E_DEV_ID_ADAPTIVE_VF                0x1889
> + #endif /* VF_DRIVER */
> +

Do not need to attach patch here, you can save it in dep folder. 
Just mention that which patch is needed is enough. 

> +2,
> +make install RTE_SDK=`pwd` T=x86_64-native-linuxapp-gcc -j32
> +modprobe uio;
> +insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko;
> +

You can simply these descriptions like "bind PF device to igb_uio.ko".
Not need to emphasize things that everyone known about. 

> +./usertools/dpdk-devbind.py --bind=igb_uio 05:00.0 07:00.0;
> +echo 1 > /sys/bus/pci/devices/0000\:05\:00.0/max_vfs;
> +echo 1 > /sys/bus/pci/devices/0000\:07\:00.0/max_vfs;
> +
> +./usertools/dpdk-devbind.py --bind=vfio-pci 05:02.0 07:02.0
> +or
> +(./usertools/dpdk-devbind.py --bind=igb_uio 05:02.0 07:02.0)
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 1-5 -n 4 --socket-
> mem=1024,1024 --file-prefix=pf -w 05:00.0 -w 07:00.0 -- -i
> +

Add some description for those commands like "Set VF0 and VF1 mac addresses".

> +set vf mac addr 0 0 00:12:34:56:78:01
> +set vf mac addr 1 0 00:12:34:56:78:02
> +

Add some description for those commands.

> +./x86_64-native-linuxapp-gcc/app/testpmd -l 6-10 -n 4 --master-lcore=6  -
> -socket-mem=1024,1024 --file-prefix=vf \
> +-w 05:02.0 -w 07:02.0 -- -i --txq=4 --rxq=4  --nb-cores=4
> +
> +
> +Detail test plan about each features :
> +=====================================
> +

Please separated into different test case. Each test case name should have "=" underlined. 

> +1. VF basic RX/TX
> +-----------------
> +
> +send 2000 random packets from traffic generator, verify the packets can
> be received
> +by one VF and can be forward to another VF correctly.
> +
> +2. VF mac filter
> +----------------

This case should be test case two, named it as "Test case: AVF VF MAC Filter".
> +
> +1) unicast test
> +
> +testpmd>
> +set promisc all off
> +set allmulti all off
> +
> +mac_addr set 0 00:12:34:56:78:03 [not work]
> +
> +disable promisc mode, verify VF packet rx/tx can work fine with the
> specific mac addr.

Description should be in previous of commands. 
Please add more details about how to check rx/tx work. 

> +
> +2) multicast test
> +
> +testpmd>
> +set promisc all off
> +set allmulti all off
> +
> +Send packet with multicast MAC 01:80:C2:00:00:08, and check VF can not
> receive the packet.
> +
> +testpmd>
> +set allmulti all on
> +
> +Send packet with multicast MAC 01:80:C2:00:00:08, and check VF can
> receive the packet.
> +
> +3) broadcast test
> +
> +testpmd>
> +set promisc all off
> +
> +Send packets with broadcast address ff:ff:ff:ff:ff:ff, and check VF can
> receive the packet
> +
> +4) promiscuous mode
> +
> +testpmd>
> +set promisc all on
> +
> +Send packet that different with vf mac, check packets can be received.
> +
> +5) mac spoof on (N/A)
> +
> +3. VF vlan
> +----------
> +
> +1) Vlan filter (vlan filter only work with promisc mode off)
> +
> +port stop all
> +set promisc all off
> +vlan set filter off 0
> +vlan set filter off 1
> +vlan set strip off 0
> +vlan set strip off 1
> +
> +vlan set filter on 0
> +set fwd mac
> +port start all
> +start
> +
> +packet with vlan can not be received, packet without vlan packet can be
> received.
> +
> +2) rx_vlan (vlan filter only work with promisc mode off)
> +
> +setup rx vlan like the follow command, verify packet with
> +vlan_id can receive by VF, and packet with other vlan_id can not be
> received by VF
> +(execept vlan 0).
> +`testpmd> rx_vlan add <vlan_id> <vf_index>`
> +
> +port stop all
> +set promisc all off
> +vlan set filter off 0
> +vlan set filter off 1
> +vlan set strip off 0
> +vlan set strip off 1
> +
> +vlan set filter on 0
> +rx_vlan add 20 0
> +set fwd mac
> +port start all
> +start
> +
> +3) tx_vlan, setup VF vlan like the follow command, verify packet
> +that out from VF contain the vlan tag.
> +
> +`testpmd> tx_vlan <vf_index> <vlan_id>`
> +
> +port stop all
> +set promisc all on
> +set fwd mac
> +vlan set filter off 0
> +vlan set filter off 1
> +vlan set strip off 0
> +vlan set strip off 1
> +tx_vlan set 1 20
> +port start all
> +start
> +
> +4) vlan strip, setup strip on, vlan promisc on, verify vlan_id
> +was stripped by VF when it forward the packet to other vf port.
> +
> +`testpmd> vlan set strip on <vf_index>`
> +
> +port stop all
> +set promisc all on
> +set fwd mac
> +vlan set filter off 0
> +vlan set filter off 1
> +vlan set strip off 0
> +vlan set strip off 1
> +vlan set strip on 0
> +port start all
> +start
> +
> +5) vlan promisc mode,
> +
> +port stop all
> +vlan set filter off 0
> +vlan set filter off 1
> +vlan set strip off 0
> +vlan set strip off 1
> +
> +set promisc all on
> +set fwd mac
> +port start all
> +start
> +
> +send packet with vlan or without vlan, both can be received and forwarded.
> +
> +6) vlan spoof on (N/A)
> +
> +
> +4. VF jumboframe
> +----------------
> +
> +1) Check that no jumbo frame support,
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 6-10 -n 4 --master-lcore=6 --
> socket-mem=1024,1024 --file-prefix=vf \
> +-w 05:02.0 -w 07:02.0 -- -i --txq=4 --rxq=4  --nb-cores=4
> +testpmd>
> +set fwd mac
> +start
> +
> +packet less than the standard maximum frame (1518) can be received.
> +packet more than the standard maximum frame (1518) can not be received.
> +
> +2)Check that with Jumbo Frames support,
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 6-10 -n 4 --master-lcore=6 --
> socket-mem=1024,1024 --file-prefix=vf \
> +-w 05:02.0 -w 07:02.0 -- -i --txq=4 --rxq=4  --nb-cores=4 --max-pkt-
> len=9000 --tx-offloads=0x8000
> +testpmd>
> +set fwd mac
> +start
> +
> +packet lengths greater than the standard maximum frame (1518) and \
> +lower or equal to the maximum frame length can be received.
> +Check that packets larger than the configured maximum packet length are
> +dropped by the hardware.
> +
> +Note the following was expected!
> +packet size > 9001,  not forward , but RX-packets: counter increased
> (issue ??????)
> +
> +5. VF RSS
> +---------
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 6-10 -n 4 --master-lcore=6 --
> socket-mem=1024,1024 --file-prefix=vf \
> +-w 05:02.0 -w 07:02.0 -- -i --txq=4 --rxq=4  --nb-cores=4 --txqflags=0
> +
> +1) test redirection Table Config,
> +config hash reta table, send different flow type packet to VF port,
> +check packet received by different queue.
> +
> +port config 0 rss reta (0,0)
> +port config 0 rss reta (1,1)
> +port config 0 rss reta (2,2)
> +port config 0 rss reta (3,3)
> +port config 0 rss reta (60,0)
> +port config 0 rss reta (61,1)
> +port config 0 rss reta (62,2)
> +port config 0 rss reta (63,3)
> +
> +testpmd> port config all rss
> (all|ip|tcp|udp|sctp|ether|port|vxlan|geneve|nvgre|none)
> +
> +
> +6. VF offload (checksum and tso)
> +--------------------------------
> +1) enable HW checksum offload on VF port,
> +Send packets with incorrect checksum to vf port, verify that the packets
> +can be received by VF port and checksum error reported,
> +the packets forwarded by VF port have the correct checksum value.
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 6-10 -n 4 --master-lcore=6 --
> socket-mem=1024,1024 --file-prefix=vf \
> +-w 05:02.0 -w 07:02.0 -- -i --txq=4 --rxq=4  --nb-cores=4
> +
> +port stop all
> +
> +csum set ip hw 0
> +csum set udp hw 0
> +csum set tcp hw 0
> +csum set sctp hw 0
> +
> +csum set ip hw 1
> +csum set udp hw 1
> +csum set tcp hw 1
> +csum set sctp hw 1
> +
> +set fwd csum
> +set verbose 1
> +
> +port start all
> +start
> +
> +2) SW checksum, disable HW checksum offload on VF port,
> +Send packets with incorrect checksum to vf port, verify that the packets
> +can be received by VF port and checksum error reported, the packets
> +forwarded by VF port have the correct checksum value.
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 6-10 -n 4 --master-lcore=6 --
> socket-mem=1024,1024 --file-prefix=vf \
> +-w 05:02.0 -w 07:02.0 -- -i --txq=4 --rxq=4  --nb-cores=4
> +
> +port stop all
> +csum set ip sw 0
> +csum set udp sw 0
> +csum set tcp sw 0
> +csum set sctp sw 0
> +
> +csum set ip sw 1
> +csum set udp sw 1
> +csum set tcp sw 1
> +csum set sctp sw 1
> +
> +set fwd csum
> +set verbose 1
> +port start all
> +start
> +
> +3) tso, enable tso on VF port, verify tcp packets that send out by
> +VF port was splitted according to tso size.  [no implemented yet]
> +
> +port stop all
> +set verbose 1
> +csum set ip hw 0
> +csum set udp hw 0
> +csum set tcp hw 0
> +csum set sctp hw 0
> +
> +csum set ip hw 1
> +csum set udp hw 1
> +csum set tcp hw 1
> +csum set sctp hw 1
> +
> +tso set 800 1
> +set fwd csum
> +port start all
> +start
> +
> +3) tunnel tso, enable tunnel tso on VF port, verify tcp packets that send
> out by
> +VF port was splitted according to tso size. [not implemented yet]
> +
> +port stop all
> +set verbose 1
> +csum set ip hw 0
> +csum set udp hw 0
> +csum set tcp hw 0
> +csum set sctp hw 0
> +csum set outer-ip hw 0
> +csum parse_tunnel on 0
> +
> +csum set ip hw 1
> +csum set udp hw 1
> +csum set tcp hw 1
> +csum set sctp hw 1
> +csum set outer-ip hw 1
> +csum parse_tunnel on 1
> +
> +tunnel_tso set 800 1
> +set fwd csum
> +port start all
> +start
> +
> +7. Rx interrupt
> +---------------
> +1). build l3fwd-power
> +make -C examples/l3fwd-power RTE_SDK=`pwd` T=x86_64-native-linuxapp-gcc
> +
> +2). build two VFs with igb_uio  [fail]
> +modprobe uio;
> +insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko;
> +./usertools/dpdk-devbind.py --bind=igb_uio 05:02.0 07:02.0
> +
> +if in VM, enable vfio noiommu
> +
> +modprobe -r vfio_iommu_type1
> +modprobe -r vfio
> +modprobe  vfio enable_unsafe_noiommu_mode=1
> +cat /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
> +modprobe vfio-pci
> +
> +3). start l3fwd power with one queue per port.
> +./examples/l3fwd-power/build/l3fwd-power -l 6,7 -n 4 -w 05:02.0 -w
> 07:02.0 -- \
> +-p 0x3 --config '(0,0,6),(1,0,7)'
> +
> +Send one packet to VF0 and VF1, check that thread on core6 and core7
> waked up::
> +
> +L3FWD_POWER: lcore 6 is waked up from rx interrupt on port 0 queue 0
> +L3FWD_POWER: lcore 7 is waked up from rx interrupt on port 0 queue 0
> +
> +Check the packet has been normally forwarded.
> +
> +After the packet forwarded, thread on core6 and core 7 will return to
> sleep::
> +
> +L3FWD_POWER: lcore 6 sleeps until interrupt triggers
> +L3FWD_POWER: lcore 7 sleeps until interrupt triggers
> +
> +Send packet flows to VF0 and VF1, check that thread on core1 and core2
> will
> +keep up awake.
> +
> +
> +8. VF veb
> +---------
> +create 2 VFs from 1 PF, start testpmd with 2VFs individually, verify
> throughput.
> +
> +echo 2 > /sys/bus/pci/devices/0000\:05\:00.0/max_vfs;
> +./usertools/dpdk-devbind.py --bind=vfio-pci 05:02.0 05:02.1
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 1,2 -n 4 --socket-
> mem=1024,1024 --file-prefix=pf \
> +-w 05:00.0 -- -i
> +
> +set vf mac addr 0 0 00:12:34:56:78:01
> +set vf mac addr 0 1 00:12:34:56:78:02
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 3-5 -n 4 --master-lcore=3 --
> socket-mem=1024,1024 --file-prefix=vf1 \
> +-w 05:02.0 -- -i --txq=2 --rxq=2 --rxd=512 --txd=512 --nb-cores=2 --rss-
> ip --eth-peer=0,00:12:34:56:78:02
> +
> +set promisc all off
> +set fwd mac
> +start
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 6-8 -n 4 --master-lcore=6 --
> socket-mem=1024,1024 --file-prefix=vf2 \
> +-w 05:02.1 -- -i --txq=2 --rxq=2 --rxd=512 --txd=512 --nb-cores=2 --rss-
> ip
> +
> +set promisc all off
> +set fwd mac
> +start
> +
> +9. VF performance
> +-----------------
> +1) vector (vector=y)
Please add more descriptions for what is vector equal to yes. Like what is needed for enabling vector mode of pmd.

> +./x86_64-native-linuxapp-gcc/app/testpmd -c 0x6 -n 4 --socket-
> mem=1024,1024 --file-prefix=pf \
> +-w 05:00.0 -w 07:00.0 -- -i
> +
> +set vf mac addr 0 0 00:12:34:56:78:01
> +set vf mac addr 1 0 00:12:34:56:78:02
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -c 0x0f8 -n 4 --master-lcore=3 -
> -socket-mem=1024,1024 --file-prefix=vf \
> +-w 05:02.0 -w 07:02.0 -- -i --txq=2 --rxq=2 --rxd=512 --txd=512 --nb-
> cores=4 --rss-ip

Please just user testpmd for app name, no need to include target. 

> +
> +set promisc all off
> +set fwd mac
> +start
> +
> +37.2 + 37.2 = 74.4Mpps
> +
What is this? Which need to check? Please add more descriptions.

> +2) scalar/bulk
> +
> +change CONFIG_RTE_LIBRTE_AVF_INC_VECTOR=n in config/common_base
> +
> +34.9 + 34.9 = 69.8Mpps
> +
> +
> +additional test case: (N/A)
> +PF change mtu will cause any VF reset or any other issues.
> +create 2 VFs from 1 PF, reset 1 VF continuously, verify PF and another VF
> can work fine.	
> +
> +
> +AVF test in qemu-kvm
> +====================
> +
> +#start PF with testpmd in host,
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 1,2 -n 4 --socket-
> mem=1024,1024 --file-prefix=pf -w 05:00.0 -w 07:00.0 -- -i
> +
> +set vf mac addr 0 0 00:12:34:56:78:01
> +set vf mac addr 1 0 00:12:34:56:78:02
> +
> +#start VF in guest,
> +
> +taskset -c 12-17 qemu-system-x86_64 -enable-kvm -cpu host -smp 6 -m 4096
> \
> +-device virtio-net-pci,netdev=net0 -netdev tap,id=net0,script=/etc/qemu-
> ifup,downscript=no,vhost=on \
> +-drive file=/home/ypei/qemu/ubuntu-16.04.img,if=virtio -vnc :1 -nographic
> +
> +(qemu) help device_add
> +device_add vfio-pci,host=05:02.0,id=vf1
> +device_add vfio-pci,host=07:02.0,id=vf2
> +
> +make install RTE_SDK=`pwd` T=x86_64-native-linuxapp-gcc -j4
> +modprobe uio;
> +insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko;
> +
> +./usertools/dpdk-devbind.py --bind=igb_uio 00:05.0 00:06.0
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 1-5 -n 4 --master-lcore=1 \
> +-w 00:05.0 -w 00:06.0 -- -i --txq=4 --rxq=4  --nb-cores=4 --txqflags=0
> +
> +examples/l3fwd-power/build/l3fwd-power -l 1,2 -n 4 -w 00:05.0 -w 00:06.0
> -- \
> +-p 0x3 --config '(0,0,1),(1,0,2)'
> +
> +./x86_64-native-linuxapp-gcc/app/testpmd -l 1-5 -n 4 --master-lcore=1 \
> +-w 00:05.0 -w 00:06.0 -- -i --txq=4 --rxq=4  --nb-cores=4 --max-pkt-
> len=9000 --txqflags=0
> \ No newline at end of file
> --
> 2.7.4