[dts] [PATCH V2 2/2] add test plan vhost_pmd_xstats

[Liu, Yong]

Gang, some comments below.

> -----Original Message-----
> From: dts [mailto:dts-bounces at dpdk.org] On Behalf Of xu,gang
> Sent: Wednesday, December 28, 2016 10:15 AM
> To: dts at dpdk.org
> Cc: Xu, GangX
> Subject: [dts] [PATCH V2 2/2] add test plan vhost_pmd_xstats
> 
> Signed-off-by: xu,gang <gangx.xu at intel.com>
> ---
>  test_plans/vhost_pmd_xstats_test_plan.rst | 137
> ++++++++++++++++++++++++++++++
>  1 file changed, 137 insertions(+)
>  create mode 100644 test_plans/vhost_pmd_xstats_test_plan.rst
> 
> diff --git a/test_plans/vhost_pmd_xstats_test_plan.rst
> b/test_plans/vhost_pmd_xstats_test_plan.rst
> new file mode 100644
> index 0000000..7cf20bf
> --- /dev/null
> +++ b/test_plans/vhost_pmd_xstats_test_plan.rst
> @@ -0,0 +1,137 @@
> +.. Copyright (c) <2016>, Intel Corporation
> +   All rights reserved.
> +
> +   Redistribution and use in source and binary forms, with or without
> +   modification, are permitted provided that the following conditions
> +   are met:
> +
> +   - Redistributions of source code must retain the above copyright
> +     notice, this list of conditions and the following disclaimer.
> +
> +   - Redistributions in binary form must reproduce the above copyright
> +     notice, this list of conditions and the following disclaimer in
> +     the documentation and/or other materials provided with the
> +     distribution.
> +
> +   - Neither the name of Intel Corporation nor the names of its
> +     contributors may be used to endorse or promote products derived
> +     from this software without specific prior written permission.
> +
> +   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> +   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> +   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> +   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> +   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> +   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> +   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
> +   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> +   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> +   OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +==================================
> +Vhost PMD xstats test plan
> +==================================
> +
> +This test plan will cover the basic vhost pmd xstats case and will be
> worked
> +as a regression test plan. In the test plan, we will use vhost as a pmd
> port
> +in testpmd.
> +
> +Test Case1: Vhsot PMD xstats based on packet size
> +=========================================================================
> ======
> +
> +flow:
> +TG-->NIC-->>Vhost TX-->Virtio RX-->Virtio TX-->Vhsot RX-->NIC-->TG
> +
> +1. bind one physical port to igb_uio, then launch the testpmd
> +
> +2. Launch VM1 with using hugepage, 2048M memory, 2 cores, 1 cockets ,
> +1 virtio-net-pci
> +
It's better to add qemu command back, it's good reference for user usage.

> +3. On VM1, run testpmd
> +
> +4. On host, testpmd, set ports to the mac forward mode
> +
> +    testpmd>set fwd mac
> +    testpmd>start tx_first
> +
> +5. On VM, testpmd, set port to the mac forward mode
> +
> +    testpmd>set fwd mac
> +    testpmd>start
> +
> +6. On host run "show port xstats all" at least twice to check the packets
> number
> +
> +7. Let TG generate different size of packets, send 10000 packets for each
> packet
> +sizes(64,128,255, 512, 1024, 1523), check the statistic number is correct
> +
> +Test Case2: Vhost PMD xstats based on packet types
> +=========================================================================
> ======
> +
> +Similar as Test Case1, all steps are similar except step6,7:
> +
> +6. On host run "show port xstats all" at least twice to check the packets
> type:
> +
> +7. Let TG generate different type of packets, broadcast, multicast, ucast,
> check
> +the statistic number is correct
> +
> +Test Case3: clear Vhost PMD xstats
> +=========================================================================
> ======

As I recommend before, you can move this case to case1 and case2. There's no need to recreate environment and just only verify clear function.
And I remember there's one case covered mergeable and no-mergeable path, what's happened?

> +
> +flow:
> +TG-->NIC-->>Vhost TX-->Virtio RX-->Virtio TX-->Vhsot RX-->NIC-->TG
> +
> +1.bind one physical port to igb_uio,
> +then launch the testpmd
> +
> +2. Launch VM1, with using hugepage, 2048M memory,
> +2 cores, 1 cockets ,1 virtio-net-pci
> +
> +3. On VM1, run testpmd
> +
> +4. On host, testpmd, set ports to the mac forward mode
> +
> +    testpmd>set fwd mac
> +    testpmd>start tx_first
> +
> +5. On VM, testpmd, set port to the mac forward mode
> +
> +    testpmd>set fwd mac
> +    testpmd>start
> +
> +6. Let TG generate different size of packets, send 10000 packets for each
> +packet sizes(64,128,255, 512, 1024, 1523,3000), check the statistic
> number
> +is correct
> +
> +7. On host run "show port xstats all" at least twice to check the packets
> number
> +
> +8. On host run "clear port xstats all" , then all the statistic date
> should be 0
> +
> +
> +Test Case4: Long lasting test for Vhsot PMD xstats
> +======================================================

Please simplify the case name like "stability case with multiple queues"

> +1. no need bind any physical port to igb_uio
> +
> +2. Launch VM1, set queues=2, vectors=2xqueues+2, mq=on, with using
> hugepage,
> +2048M memory, 2 cores, 1 cockets ,1 virtio-net-pci
> +
> +3. On VM1, run testpmd
> +
> +4. On host, testpmd, set ports to the mac forward mode
> +
> +    testpmd>set fwd io retry
> +    testpmd>start tx_first 8
> +
> +5. On VM, testpmd, set port to the mac forward mode
> +
> +    testpmd>start
> +
> +6.  Send packets for 30 minutes, check the Xstatsa still can work
> correctly
> +   testpmd>show port xstats all
> +
> +
> +
> +
> +
> +
> --
> 1.9.3