[dts] [DTS][PATCH V2/2] Add test plan for suite nic_single_core_perf

[Liu, Yong]

Fei,
My comments are inline.

Thanks,
Marvin

> -----Original Message-----
> From: dts [mailto:dts-bounces at dpdk.org] On Behalf Of wang fei
> Sent: Tuesday, January 16, 2018 4:49 PM
> To: dts at dpdk.org
> Cc: Pei, Yulong <yulong.pei at intel.com>; Wang, FeiX Y
> <feix.y.wang at intel.com>
> Subject: [dts] [DTS][PATCH V2/2] Add test plan for suite
> nic_single_core_perf
> 
> For next branch: add test plan for test suite nic_single_core_perf
> 
> Signed-off-by: wang fei <feix.y.wang at intel.com>
> ---
>  test_plans/nic_single_core_perf_test_plan.rst | 117
> ++++++++++++++++++++++++++
>  1 file changed, 117 insertions(+)
>  create mode 100644 test_plans/nic_single_core_perf_test_plan.rst
> 
> diff --git a/test_plans/nic_single_core_perf_test_plan.rst
> b/test_plans/nic_single_core_perf_test_plan.rst
> new file mode 100644
> index 0000000..740a5ca
> --- /dev/null
> +++ b/test_plans/nic_single_core_perf_test_plan.rst
> @@ -0,0 +1,117 @@
> +.. Copyright (c) <2016>, Intel Corporation
> +   All rights reserved.
> +
> +   Redistribution and use in source and binary forms, with or without
> +   modification, are permitted provided that the following conditions
> +   are met:
> +
> +   - Redistributions of source code must retain the above copyright
> +     notice, this list of conditions and the following disclaimer.
> +
> +   - Redistributions in binary form must reproduce the above copyright
> +     notice, this list of conditions and the following disclaimer in
> +     the documentation and/or other materials provided with the
> +     distribution.
> +
> +   - Neither the name of Intel Corporation nor the names of its
> +     contributors may be used to endorse or promote products derived
> +     from this software without specific prior written permission.
> +
> +   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> +   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> +   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> +   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> +   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> +   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> +   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
> +   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> +   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> +   OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +==========================================================
> +Nic_single_core_perf is designed to measure the throughput
> +benchmark using single forwarding core for fortville25g,
> +fortville40g, nnt10g
> +==========================================================
> +

Words "Measure" and "benchmark" have the same meaning, please just use one.

Support to change it as:
Benchmark the performance of single core forwarding with FVL25G/40G/NNT10G. 
Suggest to use upper characters for short code name.


> +Prerequisites
> +=============
> +
> +1. Hardware:
> +    1) nic_single_core_perf test for fortvile25g: two dual port
> fortville25g nics , all installed on the same socket, pick one port per
> nic
> +    2) nic_single_core_perf test for fortvile40g: two dual port
> fortville40g nics , all installed on the same socket, pick one port per
> nic
> +    3) nic_single_core_perf test for nnt10g : four 82599 nics, all
> installed on the same socket, pick one port per nic
> +
> +2. software:
> +    dpdk: git clone http://dpdk.org/git/dpdk
> +    scapy: http://www.secdev.org/projects/scapy/
> +    dts (next branch): git clone http://dpdk.org/git/tools/dts, then git
> checkout next
> +    Trex source code: http://trex-tgn.cisco.com/trex/release/v2.26.tar.gz
> +    python-prettytable(uesed to output the test result matrix as a
> separate .txt file):
> +        apt install python-prettytable (for ubuntu os)
> +        or dnf install python-prettytable (for fedora os).
> +
> +3. Connect all the selected nic ports to TREX ports.
> +    2 ixia 25g ports for fvl25g ports
> +    2 ixia 40g ports for fvl40g ports
> +    4 ixia 10g ports for 4 nnn ports

Typo, should be nnt. 

> +
> +Test Case : nic_single_core_perf
> +==================================
> +1) bind nic ports to igb_uio
> +    for nnt10g nic, 4 ports will be binded
> +    for fvl25g and 40g nic, 2 ports will be binded.
> +

Typo here, should be bound. And this is basic setup, just one sentence "Bind tested ports to igb_uio." is enough.

> +2) start testpmd

Please add "::" after the description and add one blank line between codes and description.

> +     ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x6 -n 4 -- -i  --
> portmask=0xf  --txd=128 --rxd=128
> +      (make sure use two cores on the same socket that the nics installed
> on)
> +        testpmd> start
> +
> +3) configure ixia to send traffic

Should be Trex? Or software generator?

> +    configure one below stream for each ixia port
> +        dst mac: peer nic port mac
> +        src ip : random
> +        dst ip : random
> +        packet length : 64 byte
> +
> +4)  check throughput and compare it with the expected value.
> +
> +5)  for nnt10g, repeat above step 1-4 for txd=rxd=512,2048 separately.
> +    for fvl25g and fvl40g nic, just test txd=rxd=512,2048 following above
> steps 1-4.
> +
> +6) print out below test results table for nnt, fvl25g,fvl40g separately
> +        fvl25g:
> ++------------+---------+------------+------------+---------------------+
> +| Frame Size | txd/rxd | Throughput | % linerate | Expected throughput |
> ++============+=========+============+============+=====================+
> +| 64         | 512     | xxxxxx     | 58.327     | 43.365              |
> ++------------+---------+------------+------------+---------------------+
> +| 64         | 2048    | xxxxxx     | 51.964     | 38.658              |
> ++------------+---------+------------+------------+---------------------+
> +
> +        fvl40g:
> ++------------+---------+------------+------------+---------------------+
> +| Frame Size | txd/rxd | Throughput | % linerate | Expected throughput |
> ++============+=========+============+============+=====================+
> +| 64         | 512     | xxxxxx     | 36.618     | 43.614              |
> ++------------+---------+------------+------------+---------------------+
> +| 64         | 2048    | xxxxxx     | 32.275     | 38.407              |
> ++------------+---------+------------+------------+---------------------+
> +
> +        for nnt10g:
> ++------------+---------+------------+------------+---------------------+
> +| Frame Size | txd/rxd | Throughput | % linerate | Expected throughput |
> ++============+=========+============+============+=====================+
> +| 64         | 128     | xxxxxx     | 85.769     | 50.746              |
> ++------------+---------+------------+------------+---------------------+
> +| 64         | 512     | xxxxxx     | 58.736     | 34.628              |
> ++------------+---------+------------+------------+---------------------+
> +| 64         | 2048    | xxxxxx     | 58.219     | 34.657              |
> ++------------+---------+------------+------------+---------------------+
> +
> +Case assert failure if the actual throughputs lower exceed 1 Mpps than
> the expected throughput.

I think threshold calculated by percentage is more meaningful.

> +
> +
> +
> --
> 2.7.4