[V1] test_plans/cvl_dcf_qos: configure QoS for vf/vsi in DCF test plan

Message ID 20210805222734.70194-1-yuan.peng@intel.com (mailing list archive)
State Superseded
Headers
Series [V1] test_plans/cvl_dcf_qos: configure QoS for vf/vsi in DCF test plan |

Checks

Context Check Description
ci/Intel-dts-test fail Testing issues

Commit Message

Peng, Yuan Aug. 5, 2021, 10:27 p.m. UTC
  Add cvl_dcf_qos_test_plan.rst to dts.

Signed-off-by: Peng Yuan <yuan.peng@intel.com>
  

Patch

diff --git a/test_plans/cvl_dcf_qos_test_plan.rst b/test_plans/cvl_dcf_qos_test_plan.rst
new file mode 100644
index 00000000..02f776ff
--- /dev/null
+++ b/test_plans/cvl_dcf_qos_test_plan.rst
@@ -0,0 +1,1700 @@ 
+.. Copyright (c) <2021>, Intel Corporation
+   All rights reserved.
+
+   Redistribution and use in source and binary forms, with or without
+   modification, are permitted provided that the following conditions
+   are met:
+
+   - Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+
+   - Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in
+     the documentation and/or other materials provided with the
+     distribution.
+
+   - Neither the name of Intel Corporation nor the names of its
+     contributors may be used to endorse or promote products derived
+     from this software without specific prior written permission.
+
+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+   OF THE POSSIBILITY OF SUCH DAMAGE.
+
+===================================
+CVL configure QoS for vf/vsi in DCF
+===================================
+
+Description
+===========
+
+Support ETS-based QoS configuration, including Arbiters configuration (strict priority, WFQ)
+and BW Allocation and limitation.
+1. Add a new virtchnl capability to indicate support for QoS
+2. Enable DCF to set BW limits (min, max) for DPDK AVFs and arbitration mode.
+   To be accomplished by way of new DCF specific virtchnl op.
+   a) Bandwidth limits per VF (VSI) are within the context of BW allocated for the individual TCs
+      at TC layer (layer 2) in the topology.  The profiles will be added at the VSI level.
+   b) In strict arbitration mode (vs ETS mode) a PIR can be applied to layer 2 TC nodes
+      by the DCF for predictable behavior.
+3. Enable DPDK iAVF to query QoS capability, such as UP to TC mapping,
+   available BW (min, max) per TC etc. To be accomplished by way of new advance virtchnl op.
+4. Enable DPDK iAVF to set TC to queue mapping, to be accomplished by way of new advance virtchnl op.
+Note: The AVF inherits TCs that are configured on the PF. DCF tunes the BW allocation per TC for the target AVF.
+
+Prerequisites
+=============
+
+1. Hardware:
+   1 port from columbiaville_100g(NIC-1), 2 ports from columbiaville_25g(NIC-2)
+   IXIA traffic generator::
+
+      +---------------------------------+
+      |  DUT           |  IXIA          |
+      +=================================+
+      | NIC-1,Port-1  ---  IXIA, Port-1 |
+      | NIC-2,Port-1  ---  NIC-2,Port-2 |
+      +---------------------------------+
+
+   assume that device ID and pci address of NIC-1,Port-1 are ens785f0 and 18:00.0,
+   device ID and pci address of NIC-2,Port-1 are ens802f0 and 86:00.0.
+
+2. software:
+
+   - dpdk: http://dpdk.org/git/dpdk
+
+3. don't allow the peer port to modify the DCB parameter::
+
+    lldptool -T -i ens802f0 -V ETS-CFG willing=no
+    lldptool -T -i ens785f0 -V ETS-CFG willing=no
+
+4. Configure DCB TC bandwidth with lldptool::
+
+    lldptool -T -i enp785s0f0 -V ETS-CFG willing=no tsa=0:strict,1:strict,2:strict,3:strict,4:strict,5:strict,6:strict,7:strict up2tc=0:0,1:0,2:0,3:1,4:2,5:0,6:0,7:0 tcbw=10,30,60,0,0,0,0,0
+
+   or configure DCB with dcbgetset tool::
+
+    ./dcbgetset enp785s0f0 --ieee --up2tc 0,0,0,1,2,0,0,0 --tcbw 10,30,60,0,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+
+5. Generate 2 VFs on NIC-1,Port-1 and NIC-2,Port-1::
+
+    echo 2 > /sys/bus/pci/devices/0000:18:00.0/sriov_numvfs
+    echo 2 > /sys/bus/pci/devices/0000:86:00.0/sriov_numvfs
+
+6. Set VF0 as trust and set VF1 mac address::
+
+    ip link set dev ens785f0 vf 0 trust on
+    ip link set ens785f0 vf 1 mac 00:11:22:33:44:55
+    ip link set dev ens802f0 vf 0 trust on
+    ip link set ens802f0 vf 1 mac 00:11:22:33:44:66
+
+7. Bind VFs to dpdk driver::
+
+    modprobe vfio-pci
+    ./usertools/dpdk-devbind.py -b vfio-pci 0000:18:01.0 0000:18:01.1 0000:86:01.0 0000:86:01.1
+    ./usertools/dpdk-devbind.py -s
+    0000:18:01.0 'Ethernet Adaptive Virtual Function 1889' drv=vfio-pci unused=iavf
+    0000:18:01.1 'Ethernet Adaptive Virtual Function 1889' drv=vfio-pci unused=iavf
+    0000:86:01.0 'Ethernet Adaptive Virtual Function 1889' drv=vfio-pci unused=iavf
+    0000:86:01.1 'Ethernet Adaptive Virtual Function 1889' drv=vfio-pci unused=iavf
+
+8. 100G testpmd setting, start testpmd::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -- -i --txq=8 --rxq=8 --nb-cores=8 --port-topology=loop
+
+   10G testpmd setting, start testpmd::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -a 86:01.0,cap=dcf -a 86:01.1 -- -i --txq=8 --rxq=8 --nb-cores=8
+
+Test case 1: strict mode, check peak_tb_rate
+============================================
+
+1. DCB setting, set 3 TCs bandwidth with strict mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,0,1,2,0,0,0 --tcbw 10,30,60,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0   
+    ifconfig ens785f0 up
+
+2. start testpmd with 100G setting, then set profile and TC mapping::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -- -i --txq=8 --rxq=8 --nb-cores=8 --port-topology=loop
+    port stop all
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0 
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 no
+
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 no
+    port start all
+    set fwd mac
+    start
+
+3. send four streams from IXIA,
+   mac address is "00:11:22:33:44:55", vlan=0, priority=2,5,3,4,
+   frame size is 68 bytes, each stream desired 25% max rate.
+   send each stream separately, check the TX throughput of each priority and queue mapping:
+   2MBps queue0-3; 2MBps queue0-3; 4MBps queue4-5; 4MBps queue6-7.
+   send the four streams synchronously, the sum of throughput is 10MBps.
+   and the queue mapping is correct.
+
+4. restart testpmd, and map all the queues to 2 TCs::
+
+    port stop all
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 no
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 no
+    port start all
+    set fwd mac
+    start
+
+5. send the same four streams as step3.
+   check stream 4 are dropped by vf1.
+   stream1-3 can match the peak_tb_rate limit, and the queue mapping is correct.
+
+Test case 2: ets mode, check peak_tb_rate
+=========================================
+
+1. DCB setting, set 2 TCs bandwidth with ets mode::
+
+    ./dcbgetset ens785f0 --ieee --up2tc 0,0,0,0,1,1,1,1 --tcbw 20,80,0,0,0,0,0,0 --tsa 2,2,2,2,2,2,2,2 --pfc 0,0,0,0,0,0,0,0
+    ./dcbgetset ens802f0 --ieee --up2tc 0,0,0,0,1,1,1,1 --tcbw 20,80,0,0,0,0,0,0 --tsa 2,2,2,2,2,2,2,2 --pfc 0,0,0,0,0,0,0,0
+    ifconfig ens785f0 up
+    ifconfig ens802f0 up
+
+2. start testpmd with 10G setting, then set profile and TC mapping::
+
+    set portlist 0,2,1,3
+    show config fwd
+    port stop all
+    add port tm node shaper profile 0 1 10000000 0 4000000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm node shaper profile 2 1 10000000 0 1000000000 0 0 0
+    add port tm nonleaf node 2 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 2 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 2 800 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 2 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 2 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 2 2 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 2 3 800 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 2 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    add port tm nonleaf node 3 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 3 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 3 800 1000 0 1 1 0 1 0 0
+    add port tm leaf node 3 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 3 yes
+    port start all
+    set fwd mac
+    start
+
+3. send two streams from IXIA,
+   mac address is "00:11:22:33:44:55", vlan=0, priority=0/4,
+   frame size is 68 bytes, each stream desired 50% max rate.
+   send each stream separately, check the tx rate is about 7.3Gbps, check queue mapping is correct.
+   send 2 streams synchronously, each 50%max
+   check the port3 stats, the tx rate is about 7.3Gbps,
+   stop forward, check the result, queue0-queue3 is TC0, queue4-queue7 is TC1
+   the two stream’s occupation is about 20:80 as the DCB TC bandwidth setting.
+
+4. set both two profiles' PIR to 500000000, other settings are the same as step2, 
+   send same streams as step3.
+   send each stream separately, check the tx rate is about 3.95Gbps, closed to the PIR 4Gbps,
+   check queue mapping is correct.
+   Send the two streams synchronously, the throughput is limited by the cable to 7.27Gbps.
+   check the port3 stats, the tx rate is still about 7.27Gbps,
+   stop forward, check the result, queue0-queue3 is TC0, queue4-queue7 is TC1
+   TC1’s rate is about 3.95Gbps, TC0’s rate is about 3.32Gbps.
+   the two stream’s occupation is more than 20:80, about 45:55.
+   because TC1 throughput is limited by PIR, so the rest throughput is occupied by TC0.
+
+Test case 3: strict mode, check cmit_tb_rate
+============================================
+
+1. DCB setting, set 2 TCs bandwidth with strict mode::
+
+    ./dcbgetset ens785f0 --ieee --up2tc 0,0,0,1,0,0,0,0 --tcbw 10,90,0,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+    ./dcbgetset ens802f0 --ieee --up2tc 0,0,0,1,0,0,0,0 --tcbw 10,90,0,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+    ifconfig ens785f0 up
+    ifconfig ens802f0 up
+
+2. start testpmd with 10G setting, then set profile and TC mapping as test_case 2 step2.
+
+3. send two streams from IXIA,
+   mac address is "00:11:22:33:44:55", vlan=0, priority=0/3,
+   frame size is 68 bytes, each stream desired 50% max rate.
+   send each stream separately, check the tx rate is about 7.3Gbps, check queue mapping is correct.
+   send 2 streams synchronously, each 50%max
+   check the port3 stats, the tx rate is about 7.3Gbps,
+   stop forward, check the result,
+   queue0-queue3 which mapping to TC0 should have 80Mbps rate
+   queue4-queue7 which mapping to TC1 have about 7.22Gbps.
+
+Note: the cmit of the profile is not supported now, so the current expected result is:
+all the TX throughput should at TC1, TC0 should has no throughput.
+
+Test case 4: ets mode, check cmit_tb_rate
+=========================================
+
+1. DCB setting, set 3 TCs bandwidth with ets mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,1,1,2,2,2,2 --tcbw 1,10,89,0,0,0,0,0 --tsa 2,2,2,2,2,2,2,2 --pfc 0,0,0,0,0,0,0,0
+    ./dcbgetset ens802f0  --ieee --up2tc 0,0,1,1,2,2,2,2 --tcbw 1,10,89,0,0,0,0,0 --tsa 2,2,2,2,2,2,2,2 --pfc 0,0,0,0,0,0,0,0
+    ifconfig ens785f0 up
+    ifconfig ens802f0 up
+
+2. start testpmd with 10G setting::
+
+    set portlist 0,2,1,3
+    show config fwd
+    port stop all
+    add port tm node shaper profile 0 1 1000000000 0 4000000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm node shaper profile 2 1 100000000 0 1000000000 0 0 0
+    add port tm node shaper profile 2 2 100000000 0 150000000 0 0 0
+    add port tm nonleaf node 2 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 2 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 2 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 2 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 2 0 900 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 2 1 900 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 2 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 2 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 2 4 700 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 2 5 700 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 2 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    add port tm nonleaf node 3 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 3 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 3 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 3 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 3 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 3 yes
+    port start all
+    set fwd mac
+    start
+
+3. send 8 streams from IXIA,
+   mac address is "00:11:22:33:44:55", vlan=0, priority=0-7,
+   frame size is 1024 bytes, each stream desired 12.5% max rate.
+   send each stream, the PIR can be reached, and the queue mapping is correct.
+   send 8 streams synchronously, check throughput is 9.77Gbps,
+   TC2 and TC1 can be satisfied, and the rest rate is given to TC0.
+
+4. set frame size to 68bytes, send 8 streams synchronously,
+   check the throughput is 7.273Gbps. all the TC can’t reach PIR.
+   TC0 rate is 0.072Gbps, occupy 0.01 ets BW.
+   TC1 rate is 0.72Gbps, occupy 0.1 ets BW.
+   TC2 rate is 6.48Gbps, occupy 0.89 ets BW.
+   The TC0-TC2’s rate allocation is same as bandwidth allocation: 1:10:89.
+
+Test case 5: 2 iavf VFs, strict mode, check peak_tb_rate
+========================================================
+
+1. DCB setting, set 3 TCs bandwidth with strict mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,0,1,2,0,0,0 --tcbw 10,30,60,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+
+2. create 3 VFs::
+
+    echo 3 > /sys/bus/pci/devices/0000\:18\:00.0/sriov_numvfs
+    ./usertools/dpdk-devbind.py -b vfio-pci 18:01.0 18:01.1 18:01.2
+    ip link set dev ens785f0 vf 0 trust on
+    ip link set ens785f0 vf 1 mac 00:11:22:33:44:55
+    ip link set ens785f0 vf 2 mac 00:11:22:33:44:66
+
+3. start testpmd with 100G setting, different vsi node of same TC node use different profiles::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -a 18:01.2 -a 18:01.3 -- -i --txq=8 --rxq=8 --port-topology=loop --nb-cores=8
+    port stop all
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 -1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 -1 0 0xffffffff 0 0
+    add port tm leaf node 0 4 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 6 700 0 1 2 -1 0 0xffffffff 0 0
+    add port tm leaf node 0 7 700 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 8 700 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    add port tm nonleaf node 2 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 2 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 2 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 2 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 2 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 2 yes
+    port start all
+    set fwd mac
+    start
+
+4. send 8 streams, stream0-3’s mac address is vf1, others' is vf2, vlan=0, priority=1,2,3,4, 
+   send each stream separatly, check the stats:
+   stream0 maps queue0-1 of port 1, the throughput reaches PIR of profile 1.
+   stream1 maps queue0-1 of port 1, the throughput reaches PIR of profile 1.
+   stream2 maps queue2-3 of port 1, the throughput reaches PIR of profile 2.
+   stream3 maps queue4-7 of port 1, the throughput reaches PIR of profile 1.
+   stream4 maps queue0-1 of port 2, the throughput reaches PIR of profile 1.
+   stream5 maps queue0-1 of port 2, the throughput reaches PIR of profile 1.
+   stream6 maps queue2-5 of port 2, the throughput reaches PIR of profile 1.
+   stream7 maps queue6-7 of port 2, the throughput reaches PIR of profile 2.
+   send all streams synchronously, each 12.5%max, check the sum of throughput reach 16MBps.
+   each stream's queue mapping is correct.
+
+Test case 6: 2 iavf VFs, strict mode, check cmit_tb_rate
+========================================================
+
+1. DCB setting, set 3 TCs bandwidth with strict mode::
+
+    ./dcbgetset ens785f0 --ieee --up2tc 0,0,0,1,0,0,0,0 --tcbw 20,80,0,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+    ./dcbgetset ens802f0 --ieee --up2tc 0,0,0,1,0,0,0,0 --tcbw 20,80,0,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+
+2. create 3 VFs on each pf::
+
+    echo 3 > /sys/bus/pci/devices/0000\:18\:00.0/sriov_numvfs
+    ip link set dev ens785f0 vf 0 trust on
+    ip link set ens785f0 vf 1 mac 00:11:22:33:44:55
+    ip link set ens785f0 vf 2 mac 00:11:22:33:44:66
+    ./usertools/dpdk-devbind.py -b vfio-pci 18:01.0 18:01.1 18:01.2
+    echo 3 > /sys/bus/pci/devices/0000\:86\:00.0/sriov_numvfs
+    ip link set dev ens802f0 vf 0 trust on
+    ip link set ens802f0 vf 1 mac 00:11:22:33:44:77
+    ip link set ens802f0 vf 2 mac 00:11:22:33:44:88
+    ./usertools/dpdk-devbind.py -b vfio-pci 86:01.0 86:01.1 86:01.2
+
+3. start testpmd with 10G setting::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -a 18:01.2 -a 86:01.0,cap=dcf -a 86:01.1 -a 86:01.2 -- -i --txq=8 --rxq=8 --nb-cores=8
+    set portlist 0,3,1,4,2,5
+    show config fwd
+    port stop all
+    add port tm node shaper profile 0 1 100000000 0 4000000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 4 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 5 800 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 no
+    add port tm node shaper profile 3 1 100000000 0 500000000 0 0 0
+    add port tm nonleaf node 3 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 3 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 3 800 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 3 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 2 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 3 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 4 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 5 800 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 3 no
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 no
+    add port tm nonleaf node 4 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 4 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 4 800 1000 0 1 1 0 1 0 0
+    add port tm leaf node 4 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 4 no
+    add port tm nonleaf node 2 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 2 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 2 800 1000 0 1 1 0 1 0 0
+    add port tm leaf node 2 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 2 no
+    add port tm nonleaf node 5 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 5 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 5 800 1000 0 1 1 0 1 0 0
+    add port tm leaf node 5 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 5 no
+    port start all
+    set fwd mac
+    start
+
+4. send four streams, two streams' mac address is to vf1, the other two streams' mac address is to vf2,
+   with vlan0, frame size 68 bytes, UP=2/3, each 25%max.
+   check the vf4 and vf5 stats, the sum of tx rate is 7.27Gbps, each vf tx is 3.64Gbps.
+   in each vf, TC0 should occupied 0.8Gbps, the rest of throughput is occupied by TC1, which is about 2.84Gbps
+   stop the fwd, check each queue's tx stats,
+   vf4's queue0-queue3 and vf5's queue0-queue1 map to TC0, which occupied 0.8Gbps,
+   vf4's queue4-queue7 and vf5's queue2-queue7 map to TC1, which occupied 2.84Gbps.
+
+Note: now, the cmit_tb_rate can't take work, it is not supported by kernel.
+so the current status should be: all the TX throughput are occupied by TC1, TC0 should have no throughput.
+
+Test case 7: 2 iavf VFs, ets mode
+=================================
+
+1. DCB setting, set 3 TCs bandwidth with ets mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,0,1,2,0,0,0 --tcbw 10,30,60,0,0,0,0,0 --tsa 2,2,2,2,2,2,2,2 --pfc 0,0,0,0,0,0,0,0   
+    ./dcbgetset ens802f0  --ieee --up2tc 0,0,0,1,2,0,0,0 --tcbw 10,30,60,0,0,0,0,0 --tsa 2,2,2,2,2,2,2,2 --pfc 0,0,0,0,0,0,0,0   
+
+2. create 3 VFs on each pf::
+
+    echo 3 > /sys/bus/pci/devices/0000\:18\:00.0/sriov_numvfs
+    ip link set dev ens785f0 vf 0 trust on
+    ip link set ens785f0 vf 1 mac 00:11:22:33:44:55
+    ip link set ens785f0 vf 2 mac 00:11:22:33:44:66
+    ./usertools/dpdk-devbind.py -b vfio-pci 18:01.0 18:01.1 18:01.2
+    echo 3 > /sys/bus/pci/devices/0000\:86\:00.0/sriov_numvfs
+    ip link set dev ens802f0 vf 0 trust on
+    ip link set ens802f0 vf 1 mac 00:11:22:33:44:77
+    ip link set ens802f0 vf 2 mac 00:11:22:33:44:88
+    ./usertools/dpdk-devbind.py -b vfio-pci 86:01.0 86:01.1 86:01.2
+
+3. start testpmd with 10G setting::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -a 18:01.2 -a 86:01.0,cap=dcf -a 86:01.1 -a 86:01.2 -- -i --txq=8 --rxq=8 --nb-cores=8
+    set portlist 0,3,1,4,2,5
+    show config fwd
+    port stop all
+    add port tm node shaper profile 0 1 0 0 0 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 4 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 5 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 6 700 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 7 700 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 8 700 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm node shaper profile 3 1 0 0 0 0 0 0
+    add port tm nonleaf node 3 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 3 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 3 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 3 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 3 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 2 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 3 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 4 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 5 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 6 700 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 7 700 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 3 8 700 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 3 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    add port tm nonleaf node 2 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 2 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 2 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 2 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 2 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 2 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 2 yes
+    add port tm nonleaf node 4 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 4 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 4 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 4 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 4 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 4 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 5 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 4 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 4 yes
+    add port tm nonleaf node 5 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 5 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 5 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 5 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 5 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 5 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 5 yes
+    port start all
+    set fwd mac
+    start
+
+4. send 8 streams, stream0-3’s mac address is vf1, others' is vf2, vlan=0, priority=1,2,3,4, 
+   calculate the sum of vf1 and vf2 tx rate which belongs to TC0, mark it as r0,
+   calculate the sum of vf1 and vf2 tx rate which belongs to TC1 and TC2, mark them as r1 and r2.
+   check the proportion of r0:r1:r2 is 1:3:6, which can match the ets bandwidth limit 1:3:6,
+   and the queue mapping is correct.
+
+Test case 8: strict mode, 8 TCs
+===============================
+
+1. DCB setting, set 8 TCs bandwidth with strict mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,1,2,3,4,5,6,7 --tcbw 10,30,60,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+
+2. start testpmd with 100G setting::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -- -i --txq=8 --rxq=8 --port-topology=loop --nb-cores=8
+    port stop all
+    add port tm node shaper profile 0 1 1000000 0 400000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 200000000 0 0 0
+    add port tm node shaper profile 0 3 1000000 0 100000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 600 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 500 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 400 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 300 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 200 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 6 600 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 7 600 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 8 500 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 9 500 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 10 400 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 11 400 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 12 300 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 13 300 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 14 200 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 15 200 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 600 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 500 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 400 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 300 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 200 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 600 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 500 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 400 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 300 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 200 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    port start all
+    set fwd mac
+    start
+
+3. send 8 streams UP0-UP7,68bytes, each stream 12.5%max, which is much more than PIR.
+   Tx is limited by PIR, each TC can reach to PIR.
+
+4. change the shaper profile::
+
+    port stop all
+    add port tm node shaper profile 0 1 1000000 0 1780000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 600 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 500 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 400 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 300 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 200 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 6 600 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 7 600 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 8 500 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 9 500 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 10 400 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 11 400 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 12 300 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 13 300 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 14 200 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 15 200 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 600 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 500 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 400 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 300 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 200 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 600 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 500 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 400 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 300 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 200 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    port start all
+    set fwd mac
+    start
+
+5. send 8 streams UP0-UP7,68bytes, each stream 12.5%max, which is less than PIR.
+   stop the forward, check all the Tx packet drop is at queue0, which maps to TC0.
+   the throughput satisfy TC7-TC1 by priority.
+
+6. send 8 streams UP0-UP7,1024bytes, each stream 12.5%max, which is less than PIR.
+   stop the forward, check all the Tx packet drop is at queue0, which maps to TC0.
+   the throughput satisfy TC7-TC1 by priority.
+
+Test case 9: strict mode, 1 TC
+==============================
+
+1. DCB setting, set 1 TC bandwidth with strict mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,0,0,0,0,0,0 --tcbw 10,30,60,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+
+2. start testpmd with 100G setting::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -- -i --txq=8 --rxq=8 --port-topology=loop --nb-cores=8
+    port stop all
+    add port tm node shaper profile 0 1 1000000 0 1000000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 900 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    port start all
+    set fwd mac
+    start
+
+3. send 8 streams UP0-UP7,68bytes, each stream 12.5%max.
+   check the sum of Tx throughput can reach PIR.
+   only send 1 stream, check the Tx throughput can reach PIR too.
+
+Test case 10: ets mode, 8 TCs
+=============================
+
+1. DCB setting, set 8 TCs bandwidth with ets mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,1,2,3,4,5,6,7 --tcbw 5,10,15,10,20,1,30,9 --tsa 2,2,2,2,2,2,2,2 --pfc 0,0,0,0,0,0,0,0   
+    ./dcbgetset ens802f0  --ieee --up2tc 0,1,2,3,4,5,6,7 --tcbw 5,10,15,10,20,1,30,9 --tsa 2,2,2,2,2,2,2,2 --pfc 0,0,0,0,0,0,0,0   
+
+2. start testpmd with 10G setting::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -a 86:01.0,cap=dcf -a 86:01.1 -- -i --txq=8 --rxq=8 --nb-cores=8
+    set portlist 0,2,1,3
+    show config fwd
+    port stop all
+    add port tm node shaper profile 0 1 1000000 0 4000000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 2000000000 0 0 0
+    add port tm node shaper profile 0 3 1000000 0 1000000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 600 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 500 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 400 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 300 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 200 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 6 600 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 7 600 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 8 500 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 9 500 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 0 10 400 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 11 400 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 12 300 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 13 300 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 14 200 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 15 200 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm node shaper profile 2 1 1000000 0 400000000 0 0 0
+    add port tm node shaper profile 2 2 1000000 0 200000000 0 0 0
+    add port tm node shaper profile 2 3 1000000 0 100000000 0 0 0
+    add port tm nonleaf node 2 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 2 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 2 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 2 700 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 2 600 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 2 500 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 2 400 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 2 300 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 2 200 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 2 0 900 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 2 1 900 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 2 2 800 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 2 3 800 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 2 4 700 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 2 5 700 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 2 6 600 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 2 7 600 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 2 8 500 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 2 9 500 0 1 2 3 0 0xffffffff 0 0
+    add port tm leaf node 2 10 400 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 2 11 400 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 2 12 300 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 2 13 300 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 2 14 200 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 2 15 200 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 2 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 600 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 500 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 400 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 300 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 200 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 600 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 500 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 400 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 300 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 200 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    add port tm nonleaf node 3 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 3 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 3 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 3 700 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 3 600 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 3 500 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 3 400 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 3 300 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 3 200 1000 0 1 1 0 1 0 0
+    add port tm leaf node 3 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 1 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 2 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 3 600 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 4 500 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 5 400 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 6 300 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 7 200 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 3 yes
+    port start all
+    set fwd mac
+    start
+
+3. send 8 streams UP0-UP7,68bytes, each stream 12.5%max,
+   check port3 stats, the Tx rate is 7.3Gbps.
+   stop forward, check the result, queue0-queue4 correspond to TC0-TC4, can reach the PIR(100MBps),
+   queue6 which corresponds to TC6 is limited by PIR(200MBps) too.
+   queue7(maps to TC7) is limited by Rx IXIA traffic, can’t reach PIR(400MBps),
+   and TC5(maps to queue5) is the lowest priority, other TCs must be satisfied first,
+   so TC5 and TC7 are limited by the bandwidth distribution 1:9.
+
+4. set profile of port2 as below::
+
+    add port tm node shaper profile 2 1 1000000 0 100000000 0 0 0
+    add port tm node shaper profile 2 2 1000000 0 250000000 0 0 0
+    add port tm node shaper profile 2 3 1000000 0 100000000 0 0 0
+
+   queue0-queue4 and queue6-7 can reach PIR(are limited by PIR),
+   queue5(corresponds to TC5) is the lowest priority (1% BW set by DCB), 
+   the rest rate are put to queue 5, may be more than 1% of whole throughput.
+
+5. Set all the profile PIR=0::
+
+    add port tm node shaper profile 2 1 0 0 0 0 0 0
+    add port tm node shaper profile 2 2 0 0 0 0 0 0
+    add port tm node shaper profile 2 3 0 0 0 0 0 0
+
+   check all the steam allocated the throughput by ets bandwidth.
+
+Test case 11: ets mode, 1 TC
+============================
+
+1. DCB setting, set 1 TC bandwidth with ets mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,0,0,0,0,0,0 --tcbw 100,0,0,0,0,0,0,0 --tsa 2,2,2,2,2,2,2,2 --pfc 0,0,0,0,0,0,0,0   
+    ./dcbgetset ens802f0  --ieee --up2tc 0,0,0,0,0,0,0,0 --tcbw 100,0,0,0,0,0,0,0 --tsa 2,2,2,2,2,2,2,2 --pfc 0,0,0,0,0,0,0,0   
+
+2. start testpmd with 10G setting::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -a 86:01.0,cap=dcf -a 86:01.1 -- -i --txq=8 --rxq=8 --nb-cores=8
+    set portlist 0,2,1,3
+    show config fwd
+    port stop all
+    add port tm node shaper profile 0 1 1000000 0 10000000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 900 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    add port tm node shaper profile 2 1 1000000 0 1000000000 0 0 0
+    add port tm nonleaf node 2 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 2 900 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 2 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 2 1 900 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 2 yes
+    add port tm nonleaf node 3 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 3 900 1000 0 1 1 0 1 0 0
+    add port tm leaf node 3 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 4 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 5 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 6 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 3 7 900 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 3 yes
+    port start all
+    set fwd mac
+    start
+
+3. send 8 streams UP0-UP7,68bytes, each stream 12.5%max.
+   check the sum of Tx throughput can reach 7.3Gbps.
+   only send 1 stream, check the Tx throughput can reach 7.3Gbps too.
+
+Test case 12: query qos setting
+===============================
+
+1. DCB setting, set 3 TCs bandwidth with strict mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,0,1,2,0,0,0 --tcbw 10,30,60,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+    ifconfig ens785f0 up
+
+2. start testpmd with 100G setting, then set profile and TC mapping::
+
+    port stop all
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 no
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 no
+    port start all
+
+3. show port tm capability::
+
+    show port tm cap 1
+
+   show port tm level capability::
+
+    show port tm level cap 1 0
+    show port tm level cap 1 1
+    show port tm level cap 1 2
+
+   show port tm node capability::
+
+    show port tm node cap 1 900
+    show port tm node cap 1 800
+
+   check all the unit of rate is consistent.
+
+4. show port tm node type::
+
+    show port tm node type 1 0
+    show port tm node type 1 900
+    show port tm node type 1 1000
+
+   check the type is correct.
+
+Test case 13: pf reset
+======================
+
+1. run the test case 1, the result is as expected.
+
+2. reset pf::
+
+    echo 1 > /sys/devices/pci0000:17/0000:17:00.0/0000:18:00.0/reset
+
+3. send same streams as step1, check no packets received and transmitted.
+
+Test case 14: vf reset
+======================
+
+1. run the test case 1, the result is as expected.
+
+2. reset VF1 by set mac addr, to trigger VF reset ::
+
+    ip link set ens785f0 vf 1 mac 00:11:22:33:44:66
+
+   then execute below command in testpmd::
+
+    port stop 1
+    port reset 1
+    port start 1
+    start
+
+3. send same streams in step1 but with new mac address 00:11:22:33:44:66,
+   check all the TC0 map to all queues, TC1 and TC2 map to queue0.
+
+4. set the qos settings as test case 1 step2 again.
+   send same steams, check the same result as step 1.
+
+Note: the current status of step3 is that: after vf reset, TC0 maps to queue0-7,
+TC1 maps to queue 5-7, TC2 maps to no queue.
+all of TCs throughput is limited by profile 1.
+
+Test case 15: link status change
+================================
+
+1. run the test case 1, the result is as expected.
+
+2. change the link status::
+
+    ifconfig ens785f0 down
+
+   check TC setting is not changed, the queue mapping is not changed,
+   The Tx rate is not changed.
+
+3. change the link status again::   
+
+    ifconfig ens785f0 up
+
+   check the status, get the same result.
+
+Test case 16: DCB setting TC change
+===================================
+
+1. run the test case 1, the result is as expected.
+
+2. reset the DCB setting as below::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,0,1,2,0,0,0 --tcbw 10,40,50,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+
+3. send the same streams as step 1,
+   Only send TC0 stream, queue0-queue7 of both Rx and Tx have traffic, load is balancing.
+   Only send TC1/TC2 stream, Rx and Tx only queue0 has traffic.
+
+NOTE: The kernel default status is all the queues mapping to TC0, other TCs mapping to queue 0.
+
+Test case 17: negative case for requested VF
+============================================
+1. DCB setting, set 2 TCs bandwidth with strict mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,0,1,0,0,0,0 --tcbw 20,80,0,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+
+2. create 2 VFs::
+
+    echo 2 > /sys/bus/pci/devices/0000\:18\:00.0/sriov_numvfs
+    ./usertools/dpdk-devbind.py -b vfio-pci 18:01.0 18:01.1
+    ip link set dev ens785f0 vf 0 trust on
+    ip link set ens785f0 vf 1 mac 00:11:22:33:44:55
+
+3. start testpmd with 100G setting::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -- -i --txq=8 --rxq=8 --port-topology=loop --nb-cores=8
+    port stop all
+
+Subcase 1: Requested VF id is valid
+-----------------------------------
+Set 3 VSI, more than 2 VFs created::
+
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 900 0 1 2 1 0 0xffffffff 0 0
+    node id: too many VSI for one TC (error 33)
+
+Subcase 2: Valid number of TCs for the target VF
+------------------------------------------------
+1. configured 2 TCs by DCB, but only set 1 TC node::
+
+    add port tm node shaper profile 0 1 63000 0 12500000000 0 0 0
+    add port tm nonleaf node 0 1000000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900000 1000000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900000 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    ice_dcf_commit_check(): Not all enabled TC nodes are set
+    no error: (no stated reason) (error 0)
+
+2. Not all VFs are binded to TC node::
+
+    add port tm node shaper profile 0 1 63000 0 12500000000 0 0 0
+    add port tm nonleaf node 0 1000000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900000 1000000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800000 1000000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800000 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    ice_dcf_commit_check(): Not all VFs are binded to TC1
+    no error: (no stated reason) (error 0)
+
+3. add 1 profile, but use 2 profiles::
+
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    shaper profile id field (node params): shaper profile not exist (error 23)
+
+Subcase 3: Valid Min and Max values
+-----------------------------------
+1. Min default value is 500Kbps::
+
+    add port tm node shaper profile 0 1 62999 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900000 1000000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800000 1000000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800000 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800000 0 1 2 2 0 0xffffffff 0 0
+    testpmd> port tm hierarchy commit 0 yes
+    ice_dcf_execute_virtchnl_cmd(): No response (1 times) or return failure (-5) for cmd 37
+    ice_dcf_set_vf_bw(): fail to execute command VIRTCHNL_OP_DCF_CONFIG_BW
+    no error: (no stated reason) (error 0)
+
+    add port tm node shaper profile 0 1 63000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900000 1000000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800000 1000000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800000 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800000 0 1 2 2 0 0xffffffff 0 0
+    testpmd> port tm hierarchy commit 0 no
+
+   the setting commit successfully.
+
+2.Min BW for the given TC must be less than that of Max BW::
+
+    add port tm node shaper profile 0 1 2001000 0 2000000 0 0 0
+    add port tm nonleaf node 0 1000000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900000 1000000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800000 1000000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800000 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    ice_dcf_execute_virtchnl_cmd(): No response (1 times) or return failure (-5) for cmd 37
+    ice_dcf_set_vf_bw(): fail to execute command VIRTCHNL_OP_DCF_CONFIG_BW
+    no error: (no stated reason) (error 0)
+
+    add port tm node shaper profile 0 1 2000000 0 2000000 0 0 0
+    add port tm nonleaf node 0 1000000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900000 1000000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800000 1000000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800000 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+
+   the setting commit successfully.
+
+3. Max BW must be less than or equal to negotiated link speed for the port
+1).one iavf VF, two TCs::
+
+    add port tm node shaper profile 0 1 1000000000 0 12000000000 0 0 0
+    add port tm nonleaf node 0 1000000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900000 1000000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800000 1000000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800000 0 1 2 1 0 0xffffffff 0 0
+
+    port tm hierarchy commit 0 yes
+    ice_dcf_validate_tc_bw(): Total value of TC0 min bandwidth and other TCs' max bandwidth 104000000kbps should be less than port link speed 100000000kbps
+
+2).two iavf VF, two TCs::
+
+    echo 3 > /sys/bus/pci/devices/0000\:18\:00.0/sriov_numvfs
+    ./usertools/dpdk-devbind.py -b vfio-pci 18:01.0 18:01.1 18:01.2
+    ip link set dev ens785f0 vf 0 trust on
+    ip link set ens785f0 vf 1 mac 00:11:22:33:44:55
+    ip link set ens785f0 vf 2 mac 00:11:22:33:44:66
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -a 18:01.2 -a 18:01.3 -- -i --txq=8 --rxq=8 --port-topology=loop --nb-cores=8
+    port stop all
+    add port tm node shaper profile 0 1 10000000 0 1000000000 0 0 0
+    add port tm node shaper profile 0 2 10000000 0 8500000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 4 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 800 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    ice_dcf_validate_tc_bw(): Total value of TC0 min bandwidth and other TCs' max bandwidth 136160000kbps should be less than port link speed 100000000kbps
+    no error: (no stated reason) (error 0)
+
+4. Max BW cannot be 0 and must be greater than or equal to Min BW
+   If set max BW to 0, there is not max rate limit::
+
+    add port tm node shaper profile 0 1 0 0 0 0 0 0    
+    add port tm nonleaf node 0 1000000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900000 1000000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800000 1000000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800000 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800000 0 1 2 1 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    port start all
+    set fwd mac
+    start
+
+   send two streams from IXIA,
+   mac address is "00:11:22:33:44:55", vlan=0, priority=0,3, frame size is 1024 bytes, 100% max rate.
+   send each stream separately, check the TX throughput of each TC can reach linerate.
+   and the queue mapping is correct.
+
+Test case 18: negative case for req VF to update its queue to TC mapping
+========================================================================
+1. DCB setting, set 3 TCs bandwidth with strict mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,0,1,2,0,0,0 --tcbw 10,30,60,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+
+2. create 2 VFs::
+
+    echo 2 > /sys/bus/pci/devices/0000\:18\:00.0/sriov_numvfs
+    ./usertools/dpdk-devbind.py -b vfio-pci 18:01.0 18:01.1
+    ip link set dev ens785f0 vf 0 trust on
+    ip link set ens785f0 vf 1 mac 00:11:22:33:44:55
+
+3. start testpmd with 100G setting::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -- -i --txq=8 --rxq=8 --port-topology=loop --nb-cores=8
+    port stop all
+
+Subcase 1: Total number of queue pairs match to what the VF is allocated
+------------------------------------------------------------------------
+1. 8 queues, only mapped 7 queues to TC::
+
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    iavf_hierarchy_commit(): queue node is less than allocated queue pairs
+    no error: (no stated reason) (error 0)
+
+2. 8 queues, mapped 9 queues to TC::
+
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 8 700 0 1 2 0 0 0xffffffff 0 0
+    num strict priorities field (node params): SP priority not supported (error 27)
+
+Subcase 1: Number of TCs match is less than TC enabled on the VF
+----------------------------------------------------------------
+1. VF0 VSI nodes less than TC nodes::
+
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    ice_dcf_commit_check(): Not all VFs are binded to TC2
+    no error: (no stated reason) (error 0)
+
+2. VF1 VSI nodes less than TC nodes::
+
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 no
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    iavf_hierarchy_commit(): Does not set VF vsi nodes to all TCs
+    no error: (no stated reason) (error 0)
+
+3. not all VF1 VSI nodes mapping to queues, set successfully::
+
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+
+   send TC0 and TC1 streams, the queue mapping is correct.
+
+Subcase 3: Number of TCs match is more than TC enabled on the VF
+----------------------------------------------------------------
+The TC number should map the TC enabled by lldptool, not set by port 0
+run the below steps sequentially.
+
+1. TC nodes more than TC enabled by lldptool::
+
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 600 1000 0 1 1 -1 1 0 0
+    node id: too many TCs (error 33)
+
+2. VF0 VSI nodes more than TC nodes::
+
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 6 600 0 1 2 2 0 0xffffffff 0 0
+    parent node id: parent not exist (error 19)
+
+3. VF1 VSI nodes more than TC nodes::
+
+    port tm hierarchy commit 0 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 600 1000 0 1 1 0 1 0 0
+    node id: too many TCs (error 33)
+
+4. more than VF1 VSI nodes mapping to queues::
+
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 600 0 1 2 0 0 0xffffffff 0 0
+    parent node id: parent not exist (error 19)
+
+Subcase 4: overlap between queue to TC mapping
+----------------------------------------------
+There can't be overlap between queue to TC mapping::
+
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 no
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 800 0 1 2 0 0 0xffffffff 0 0
+    node id: node id already used (error 33)
+
+Subcase 5: Non-contiguous TC setting in queue mapping
+-----------------------------------------------------
+1. set non-contiguous TC and queue mapping::
+
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 yes
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 800 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 yes
+    port start all
+    set fwd mac
+    start
+
+2. send four streams from IXIA,
+   mac address is "00:11:22:33:44:55", vlan=0, priority=2,5,3,4,
+   frame size is 68 bytes, each stream desired 25% max rate.
+   send each stream separately, check the TX throughput of each priority and queue mapping:
+   UP=2/5 which matches to TC0, maps to queue0-1, throughput can reach 2MBps,
+   UP=3 which matches to TC1, maps to queue2-3, throughput can reach 4MBps,
+   UP=4 which matches to TC2, maps to queue4-7. throughput can reach 4MBps.
+
+Test case 19: different vlan ID
+===============================
+
+1. DCB setting, set 3 TCs bandwidth with strict mode::
+
+    ./dcbgetset ens785f0  --ieee --up2tc 0,0,0,1,2,0,0,0 --tcbw 10,30,60,0,0,0,0,0 --tsa 0,0,0,0,0,0,0,0 --pfc 0,0,0,0,0,0,0,0
+    ifconfig ens785f0 up
+
+2. start testpmd with 100G setting, add vlan filter,
+   then set profile and TC mapping::
+
+    ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 2-10 -n 4 -a 18:01.0,cap=dcf -a 18:01.1 -- -i --txq=8 --rxq=8 --nb-cores=8 --port-topology=loop
+    port stop all
+    vlan set filter on 1
+    rx_vlan add 1 1
+    rx_vlan add 2 1
+
+    add port tm node shaper profile 0 1 1000000 0 2000000 0 0 0
+    add port tm node shaper profile 0 2 1000000 0 4000000 0 0 0
+    add port tm nonleaf node 0 1000 -1 0 1 0 -1 1 0 0
+    add port tm nonleaf node 0 900 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 800 1000 0 1 1 -1 1 0 0
+    add port tm nonleaf node 0 700 1000 0 1 1 -1 1 0 0
+
+    add port tm leaf node 0 0 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 1 900 0 1 2 1 0 0xffffffff 0 0
+    add port tm leaf node 0 2 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 3 800 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 4 700 0 1 2 2 0 0xffffffff 0 0
+    add port tm leaf node 0 5 700 0 1 2 2 0 0xffffffff 0 0
+    port tm hierarchy commit 0 no
+
+    add port tm nonleaf node 1 1000 -1 0 1 0 0 1 0 0
+    add port tm nonleaf node 1 900 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 800 1000 0 1 1 0 1 0 0
+    add port tm nonleaf node 1 700 1000 0 1 1 0 1 0 0
+    add port tm leaf node 1 0 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 1 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 2 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 3 900 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 4 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 5 800 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 6 700 0 1 2 0 0 0xffffffff 0 0
+    add port tm leaf node 1 7 700 0 1 2 0 0 0xffffffff 0 0
+    port tm hierarchy commit 1 no
+    port start all
+    set fwd mac
+    start
+
+3. send 8 streams from IXIA, mac address is "00:11:22:33:44:55",
+   frame size is 68 bytes, each stream desired 25% max rate.
+   stream 0-3, vlan=0, priority=0,1,3,4,
+   stream 4-5, vlan=1, priority=0,3,
+   stream 6-7, vlan=2, priority=2,4.
+   only send stream 0,1,4,6 synchronously, the throughput is 2MBps, map queue 0-3
+   only send steam 2 and 5 synchronously, the throughput is 4MBps, map queue 4-5
+   only send steam 3 and 7 synchronously, the throughput is 4MBps, map queue 6-7
+   send all the streams synchronously, the throughput is 10MBps, queue mapping is correct.