[dts] [PATCH] tests/dynamic_queue: add dynamic queue test suite

[Lin, Xueqin]

Thanks Marvin a lot for you review. My comment as below, I will send v2 version to fix them.

Best regards,
Xueqin

> -----Original Message-----
> From: Liu, Yong
> Sent: Monday, August 20, 2018 3:31 PM
> To: Lin, Xueqin <xueqin.lin at intel.com>; dts at dpdk.org
> Cc: Lin, Xueqin <xueqin.lin at intel.com>
> Subject: RE: [dts] [PATCH] tests/dynamic_queue: add dynamic queue test
> suite
> 
> > -----Original Message-----
> > From: dts [mailto:dts-bounces at dpdk.org] On Behalf Of Xueqin Lin
> > Sent: Tuesday, August 07, 2018 12:24 PM
> > To: dts at dpdk.org
> > Cc: Lin, Xueqin <xueqin.lin at intel.com>
> > Subject: [dts] [PATCH] tests/dynamic_queue: add dynamic queue test
> > suite
> >
> > From: "xueqin.lin" <xueqin.lin at intel.com>
> >
> > Signed-off-by: Xueqin Lin <xueqin.lin at intel.com>
> >
> > ---
> >  tests/TestSuite_dynamic_queue.py | 180
> > +++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 180 insertions(+)
> >  create mode 100644 tests/TestSuite_dynamic_queue.py
> >
> > diff --git a/tests/TestSuite_dynamic_queue.py
> > b/tests/TestSuite_dynamic_queue.py
> > new file mode 100644
> > index 0000000..880d421
> > --- /dev/null
> > +++ b/tests/TestSuite_dynamic_queue.py
> > @@ -0,0 +1,180 @@
> > +# <COPYRIGHT_TAG>
> > +
> > +import time
> > +import re
> > +import utils
> > +from test_case import TestCase
> > +from pmd_output import PmdOutput
> > +from settings import get_nic_name
> > +from packet import Packet, sniff_packets, load_sniff_packets import
> > +random
> > +
> > +
> > +class TestDynamicQueue(TestCase):
> > +
> > +    def set_up_all(self):
> > +        self.dut_ports = self.dut.get_ports(self.nic)
> > +        self.verify(len(self.dut_ports) >= 1, "Insufficient ports")
> > +        out = self.dut.send_expect("cat config/common_base", "]# ", 10)
> > +        self.PF_Q_strip = 'CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF'
> > +        pattern = "%s=(\d*)" % self.PF_Q_strip
> > +        self.PF_QUEUE = self.element_strip(out, pattern)
> 
> We can strip config value by existing API self.dut.get_def_rte_config. Please
> use that one.

If use self.dut.get_def_rte_config, return value is None, it is wrong to strip the PF queue number.
The function call below to return search loop, the out range is wrong.
out = self.session.send_command("cat config/defconfig_%s | sed '/^#/d' | sed '/^\s*$/d'"
                                        % self.target, 1)

> 
> > +        self.used_dut_port = self.dut_ports[0]
> > +        tester_port = self.tester.get_local_port(self.used_dut_port)
> > +        self.tester_intf = self.tester.get_interface(tester_port)
> > +        self.dut_testpmd = PmdOutput(self.dut)
> > +
> > +    def set_up(self):
> > +        self.dut_testpmd.start_testpmd(
> > +            "Default", "--port-topology=chained --txq=%s --rxq=%s"
> > +            % (self.PF_QUEUE, self.PF_QUEUE))
> > +
> > +    def element_strip(self, out, pattern):
> > +        """
> > +        Strip and get queue number.
> > +        """
> > +        s = re.compile(pattern, re.DOTALL)
> > +        res = s.search(out)
> > +        if res is None:
> > +            print utils.RED('Fail to search number.')
> > +            return None
> > +        else:
> > +            result = res.group(1)
> > +            return int(result)
> > +
> > +    def send_packet(self):
> > +        """
> > +        Generate packets and send them to dut
> > +        """
> > +        mac = self.dut.get_mac_address(0)
> > +        for i in range(self.PF_QUEUE * 2):
> > +            pkt = Packet(pkt_type='IP_RAW')
> > +            pkt.config_layer('ether', {'dst': mac})
> > +            pkt.config_layer(
> > +                'ipv4', {'dst': '192.168.0.%d' % i, 'src': '191.168.0.1'})
> > +            pkt.send_pkt(tx_port=self.tester_intf)
> > +
> > +    def rxq_setup_test(self, chgflag=0):
> > +        """
> > +        Dynamic to setup rxq and reconfigure ring size at runtime.
> > +        chgflag: reconfigure ring size flag
> > +                 1:reconfigure Rx ring size
> > +                 0:no change on Rx ring size
> > +        """
> > +        queue = range(3)
> 
> Queue is just list here, why not just use list() ?
Agree, will fix it.
> 
> > +        for i in range(3):
> 
> Recommend not use hard-code 3 here, you can define one global variable for
> test loop.
Agree, will fix it.

> 
> > +            queue[i] = random.randint(1, self.PF_QUEUE - 1)
> > +            self.dut_testpmd.execute_cmd('port 0 rxq %d stop' %
> > + queue[i])
> 
> Please add blank line here for different code block.
NP, will add it.

> 
> > +        self.dut_testpmd.execute_cmd('set fwd rxonly')
> > +        self.dut_testpmd.execute_cmd('start')
> > +        self.send_packet()
> > +        self.dut.get_session_output(timeout=10)
> > +        out = self.dut_testpmd.execute_cmd('stop')
> > +        for i in range(3):
> > +            self.verify(
> > +                "Forward Stats for RX Port= 0/Queue=%2d" % queue[i]
> > + not in
> > out,
> > +                "Fail to verify rxq stop!")
> 
> Check queue stop by "Forward stats" not shown is not straight-forward. Is
> there any other method like check rx stats?

Don't find other better method to verify result. But if queue stop, don't have the Rx forward stats on this queue, if queue start,  have the forward stats.
Also check this with dev, he is agree this method. 
If you have other better idea, pls let me know, thanks.

> 
> > +        if chgflag == 1:
> > +            for i in range(3):
> > +                out = self.dut_testpmd.execute_cmd(
> > +                        'show rxq info 0 %d' % queue[i])
> > +                qring_strip = 'Number of RXDs: '
> > +                pattern = "%s([0-9]+)" % qring_strip
> > +                qringsize = self.element_strip(out, pattern)
> > +                chg_qringsize = qringsize % 1024 + 256
> 
> Xueqin, why first mod 1024 and then add 256? Why not just add 256?
If qringsize is max value, add 256 may not set successfully.

> 
> > +                self.dut_testpmd.execute_cmd(
> > +                    'port config 0 rxq %d ring_size %d'
> > +                    % (queue[i], chg_qringsize))
> > +                self.dut_testpmd.execute_cmd('port 0 rxq %d setup' %
> > queue[i])
> > +                out = self.dut_testpmd.execute_cmd(
> > +                    'show rxq info 0 %d' % queue[i])
> > +                chk_qringsize = self.element_strip(out, pattern)
> > +                self.verify(chk_qringsize == chg_qringsize,
> > +                            "Fail to change ring size at runtime!")
> > +        for i in range(3):
> > +            if chgflag == 0:
> > +                self.dut_testpmd.execute_cmd('port 0 rxq %d setup' %
> > queue[i])
> > +            self.dut_testpmd.execute_cmd('port 0 rxq %d start' % queue[i])
> > +        self.dut_testpmd.execute_cmd('start')
> > +        self.send_packet()
> > +        self.dut.get_session_output(timeout=10)
> > +        out = self.dut_testpmd.execute_cmd('stop')
> 
> 
> Please add blank line here, one line comment will be helpful.
Okay.

> 
> > +        for i in range(3):
> > +            self.verify("Forward Stats for RX Port= 0/Queue=%2d"
> > +                        % queue[i] in out, "Fail to setup rxq %d at runtime"
> > +                        % queue[i])
> > +
> > +    def txq_setup_test(self, chgflag=0):
> > +        """
> > +        Dynamic to setup txq and reconfigure ring size at runtime.
> > +        chgflag: reconfigure ring size flag
> > +                 1:reconfigure Tx ring size
> > +                 0:no change on Tx ring size
> > +        """
> > +        for i in range(3):
> > +            queue = random.randint(1, self.PF_QUEUE - 1)
> > +            out = self.dut_testpmd.execute_cmd('show txq info 0 %d' % queue)
> > +            qring_strip = 'Number of TXDs: '
> > +            pattern = "%s([0-9]+)" % qring_strip
> > +            qringsize = self.element_strip(out, pattern)
> > +            self.dut_testpmd.execute_cmd('port 0 txq %d stop' % queue)
> > +            self.dut_testpmd.execute_cmd('set fwd txonly')
> > +            self.dut_testpmd.execute_cmd('start')
> > +            time.sleep(10)
> > +            out = self.dut_testpmd.execute_cmd('stop')
> > +            tx_num = qringsize - 1
> > +            self.verify("TX-packets: %d" % tx_num in out,
> > +                        "Fail to stop txq at runtime")
> > +            if chgflag == 1:
> > +                chg_qringsize = qringsize % 1024 + 256
> > +                self.dut_testpmd.execute_cmd(
> > +                    'port config 0 txq %d ring_size %d'
> > +                    % (queue, chg_qringsize))
> > +                self.dut_testpmd.execute_cmd('port 0 txq %d setup' % queue)
> > +                out = self.dut_testpmd.execute_cmd(
> > +                    'show txq info 0 %d' % queue)
> > +                chk_qringsize = self.element_strip(out, pattern)
> > +                self.verify(chk_qringsize == chg_qringsize,
> > +                            "Fail to change ring size at runtime!")
> > +            if chgflag == 0:
> > +                self.dut_testpmd.execute_cmd('port 0 txq %d setup' % queue)
> > +            self.dut_testpmd.execute_cmd('port 0 txq %d start' % queue)
> > +            self.dut_testpmd.execute_cmd('start')
> > +            time.sleep(10)
> > +            out = self.dut_testpmd.execute_cmd('stop')
> > +            self.verify("TX-packets: %d" % tx_num not in out,
> > +                        "Fail to setup txq at runtime")
> 
> Tx stats should be much larger than tx_num, please check that number.

If queue stops, only fwd 255 packets(ringsize is 256) on this queue, other packets will be dropped.
If queue setups and starts, Tx stats on his queue is bigger than 255.

------- Forward Stats for RX Port= 0/Queue= 2 -> TX Port= 0/Queue= 2 -------
  RX-packets: 0              TX-packets: 255            TX-dropped: 177864385


> 
> > +            if chgflag == 1:
> > +                chgtx_num = chg_qringsize - 1
> > +                self.verify("TX-packets: %d" % chgtx_num not in out,
> > +                            "Fail to change txq ring size at
> > + runtime")
> > +
> 
> Same as previous comment.
> 
> > +    def test_rxq_setup(self):
> > +        """
> > +        Dynamic to setup rxq test
> > +        """
> > +        self.rxq_setup_test()
> > +
> > +    def test_rxq_chgring_setup(self):
> > +        """
> > +        Dynamic to setup rxq and change ring size test
> > +        """
> > +        self.rxq_setup_test(chgflag=1)
> > +
> > +    def test_txq_setup(self):
> > +        """
> > +        Dynamic to setup txq test
> > +        """
> > +        self.txq_setup_test()
> > +
> > +    def test_txq_chgring_setup(self):
> > +        """
> > +        Dynamic to setup txq and change ring size test
> > +        """
> > +        self.txq_setup_test(chgflag=1)
> > +
> > +    def tear_down(self):
> > +        self.dut_testpmd.quit()
> > +
> > +    def tear_down_all(self):
> > +        pass
> > --
> > 2.7.5