[RFC] Testpmd Maintenance affecting DTS

[Owen Hilyard]

Currently, DTS tests DPDK via the example applications (/examples) and the
apps (/app) included with DPDK. The DTS Working group has been trying to
move to primarily using Testpmd, since it is the most complete and only
using one application causes less maintenance work. Since Testpmd provides
a text interface into DPDK, it must be continually updated to expose this
new functionality, sometimes it may lag behind the current feature set of
DPDK. For example, Testpmd currently does not support all of the options
available for rte flow, which limits the testing coverage that can be
provided by DTS for RTE flow. Because DTS is a Python application, it is
not able to directly interface with DPDK. This means that whenever Testpmd
falls behind DPDK, those unexposed parts of DPDK are functionally
untestable as far as DTS is concerned. DTS is the tool used by the CI and
labs to functionally test DPDK in a working configuration, compared to the
unit testing, that may test individual components or parts of the stack.
The DTS improvements WG and the Community Lab have a goal to continue the
expansion of the test functional and performance test coverage of DPDK.
This means that there must be some way for everything that the DPDK
community wants to be tested to be exposed to DTS. I have a few possible
solutions to this problem:


   1.

   Continue to update Testpmd every time new functionality is added to DPDK
   to expose that functionality
   2.

   Use a parser generator or some other method to make adding new options
   to Testpmd much easier, updating this every time new functionality is added
   to testpmd
   3.

   Create a dedicated testing application for DPDK that uses a binding
   generator and Python’s XMLRPC to allow more programmatic access to DPDK on
   the DUT, adding new RPC calls to expose new functionality.


I personally think that option 3 is the best because it would involve a bit
of work up front to make it much easier to expose parts of DPDK to DTS.
This is because after the initial work is done, we would just need to write
a wrapper function in C to expose the functionality we want, and then
expose the wrapper function via RPC.

If you have thoughts or suggestions, we will be discussing this at the DTS
working group meeting.

2:00 PM UTC, Wednesday, February 16
https://armltd.zoom.us/j/97503259680?pwd=VVlmWnlnTXJkVGkwR2JOU3R3b3Vndz09&from=addon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/dts/attachments/20220208/34ba837c/attachment.htm>