Bug 1357

Summary:	Test/test suite pattern
Product:	DPDK	Reporter:	Juraj Linkeš (juraj.linkes)
Component:	DTS	Assignee:	dev
Status:	UNCONFIRMED ---
Severity:	major	CC:	juraj.linkes, probb
Priority:	Low
Version:	unspecified
Target Milestone:	---
Hardware:	All
OS:	All

Description Juraj Linkeš 2024-01-10 11:58:19 CET

A test defines a testing procedure without any particular values to test. A test case is a particular instance of a test with all the values defined.

A usual pattern of test case development is a test method implementing the test procedure and a number of test cases calling the procedure with different values.

If there are multiple variables with different values, the number of (possible) test cases could be very high, which would be tedious and error-prone to implement and maintain. This could be alleviated with scripts, but at that point, we could have a more centrallized solution where the definition of test cases could be just a matrix/dictionary of all the vectors of values to test.

The question is whether this is worth implementing (Are there going to be that many test cases? Do we want that many?)?

The solution needs to be cognisant of how the test cases are going to be executed and recorded and most importantly, how are they going to be recorded in case they're blocked or skipped (i.e. when not executed).