patch 'test: do not count skipped tests as executed' has been queued to stable release 21.11.7

[Kevin Traynor]

Hi,

FYI, your patch has been queued to stable release 21.11.7

Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet.
It will be pushed if I get no objections before 03/13/24. So please
shout if anyone has objections.

Also note that after the patch there's a diff of the upstream commit vs the
patch applied to the branch. This will indicate if there was any rebasing
needed to apply to the stable branch. If there were code changes for rebasing
(ie: not only metadata diffs), please double check that the rebase was
correctly done.

Queued patches are on a temporary branch at:
https://github.com/kevintraynor/dpdk-stable

This queued commit can be viewed at:
https://github.com/kevintraynor/dpdk-stable/commit/3304b1f97b9a9d7b80b52692653aeda7e6e8a1f6

Thanks.

Kevin

---
>From 3304b1f97b9a9d7b80b52692653aeda7e6e8a1f6 Mon Sep 17 00:00:00 2001
From: Bruce Richardson <bruce.richardson at intel.com>
Date: Mon, 13 Nov 2023 15:05:33 +0000
Subject: [PATCH] test: do not count skipped tests as executed

[ upstream commit a620df6df6d61660661afade09760b2dfba4eb42 ]

The logic around skipped tests is a little confusing in the unit test
runner.
* Any explicitly disabled tests are counted as skipped but not
  executed.
* Any tests that return TEST_SKIPPED are counted as both skipped and
  executed, using the same statistics counters.

This makes the stats very strange and hard to correlate, since the
totals don't add up.  One would expect that SKIPPED + EXECUTED +
UNSUPPORTED == TOTAL, and that PASSED + FAILED == EXECUTED.

To achieve this, mark any tests returning TEST_SKIPPED, or ENOTSUP as
not having executed.

Signed-off-by: Bruce Richardson <bruce.richardson at intel.com>
Acked-by: Akhil Goyal <gakhil at marvell.com>
Acked-by: Ciara Power <ciara.power at intel.com>
Acked-by: Tyler Retzlaff <roretzla at linux.microsoft.com>
---
 app/test/test.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/app/test/test.c b/app/test/test.c
index 5194131026..f7375cfe88 100644
--- a/app/test/test.c
+++ b/app/test/test.c
@@ -351,9 +351,11 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 			if (test_success == TEST_SUCCESS)
 				suite->succeeded++;
-			else if (test_success == TEST_SKIPPED)
+			else if (test_success == TEST_SKIPPED) {
 				suite->skipped++;
-			else if (test_success == -ENOTSUP)
+				suite->executed--;
+			} else if (test_success == -ENOTSUP) {
 				suite->unsupported++;
-			else
+				suite->executed--;
+			} else
 				suite->failed++;
 		} else if (test_success == -ENOTSUP) {
-- 
2.43.2

---
  Diff of the applied patch vs upstream commit (please double-check if non-empty:
---
--- -	2024-03-08 13:47:49.955897864 +0000
+++ 0032-test-do-not-count-skipped-tests-as-executed.patch	2024-03-08 13:47:49.040686729 +0000
@@ -1 +1 @@
-From a620df6df6d61660661afade09760b2dfba4eb42 Mon Sep 17 00:00:00 2001
+From 3304b1f97b9a9d7b80b52692653aeda7e6e8a1f6 Mon Sep 17 00:00:00 2001
@@ -5,0 +6,2 @@
+[ upstream commit a620df6df6d61660661afade09760b2dfba4eb42 ]
+
@@ -20,2 +21,0 @@
-Cc: stable at dpdk.org
-
@@ -31 +31 @@
-index 8b25615913..680351f6a3 100644
+index 5194131026..f7375cfe88 100644
@@ -34 +34 @@
-@@ -370,9 +370,11 @@ unit_test_suite_runner(struct unit_test_suite *suite)
+@@ -351,9 +351,11 @@ unit_test_suite_runner(struct unit_test_suite *suite)