patch 'test: do not count skipped tests as executed' has been queued to stable release 23.11.1

[Xueming Li]

Hi,

FYI, your patch has been queued to stable release 23.11.1

Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet.
It will be pushed if I get no objections before 04/15/24. So please
shout if anyone has objections.

Also note that after the patch there's a diff of the upstream commit vs the
patch applied to the branch. This will indicate if there was any rebasing
needed to apply to the stable branch. If there were code changes for rebasing
(ie: not only metadata diffs), please double check that the rebase was
correctly done.

Queued patches are on a temporary branch at:
https://git.dpdk.org/dpdk-stable/log/?h=23.11-staging

This queued commit can be viewed at:
https://git.dpdk.org/dpdk-stable/commit/?h=23.11-staging&id=e8dccbca30cec3504b043b3e4da6951c0f9f8168

Thanks.

Xueming Li <xuemingl at nvidia.com>

---
>From e8dccbca30cec3504b043b3e4da6951c0f9f8168 Mon Sep 17 00:00:00 2001
From: Bruce Richardson <bruce.richardson at intel.com>
Date: Mon, 13 Nov 2023 15:05:33 +0000
Subject: [PATCH] test: do not count skipped tests as executed
Cc: Xueming Li <xuemingl at nvidia.com>

[ upstream commit a620df6df6d61660661afade09760b2dfba4eb42 ]

The logic around skipped tests is a little confusing in the unit test
runner.
* Any explicitly disabled tests are counted as skipped but not
  executed.
* Any tests that return TEST_SKIPPED are counted as both skipped and
  executed, using the same statistics counters.

This makes the stats very strange and hard to correlate, since the
totals don't add up.  One would expect that SKIPPED + EXECUTED +
UNSUPPORTED == TOTAL, and that PASSED + FAILED == EXECUTED.

To achieve this, mark any tests returning TEST_SKIPPED, or ENOTSUP as
not having executed.

Signed-off-by: Bruce Richardson <bruce.richardson at intel.com>
Acked-by: Akhil Goyal <gakhil at marvell.com>
Acked-by: Ciara Power <ciara.power at intel.com>
Acked-by: Tyler Retzlaff <roretzla at linux.microsoft.com>
---
 app/test/test.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/app/test/test.c b/app/test/test.c
index bfa9ea52e3..7b882a59de 100644
--- a/app/test/test.c
+++ b/app/test/test.c
@@ -375,11 +375,13 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 
 			if (test_success == TEST_SUCCESS)
 				suite->succeeded++;
-			else if (test_success == TEST_SKIPPED)
+			else if (test_success == TEST_SKIPPED) {
 				suite->skipped++;
-			else if (test_success == -ENOTSUP)
+				suite->executed--;
+			} else if (test_success == -ENOTSUP) {
 				suite->unsupported++;
-			else
+				suite->executed--;
+			} else
 				suite->failed++;
 		} else if (test_success == -ENOTSUP) {
 			suite->unsupported++;
-- 
2.34.1

---
  Diff of the applied patch vs upstream commit (please double-check if non-empty:
---
--- -	2024-04-13 20:43:07.128290849 +0800
+++ 0068-test-do-not-count-skipped-tests-as-executed.patch	2024-04-13 20:43:05.017753905 +0800
@@ -1 +1 @@
-From a620df6df6d61660661afade09760b2dfba4eb42 Mon Sep 17 00:00:00 2001
+From e8dccbca30cec3504b043b3e4da6951c0f9f8168 Mon Sep 17 00:00:00 2001
@@ -4,0 +5,3 @@
+Cc: Xueming Li <xuemingl at nvidia.com>
+
+[ upstream commit a620df6df6d61660661afade09760b2dfba4eb42 ]
@@ -20,2 +22,0 @@
-Cc: stable at dpdk.org
-
@@ -31 +32 @@
-index 8b25615913..680351f6a3 100644
+index bfa9ea52e3..7b882a59de 100644
@@ -34 +35 @@
-@@ -369,11 +369,13 @@ unit_test_suite_runner(struct unit_test_suite *suite)
+@@ -375,11 +375,13 @@ unit_test_suite_runner(struct unit_test_suite *suite)