Bug 1351 - Skip test cases based on testbed capabilities
Summary: Skip test cases based on testbed capabilities
Status: IN_PROGRESS
Alias: None
Product: DPDK
Classification: Unclassified
Component: DTS (show other bugs)
Version: unspecified
Hardware: All All
: High normal
Target Milestone: ---
Assignee: Juraj Linkeš
URL:
Depends on:
Blocks:
 
Reported: 2024-01-10 10:38 CET by Juraj Linkeš
Modified: 2024-04-25 15:52 CEST (History)
4 users (show)



Attachments

Description Juraj Linkeš 2024-01-10 10:38:21 CET
The different hardware DPDK supports is not homogeneous and support a variety of different features. A user-friendly solution would probe the hardware under test and skip all test cases that cannot be executed due to missing features.

There are multiple dimensions to this task.
1. Device probing. We can leverage existing DPDK probing capabilities in a DPDK app. A machine-readable output format must be defined, such as json. The fields themselves must be well-defined.
2. Test case/suite marking with required capabilities. If a test case isn't marked, it should run on all hardware. There could be different names for the same feature across different hardware, in which case we should decide whether we want to merge these under one DTS-defined name or we'll mark each test case/suite with multiple names.
Comment 1 Juraj Linkeš 2024-01-10 10:54:05 CET
Another thing to consider is making the TG optiona
Comment 2 Thomas Monjalon 2024-01-10 14:52:12 CET
DTS should test no more than DPDK API.
Feature capabilities can be checked via some DPDK API.
If there is no capability API for some features which are not supported by a hardware, it should be considered a bug.

Please do not build any HW database outside of the drivers.
Comment 3 Juraj Linkeš 2024-01-11 13:08:30 CET
(In reply to Thomas Monjalon from comment #2)
> DTS should test no more than DPDK API.
> Feature capabilities can be checked via some DPDK API.

This is what we want to use. Ideally we won't add anything new, just leverage existing APIs to check/list capabilities. I don't know how exactly this works and my wording reflects that. Is there a place where we could learn more (how to use the APIs, what they are etc.)? 

> If there is no capability API for some features which are not supported by a
> hardware, it should be considered a bug.
> 
> Please do not build any HW database outside of the drivers.
Comment 4 Thomas Monjalon 2024-01-22 17:02:44 CET
Each feature category may have a different API to check capabilities.
For instance, for rte_flow, the main idea is to "try" with a "validate" call.
Metadata have a function rte_eth_rx_metadata_negotiate().
A lot of ethdev capabilities are returned in struct rte_eth_dev_info.
etc, etc...

Unfortunately it is not unified.
For each feature, you need to look for how the capability can be checked.
If you feel it is badly documented or something is missing, then consider it is a bug to report.

Thank you

Note You need to log in before you can comment on or make changes to this bug.