[PATCH v6 00/12] Implementation of mldev test application
Srikanth Yalavarthi
syalavarthi at marvell.com
Sat Mar 11 16:08:53 CET 2023
Machine learning device APIs test application
=============================================
This series of patches introduces a test application for machine
learning device APIs. A test framework is implemented with multiple
test enabled, to validate the device, model and fast-path functions.
New tests can be added using the test framework.
List of tests supported
-----------------------
1) device_ops: Test case to validate device re-configuration
2) model_ops: Collection of 4 sub-tests to validate model slow APIs.
Each sub-test would invoke the slow path model APIs (load / start /
stop / unload) in different order.
3) inference_ordered: Test case to validate execution of end-to-end
inferences on ML device with one active model at a time. This test
can execute inference requests for multiple models, with inferences
for a model executed after completion of inferences for a previously
loaded model.
4) inference_interleave: Test case to validate end-to-end inferences
with multiple active models concurrently. This case would work as a
stress test to validate ML device.
Options supported the by tests include burst size for enqueuing and
dequeuing inference requests, enabling multiple queue pairs with a
user specified value for queue size. Support is also enabled for
batch inferencing, output validation and statistics.
v6:
* Enable error count
* Fix segfault due to incorrect i/o size types
* Revert i/o size variables to uint64_t
v5:
* Cleanup of header includes
* Addressed issues with unclean patches
* Changed input/output size variables to uint32_t
* Rebase over main branch
v4:
* Updated model_id as uint16_t
* Updated license info in SVG files
* Updated release notes
v3:
* Code rebase
Srikanth Yalavarthi (12):
app/mldev: implement test framework for mldev
app/mldev: add common test functions
app/mldev: add test case to validate device ops
app/mldev: add test case to validate model ops
app/mldev: add ordered inference test case
app/mldev: add test case to interleave inferences
app/mldev: enable support for burst inferences
app/mldev: enable support for queue pairs and size
app/mldev: enable support for inference batches
app/mldev: enable support for inference validation
app/mldev: enable reporting stats in mldev app
app/mldev: add documentation for mldev test cases
.mailmap | 1 +
MAINTAINERS | 2 +
app/meson.build | 1 +
app/test-mldev/meson.build | 24 +
app/test-mldev/ml_common.h | 29 +
app/test-mldev/ml_main.c | 113 ++
app/test-mldev/ml_options.c | 325 +++++
app/test-mldev/ml_options.h | 57 +
app/test-mldev/ml_test.c | 41 +
app/test-mldev/ml_test.h | 76 ++
app/test-mldev/parser.c | 380 ++++++
app/test-mldev/parser.h | 55 +
app/test-mldev/test_common.c | 136 ++
app/test-mldev/test_common.h | 27 +
app/test-mldev/test_device_ops.c | 228 ++++
app/test-mldev/test_device_ops.h | 17 +
app/test-mldev/test_inference_common.c | 1128 +++++++++++++++++
app/test-mldev/test_inference_common.h | 75 ++
app/test-mldev/test_inference_interleave.c | 118 ++
app/test-mldev/test_inference_ordered.c | 116 ++
app/test-mldev/test_model_common.c | 164 +++
app/test-mldev/test_model_common.h | 46 +
app/test-mldev/test_model_ops.c | 428 +++++++
app/test-mldev/test_model_ops.h | 20 +
doc/guides/rel_notes/release_23_03.rst | 8 +
.../tools/img/mldev_inference_interleave.svg | 669 ++++++++++
.../tools/img/mldev_inference_ordered.svg | 528 ++++++++
.../tools/img/mldev_model_ops_subtest_a.svg | 420 ++++++
.../tools/img/mldev_model_ops_subtest_b.svg | 423 +++++++
.../tools/img/mldev_model_ops_subtest_c.svg | 366 ++++++
.../tools/img/mldev_model_ops_subtest_d.svg | 424 +++++++
doc/guides/tools/index.rst | 1 +
doc/guides/tools/testmldev.rst | 441 +++++++
33 files changed, 6887 insertions(+)
create mode 100644 app/test-mldev/meson.build
create mode 100644 app/test-mldev/ml_common.h
create mode 100644 app/test-mldev/ml_main.c
create mode 100644 app/test-mldev/ml_options.c
create mode 100644 app/test-mldev/ml_options.h
create mode 100644 app/test-mldev/ml_test.c
create mode 100644 app/test-mldev/ml_test.h
create mode 100644 app/test-mldev/parser.c
create mode 100644 app/test-mldev/parser.h
create mode 100644 app/test-mldev/test_common.c
create mode 100644 app/test-mldev/test_common.h
create mode 100644 app/test-mldev/test_device_ops.c
create mode 100644 app/test-mldev/test_device_ops.h
create mode 100644 app/test-mldev/test_inference_common.c
create mode 100644 app/test-mldev/test_inference_common.h
create mode 100644 app/test-mldev/test_inference_interleave.c
create mode 100644 app/test-mldev/test_inference_ordered.c
create mode 100644 app/test-mldev/test_model_common.c
create mode 100644 app/test-mldev/test_model_common.h
create mode 100644 app/test-mldev/test_model_ops.c
create mode 100644 app/test-mldev/test_model_ops.h
create mode 100644 doc/guides/tools/img/mldev_inference_interleave.svg
create mode 100644 doc/guides/tools/img/mldev_inference_ordered.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_a.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_b.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_c.svg
create mode 100644 doc/guides/tools/img/mldev_model_ops_subtest_d.svg
create mode 100644 doc/guides/tools/testmldev.rst
--
2.17.1
More information about the dev
mailing list