[dpdk-ci] [RFC] DTS patches and results JSON schema

[Patrick MacArthur]

Hi, all,

I am preparing some patches against DTS to enable T-Rex to work in the
UNH-IOL environment with a couple of test nodes.

The work in progress can be accessed on the next branch at:

    https://oss.iol.unh.edu/dpdk/dts

As part of this, I am proposing the following JSON "schema" that I
would like all vendors' test cases to produce for easy insertion in
the database:

    {
      /* ... */
      "environment": "https://dpdklab.iol.unh.edu/results/environments/9/",
      "results": [
        {
          "frame_size": 64,
          "txd/rxd": 1024,
          "throughput": {
            "actual": 45.783,
            "unit": "Mpps"
          }
        },
        /* ... */
      ]
    }

That is, produce a JSON file with a key results whose value is a list
containing each individual performance result. Each result should
correspond to some expected measurement already in the database.

The parameters "frame_size" and "txd/rxd" and the key "throughput" are
used to look up the proper entries in the measurements given by the
specified environment.

To help you evaluate this proposal in context, I have attached the
latest version of the database schema (updated since the last one I
sent a couple of months ago). To put this in further context, making a
GET request to the environment URL above would return the following
JSON snippet:

{
    /* ... */
    "url": "https://dpdklab.iol.unh.edu/results/environments/9/",
    "measurements": [
        {
            "delta_limit": 7.0,
            "environment":
"https://dpdklab.iol.unh.edu/results/environments/9/",
            "expected_value": 38.908,
            "higher_is_better": true,
            "name": "throughput",
            "parameters": [
                {
                    "name": "frame_size",
                    "unit": "bytes",
                    "value": 64
                },
                {
                    "name": "txd/rxd",
                    "unit": "descriptors",
                    "value": 1024
                }
            ],
            "unit": "Mpps",
            "url": "https://dpdklab.iol.unh.edu/results/measurements/6/"
        },
        /* ... */
    ]
    /* ... */
}

By matching the parameters given in the test result entry against the
expected measurements for the given environment, I can then create the
appropriate test result objects to POST to the results database,
relating each result to its corresponding expected measurement.

I expect to have an RFC patch set posted to the DTS mailing list to
have the upcoming nic_single_core_perf test produce a JSON file as
shown above by our meeting time next week.

Any thoughts/comments/concerns?

Thanks,
Patrick

-- 
Patrick MacArthur
Research and Development, High Performance Networking and Storage
UNH InterOperability Laboratory
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Proposed DPDK Lab Database Schema v3.pdf
Type: application/pdf
Size: 53208 bytes
Desc: not available
URL: <http://dpdk.org/ml/archives/ci/attachments/20180201/fbeb8117/attachment-0001.pdf>