Re: KCIDB: Test schema enhancements

On 8/5/24 12:56 PM, Nikolai Kondrashov wrote:
> Meanwhile I'd like to propose two small, but potentially very useful changes
> to the I/O schema for tests:
>
> * Supporting non-binary outputs beyond PASS/FAIL -
>   integers/floats/booleans/strings/etc. - useful for performance tests.
>
> * Supporting recording `compatible` values from the top of the device tree
>   inside the test environment, for machines which use them - useful for
>   correlating test results by hardware.
>
> Here's the corresponding schema PR:
>
>     https://github.com/kernelci/kcidb-io/pull/85
>
> I'll follow up with a separate message for each of the changes, going over the
> details and the rationale.

The "compatible" property in the root of a device tree specifies the device
vendor, the device (board) model, as well as often the device SoC and family.
These are generally encoded as several strings, ordered from most to least
specific, each potentially containing multiple parts separated by commas.
E.g.:

    "ti,omap3-beagleboard", "ti,omap3450", "ti,omap3"

See more here, for example:

https://docs.kernel.org/devicetree/usage-model.html#platform-identification

Here's the (abbreviated) schema for the new "compatible" field, added to the
"test"'s "environment" object:

    "type": "array",
    "items": {
        "type": "string",
        "pattern": "^[^ ]+(,[^ ]+)*$"
    }

The particular regular expression is based on all I was able to find about
restrictions on the "compatible" values.

All of the above, of course, applies to systems using the device tree, only.
We will have to come up with something else for other systems (ACPI ID?), but
we can already benefit from "compatible", as it's relatively well-defined,
well-recognized, flexible, and the community is already looking after its
consistency and usefulness, so we won't have to come up with anything new
ourselves.

The database implementation for this would also let us correlate results on
more general levels than exact boards, e.g. SOCs, families, and vendors.

For the latter there's already an effort to document actual vendor names,
which would be useful in making dashboards nicer:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/vendor-prefixes.yaml

Don't hesitate to comment here, or in the PR!

Nick