kernelci-labs project

Hello,



Last Thursday there was a bit of hijacking of the Testing Quality call to discuss lab integration in KernelCI prompted by the Qualcomm team looking for the best approach. Together with us we had Texas Instruments who are looking at similar support, Penguntronix who wants to offer labgrid integration, and the usual Collabora gang.



A key challenge is to be able to run labs behind a firewall or to be able to control what and how to run the tests. Maestro already has an API that allows users to connect and listen/pull test events as we pictured in our architecture, so that means we create new components running outside the KernelCI public infra for each lab. We are calling it kernelci-labs for now. Sample service[1] and script[2] exist.



Here we need to make an important distinction: new kernelci-labs will target labs that can't or don't want to let Maestro submit tests directly to them. Other labs, e.g. Baylibre, Collabora and CIP receive jobs directly from Maestro, so they don't need to depend on these kernelci-labs components or change anything in their current configuration. These two approaches might be converged in the future to let kernelci-labs drive all the tests for KernelCI purposes..



Then, with kernelci-labs, each lab drives their test-execution and reports back their results and logs into the maestro API. That's because maestro already knows how to process that data and populate KCIDB for results interaction via notifications or the dashboard. The added benefit here is that maestro <> KCIDB bridge can evolve pretty fast, empowering the data visualization through KCIDB and dashboard. We should expect ongoing improvements in the KCIDB schema and maestro is either proposing these or adopting them instantly.



This would work well for both Qualcomm and Texas. Also the labgrid folks feel this would make it a better fit for labgrid integration given their different task/job description schema (from what has been implemented so far) and the requirement for e.g. a proxy service to allow external submissions.



I attached an updated drawing of our KernelCI architecture based on changes we are evaluating in these discussions.



The conclusion from the meeting is that we need to get started with such components.



As next steps, on the Maestro side Denys will:

- Improve pub/sub to make more flexible (id of the events, ability to request events after NNNN, etc, without constant polling)

- Storage solution for labs, KCIDB, Maestro

- Prepare code for listening events and for sending results from lab to maestro

- Command-line utilities for polling new build info and submitting test results.



Then Qualcomm will jump into integrating their lab (which is possibly going to be our kernelci-labs PoC).



Feedback welcome!



We also have the KernelCI HW Vendors channel on Discord that you can join in for discussions: https://discord.gg/KWbrbWEyqb



Best,



- Gus (on behalf of the KernelCI team)


[1] https://github.com/kernelci/kernelci-pipeline/pull/594

[2] https://gist.github.com/nuclearcat/aec542be9b2f0cb7a7ee3b164d664877





--

Gustavo Padovan

Kernel Lead 



Collabora Ltd. 

Platinum Building, St John's Innovation Park 

Cambridge CB4 0DS, UK 

Registered in England & Wales, no. 5513718