Re: [PATCH v2 0/5] kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing

Hi Leo,

On 28/01/25 00:35, Leonardo Brás wrote:
> On Mon, 2025-01-27 at 13:02 +0530, Vignesh Raman wrote:
>> Hi Leo,
>>
>> On 24/01/25 13:42, Laurent Pinchart wrote:
>>> On Fri, Jan 24, 2025 at 02:11:26AM -0300, Leonardo Brás wrote:
>>>> On Thu, 2025-01-23 at 19:23 +0530, Vignesh Raman wrote:
>>>>> This patch series introduces "kci-gitlab," a GitLab CI pipeline
>>>>> specifically designed for kernel testing. It provides kernel
>>>>> developers with an integrated, efficient, and flexible testing
>>>>> framework using GitLab's CI/CD capabilities. This patch includes
>>>>> a .gitlab-ci file in the tools/ci/gitlab-ci/ folder, along with
>>>>> additional YAML and script files, to define a basic test pipeline
>>>>> triggered by code pushes to a GitLab-CI instance.
>>>>> The initial version implements:
>>>>>       
>>>>> - Static checks: Includes checkpatch and smatch for code validation.
>>>>> - Build tests: Covers various architectures and configurations.
>>>>> - Boot tests: Utilizes virtme for basic boot testing.
>>>>>
>>>>> Additionally, it introduces a flexible "scenarios" mechanism to
>>>>> support subsystem-specific extensions.
>>>>>
>>>>> This series also introduces a drm scenario that adds a job to run IGT
>>>>> tests for vkms. This scenario includes helper scripts to build deqp-runner
>>>>> and IGT, leveraging approaches from the drm-ci/mesa-ci project.
>>>>>
>>>>> We are working towards creating a generic, upstream GitLab-CI pipeline
>>>>> (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline
>>>>> is designed with a distributed infrastructure model, making it possible
>>>>> to run on any gitLab instance. We plan to leverage KernelCI [2] as the
>>>>> backend, utilizing its hardware, rootfs, test plans, and KCIDB [3]
>>>>> integration.
>>>>>
>>>>> For an example of a fully executed pipeline with drm scenario set,
>>>>> including documentation generation,
>>>>> see: https://gitlab.freedesktop.org/vigneshraman/kernel/-/pipelines/1350262
>>>>>
>>>>> Please refer to the documentation included in the patch, or check the
>>>>> rendered version, here:
>>>>> https://vigneshraman.pages.freedesktop.org/-/kernel/-/jobs/69787927/artifacts/artifacts/Documentation-output/ci/gitlab-ci/gitlab-ci.html
>>>>>
>>>>> Differences from v1 to v2:
>>>>> - moved to tools/ci as suggested by Linus on the previous version
>>>>> - add arm64 containers for native compilation
>>>>> - added boot tests using virtme: this is the base structure for boot tests,
>>>>>     next steps would be adding other tests such as kselftests/kunit tests
>>>>> - added DRM scenario testing on vkms: this should replace current vkms test
>>>>>     in drm-ci. This work shows how a test scenario can be used by different
>>>>>     subsystems to add their tests.
>>>>> - update documentation
>>>>>
>>>>> For more details on the motivation behind this work, please refer to the
>>>>> cover letter of v1: https://patchwork.kernel.org/project/linux-kselftest/cover/[email protected]/
>>>>>
>>>>> [1] https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-pipeline-for-linux-kernel-testing/
>>>>> [2] https://kernelci.org/
>>>>> [3] https://docs.kernelci.org/kcidb/
>>>>>
>>>>> Helen Koike (3):
>>>>>     kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing
>>>>>     kci-gitlab: Add documentation
>>>>>     kci-gitlab: docs: Add images
>>>>>
>>>>> Vignesh Raman (2):
>>>>>     MAINTAINERS: Add an entry for ci automated testing
>>>>>     kci-gitlab: Add drm scenario
>>>>
>>>> Hi Vignesh Raman,
>>>> I am very happy to see this project going forward :)
>>>>
>>>> It's been a few years since I first thought on finding a good way of helping
>>>> kernel developers testing their patches, while making use of the free runner
>>>> minutes Gitlab offers. It can greatly simplify the testing for people who are
>>>> new to kernel development, or students trying to understand it better.
>>>>
>>>> And this patchset allows that to happen :)
>>
>> Thanks for testing the patch and providing feedback.
> 
> :)
> 
>>
>>>>
>>>> Actually, I spoke to Helen last year, and to enable it to run on the free
>>>> Gitlab-CI runners, there is a small extra patch which is needed:
>>>>
>>>> https://lore.kernel.org/all/[email protected]/
>>
>> Thanks for the patch.
> 
> Thank you for most of the work, it sure will be useful :)
> 
>>
>> We can keep the variable name as KCI_CI_TAGS (like other variables) and
>> add it in variables with default empty string.
> 
> Just tested, if it's an empty string, it will just be ignored, as it won't match
> anything (that is not an empty string)

Ack. Adding it to variables will allow us pass KCI_CI_TAGS when manually 
creating a pipeline without changing the CI/CD settings, but we can 
ignore this for now.

> 

>>
>>>
>>> Gitlab as an open-source software project (the community edition) is one
>>> thing, but can we please avoid advertising specific proprietary services
>>> in the kernel documentation ?
>>
>> Agree. We could remove the lines which mentions gitlab.com SaaS in
>> commit message and documentation.
> 
> Done :)
> Added a generic reason for needing better instance
> 
>>
>>>
>>>> Could you please apply it on top of your tree?
>>>> Some stuff changed places, but I can send a v2 with that fix if you want.
>>
>> I will test your patch and we can add this for v3.
> 
> Thanks!
> That's the v2 with the required changes:
> 
> https://lore.kernel.org/all/[email protected]/T/#u
> 
> Please let me know if any further change is needed.


Thanks for the patch. We will pick this for v3.

Regards,
Vignesh

> 
> Thanks!
> Leo
> 
>>
>> Regards,
>> Vignesh
>>
>>>>
>>>>
>>>> While I have yet to review this v2 patchset, I applied it on my repo for
>>>> testing, and cherry-picked the patch on above link, triggering a Pipeline:
>>>>
>>>> https://gitlab.com/linux-kernel/linux/-/pipelines/1638955600
>>>>
>>>> It seems to be working fine, please check it, as you may be more used to the
>>>> results.
>>>>
>>>> Thanks!
>>>> Leo
>>>
>