[PATCH RFC 0/8] Refactor CI
Daniel Gomez <[email protected]>
| Newsgroups | dev.linux.lists.kdevops |
|---|---|
| Message-ID | <[email protected]> |
Refactor the .github/ CI setup into modular and reusable workflows and
actions. This makes it possible to share modules from kdevops repo and
reuse them in other GitHub projects.
Highlights:
* Add support for parallel workflows
* Add support for kdevops selftest CI with xfs_reflink_4k and
blktests_nvme (more coming)
* Run workflow tests daily with the latest linux-next tag and weekly
(Mondays) with the latest Linus tag
* Update commit format in kdevops-results-archive for both CI modes
(samples below)
* Enable reproducible builds and ccache. With a fixed kernel version,
this should reduce kernel build time. However, without fragments it does
not fully work yet. The open question is whether we should enable it
only for kdevops-ci or not for workflow testing. Can we trust ccache for
testing and reporting results? I think it's just fine for kdevops-ci
path.
* Add time metrics to identify bottlenecks and track improvements
(e.g., XFS reflink saved ~1 minute on
"base_image : Copy custom image to base image location"). Report example:
TASKS RECAP ********************************************************************
Tuesday 23 September 2025 19:23:01 +0000 (0:00:42.426) 0:00:42.426 *****
===============================================================================
base_image : Generating the index for debian-13-generic-amd64-daily ---- 28.65s
Gathering Facts --------------------------------------------------------- 4.29s
guestfs : Check for dnsmasq configuration files ------------------------- 0.79s
guestfs : Create storage pool path directory and set group (libvirt system uri) --- 0.60s
guestfs : Check if directory has group write permissions (libvirt system uri) --- 0.57s
guestfs : Get the user who invoked Ansible ------------------------------ 0.54s
guestfs : Check if directory is owned by the correct group (libvirt system uri) --- 0.51s
guestfs : Create kdevops guestfs storage directory (libvirt system uri) --- 0.50s
guestfs : Check if libvirt default network is running ------------------- 0.45s
base_image : Ensure the custom image directory exists ------------------- 0.43s
base_image : Stat /var/lib/libvirt/images/kdevops/guestfs/custom_images/debian-13-generic-amd64-daily/debian-13-generic-amd64-daily.raw --- 0.41s
base_image : Check if the custom image sentinel file already exists ----- 0.40s
base_image : Check if the custom image file already exists -------------- 0.40s
base_image : Check if the custom index exists --------------------------- 0.39s
guestfs : Check if dnsmasq service is enabled --------------------------- 0.38s
guestfs : Check if dnsmasq service is active ---------------------------- 0.36s
base_image : Check if the custom source exists -------------------------- 0.27s
base_image : Create custom upstream OS image ---------------------------- 0.11s
base_image : Copy custom image to base image location (with automatic reflink optimization) --- 0.09s
base_image : Set the pathname of the custom image ----------------------- 0.06s
Commit format for kdevops-ci:
kdevops-ci: xfs_reflink_4k: d03219f7808b ansible: add lightning symbol for changed task status
BUILD INFO:
kdevops: d03219f7808b ("ansible: add lightning symbol for changed task status")
Workflow: xfs_reflink_4k
Test: generic/003
Kernel: v6.15
Duration: 1m 44s
EXECUTION RESULTS:
KERNEL: 6.15.0
CPUS: 8
xfs_reflink_4k: 1 tests, 17 seconds
generic/003 Pass 14s
Totals: 1 tests, 0 skipped, 0 failures, 0 errors, 14s
METADATA:
workflow: xfs_reflink_4k | scope: kdevops | test: generic/003 | requested: v6.15 | actual: v6.15 | result: not ok
CPUS: 8
xfs_reflink_4k: 1 tests, 17 seconds
generic/003 Pass 14s
Totals: 1 tests, 0 skipped, 0 failures, 0 errors, 14s
Commit format for workflow validation on linux:
blktests (linux master): PASS
BUILD INFO:
Kernel: master ("Merge tag 'sched_ext-for-6.17-rc7-fixes' of
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext")
Workflow: blktests
kdevops: 2115e4296e10 ("base_image: optimize VM image copying with reflinks and fix
same-file handling")
Scope: full test suite
Duration: 35m 57s
EXECUTION RESULTS:
Kernel tests results:
Blktests summary:
Tests run: 81, Passed: 81, Failed: 0
Sample passed test:
NQN:blktests-subsystem-1 disconnected 1 controller(s)
NQN:blktests-subsystem-1 disconnected 1 controller(s)
METADATA:
workflow: blktests | tree: linux
ref: master | scope: tests | result: ok
Blktests summary:
Tests run: 81, Passed: 81, Failed: 0
Sample passed test:
NQN:blktests-subsystem-1 disconnected 1 controller(s)
NQN:blktests-subsystem-1 disconnected 1 controller(s) | requested: master | actual: master
Signed-off-by: Daniel Gomez <[email protected]>
---
Daniel Gomez (8):
base_image: optimize VM image copying with reflinks and fix same-file handling
kconfig: add KDEVOPS_MAKE_VERBOSE option for persistent verbose output
defconfig: add ci fragment
ansible: enhance DIY callback output format for improved readability
ansible: add profile_tasks callback timing analysis support
scripts: fix CI_WORKFLOW basename mapping
fstests: fix ansible_host undefined error
github: refactor into reusable actions and workflows
.github/actions/archive/action.yml | 28 +++
.github/actions/cleanup/action.yml | 24 ++
.github/actions/setup/action.yml | 187 +++++++++++++++
.github/actions/test/action.yml | 213 +++++++++++++++++
.github/workflows/destroy.yml | 93 ++++++++
.github/workflows/docker-tests.yml | 2 +-
.github/workflows/fstests.yml | 98 --------
.github/workflows/main.yml | 97 ++++++++
.github/workflows/manual.yml | 122 ++++++++++
.github/workflows/push.yml | 29 +++
.github/workflows/schedule.yml | 54 +++++
Makefile | 7 +
defconfigs/configs/ci.config | 7 +
kconfigs/Kconfig.ansible_cfg | 14 ++
kconfigs/Kconfig.kdevops | 18 ++
.../roles/ansible_cfg/templates/ansible.cfg.j2 | 20 +-
playbooks/roles/base_image/tasks/custom-image.yml | 18 +-
playbooks/roles/fstests/tasks/main.yml | 10 +-
scripts/ci.Makefile | 13 +
scripts/generate_ci_commit_message.sh | 263 +++++++++++++++++++++
scripts/github_output.sh | 9 +
scripts/korg-releases.py | 94 ++++++++
22 files changed, 1305 insertions(+), 115 deletions(-)
---
base-commit: 4b42d3d3636b6b0c04ccc72b3cd766abe1526344
change-id: 20250923-ci-refactor-9fd37649f4f3
Best regards,
--
Daniel Gomez <[email protected]>