[PATCH v4 0/2] Lucid: A custom Ansible stdout callback for kdevops
Daniel Gomez <[email protected]> Tue, 02 Dec 2025 00:11:15 +0100
| Newsgroups | dev.linux.lists.kdevops |
|---|---|
| Message-ID | <[email protected]> |
The Ansible stdout callback ecosystem is quite wide, supporting multiple output formats and options. However, after testing a variety of them with kdevops workflows, none fit well for our use cases. Most existing callbacks provide verbose output that makes it difficult for users to understand what's actually happening during playbook executions. kdevops relies on Ansible's idempotency principle [1], meaning playbooks are designed to reach the desired state regardless of the current state. Users shouldn't need to worry about which tasks are running or being skipped, the output should simply guide them to the expected result. With this principle in mind and inspired by other projects [2] like bitbake (from OpenEmbedded/Yocto), we can focus the default output on what actually matters: the currently running task, a short list of recent tasks, and elapsed time for the current operation. This is the core principle behind lucid, a new Ansible stdout callback plugin tailored for kdevops where less is more. Errors are always displayed in human-readable format on stdout, while comprehensive logs remain available for detailed inspection of playbook execution. Lucid is designed to be flexible and evolve with community feedback. Users who prefer different output styles can easily configure the plugin through kdevops' Kconfig system. For example, if the default dynamic mode with live updates doesn't suit a particular workflow, users can switch to static scrollable output. The goal is to provide sensible defaults that work for most kernel and kdevops testing scenarios while allowing individual customization when needed. Here's a linux workflow output demo using lucid. The video is long, but the first few seconds already give you a sense of how the plugin works. Jump to 3:36 to see the output logs. The video will be deleted after 7 days: https://asciinema.org/a/fCqZ8S8F0vNBfjkA8FLDGUue5 [1] https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_intro.html#desired-state-and-idempotency [2] https://docs.yoctoproject.org/bitbake/ Signed-off-by: Daniel Gomez <[email protected]> --- Changes in v4: - Replace custom AV makefile variable with Ansible's native verbosity setting in ansible.cfg, controlled via Kconfig, with ANSIBLE_VERBOSITY=N environment variable for runtime override - Link to v3: https://lore.kernel.org/r/[email protected] Changes in v3: - Fix dynamic display flickering by preloading play hosts at task start - Add delegation info display (host -> delegate) matching Ansible default - Use fixed-width columns to prevent display jumping - Always show spinner for running tasks - Add longer time string support to duration formatter - Update documentation for dynamic mode and delegation display - New demo (linux-modules-kpd): https://asciinema.org/a/759342 - Link to v2: https://lore.kernel.org/r/[email protected] Changes in v2: - Drop lucid stdout callback as default (let's discuss this separately) - Add retry support when tasks are displayed - Fix for small terminal width in dynamic mode that prevented tasks to be cleared properly - Change task status symbols to be width consistent - Drop "unused" lucid options and simplify Kconfig. It's unclear the value of them so, let's add them when needed. - Add a new Ansible config to configure Ansible verbosity levels via Kconfig - Add lucid-ci.config fragment that leverages the new kconfig Ansible verbosity configuration to configure lucid in static mode with the required verbosity needed in CI scenarios. - Other minor plugin fixes - Link to v1: https://lore.kernel.org/r/[email protected] --- Daniel Gomez (2): ansible: use native verbosity config instead of custom AV variable ansible: add lucid callback plugin for clean output .gitignore | 4 + CLAUDE.md | 16 +- Makefile | 6 +- Makefile.btrfs_progs | 2 +- Makefile.build_qemu | 8 +- Makefile.docker-mirror | 4 +- Makefile.hypervisor-tunings | 2 +- Makefile.kdevops | 10 +- Makefile.postfix | 2 +- callback_plugins/__init__.py | 0 callback_plugins/lucid.py | 879 +++++++++++++++++++++ defconfigs/configs/lucid-ci.config | 3 + defconfigs/configs/lucid.config | 2 + docs/ansible-callbacks.md | 294 +++++++ docs/fio-tests.md | 2 +- docs/monitoring.md | 2 +- docs/reboot-limit.md | 2 +- kconfigs/Kconfig.ansible_cfg | 94 +++ .../roles/ansible_cfg/templates/ansible.cfg.j2 | 13 +- scripts/ansible.Makefile | 3 - scripts/archive.Makefile | 2 +- scripts/devconfig.Makefile | 4 +- scripts/dynamic-pci-kconfig.Makefile | 2 +- scripts/firstconfig.Makefile | 2 +- scripts/guestfs.Makefile | 18 +- scripts/install-menuconfig-deps.Makefile | 2 +- scripts/install-rcloud-deps.Makefile | 2 +- scripts/iscsi.Makefile | 2 +- scripts/journal-server.Makefile | 10 +- scripts/kconfig/Makefile | 2 +- scripts/kotd.Makefile | 6 +- scripts/krb5.Makefile | 4 +- scripts/ktls.Makefile | 2 +- scripts/nfsd.Makefile | 2 +- scripts/nixos.Makefile | 10 +- scripts/provision.Makefile | 2 +- scripts/rdma.Makefile | 4 +- scripts/smbd.Makefile | 2 +- scripts/systemd-timesync.Makefile | 4 +- scripts/terraform.Makefile | 14 +- scripts/update_etc_hosts.Makefile | 2 +- scripts/validate_av.py | 28 - workflows/ai/Makefile | 20 +- workflows/blktests/Makefile | 16 +- workflows/build-linux/Makefile | 8 +- workflows/common/Makefile | 2 +- workflows/cxl/Makefile | 14 +- workflows/demos/reboot-limit/Makefile | 14 +- workflows/fio-tests/Makefile | 12 +- workflows/fstests/Makefile | 28 +- workflows/gitr/Makefile | 8 +- workflows/linux/Makefile | 28 +- workflows/ltp/Makefile | 8 +- workflows/minio/Makefile | 10 +- workflows/mmtests/Makefile | 16 +- workflows/nfstest/Makefile | 8 +- workflows/pynfs/Makefile | 12 +- workflows/rcloud/Makefile | 4 +- workflows/selftests/Makefile | 8 +- workflows/steady_state/Makefile | 8 +- workflows/sysbench/Makefile | 16 +- workflows/vllm/Makefile | 20 +- 62 files changed, 1500 insertions(+), 234 deletions(-) --- base-commit: c7daa8cf23b3c7bfb8579b161f71c7520a1d088f change-id: 20251121-lucid-adeb42a71d42 Best regards, -- Daniel Gomez <[email protected]>