[PATCH 00/12] kdevops: add qemu-system-units bringup backend
Daniel Gomez <[email protected]> Thu, 23 Apr 2026 13:30:52 +0200
| Newsgroups | dev.linux.lists.kdevops |
|---|---|
| Message-ID | <20260423-kdevops-series-c-qemu-system-units-v1-0-b7bab3225a36@samsung.com> |
kdevops bringup today goes through libvirt, libguestfs, and virsh: a stack of wrappers, a root daemon on the host, sudo on the controller node for install, configure, and image manipulation steps, and LSM policy drift that hits most distros one way or another. Users have asked for more direct control over the QEMU process itself, with fewer layers between the vars file and the command line that actually boots the guest. But QEMU has two main challenges. One is the command-line incantation: QEMU has no drop-in config, no standard env-file convention. Building the right invocation for a given machine type, device topology, firmware, and boot mode is entirely on the caller, and has been an open conversation upstream for years. The other side is process management: starting, stopping, logging, dependency ordering, socket-activating virtiofsd, resource control. QEMU does not solve either, and the two together are what every existing wrapper ends up re-implementing. There are many such wrappers - virtme-ng [1], systemd-vmspawn [2], vmctl [3], and others - and all of them are opinionated. Each picks a workflow, hides or transforms QEMU flags to fit it, and ships its own defaults. Qemu-system-units (qsu) takes the opposite approach. Templates render the QEMU command line into a systemd EnvironmentFile, and systemd runs the resulting qemu-system@<name>.service. There is no new daemon, no new wrapper, no kdevops-specific lifecycle code. Template variables map one-to-one to QEMU flag names, so reading the vars file tells you exactly what QEMU will see. The lifecycle half - cgroups, journald, machined registration, socket-activated virtiofsd, QMP graceful shutdown - is delegated to systemd, which does it already. Out of the box, qsu covers host-guest shares over virtiofs, 9p, NVMe emulation, AF_VSOCK, PCIe passthrough through VFIO, direct kernel boot, imageless boot with a tmpfs root, cloud-image boot, GDB, and resource control. All of it configured through the same vars file, all of it running as user-scope systemd units by default. This series integrates qsu into kdevops as an opt-in bringup backend. qsu is imported into scripts/qemu-system-units/ in a single commit with the git-subtree-dir / git-subtree-split trailers so future syncs with upstream use git subtree pull/push. A new QEMU_SYSTEM_UNITS Kconfig symbol turns on the backend and selects KDEVOPS_ENABLE_NIXOS automatically; the guest is NixOS booted imageless, with /nix/store and /lib/modules served to the guest over virtiofs from the host. The bootlinux role gains a controller build mode so the test kernel is built once on the control node and consumed by every guest over the same virtiofs share, avoiding per-guest deployments. SSH defaults to AF_VSOCK via systemd-ssh-generator and systemd-ssh-proxy, which skips the SLIRP banner-timeout race that TCP-over-user-mode networking is prone to on first connect; a -tcp alias is always available as a fallback. A defconfig-qemu-system-units target makes the whole thing a single command away. libvirt bringup is untouched and remains the default. Users who do not opt in see no change. Follow-on work beyond this series: wiring workflows such as blktests and fstests under qsu (the role needs a split so Debian, Fedora, Opensuse specific host setup is skipped on NixOS, the /data share wired up, test devices attached through qsu vars). A large chunk of the existing host preparation - installing libvirt and libguestfs packages, configuring libvirtd, creating storage pools and default networks, tweaking AppArmor and SELinux, managing kvm group membership through sudo - is no longer needed when the chosen backend is qsu, and can be pruned from kdevops on that path. Qsu only needs a one-time sudo on the host to add the user to the kvm and systemd-journal groups and, for PCIe passthrough, to load vfio-pci and install the VFIO udev rules; the user runs those commands once by hand instead of Ansible playbooks re-escalating privileges on every bringup. Link: https://github.com/arighi/virtme-ng.git [1] Link: https://www.freedesktop.org/software/systemd/man/259/systemd-vmspawn.html [2] Link: https://github.com/SamsungDS/vmctl [3] Signed-off-by: Daniel Gomez <[email protected]> --- Daniel Gomez (12): scripts: import qemu-system-units at scripts/qemu-system-units Kconfig: add QEMU_SYSTEM_UNITS as a new bringup method Kconfig: add qemu-system-units backend knobs in their own namespace kdevops: introduce KDEVOPS_CONTROLLER_DATA_PATH for controller artefacts workflows/linux: add controller build mode bootlinux: prefer config-qsu under controller + QEMU_SYSTEM_UNITS qemu_system_units: add role for imageless NixOS bringup qemu_system_units: render three ssh_config stanzas with vsock default gen_hosts: apply NixOS python interpreter default under QEMU_SYSTEM_UNITS playbooks: migrate ansible_env and ansible_os_family to ansible_facts defconfigs: add qemu-system-units docs: add kdevops-qemu-system-units backend overview defconfigs/qemu-system-units | 24 + docs/kdevops-qemu-system-units.md | 100 + kconfigs/Kconfig.ansible_provisioning | 6 +- kconfigs/Kconfig.bringup | 18 + kconfigs/Kconfig.kdevops | 15 + kconfigs/Kconfig.nixos | 18 +- kconfigs/Kconfig.qemu_system_units | 106 + playbooks/qemu_system_units.yml | 7 + .../roles/bootlinux/tasks/build/controller.yml | 207 ++ playbooks/roles/bootlinux/tasks/config.yml | 8 + playbooks/roles/bootlinux/tasks/main.yml | 12 + playbooks/roles/bootlinux/templates/config-qsu | 2905 ++++++++++++++++++++ .../gen_nodes/templates/qemu_system_units_nodes.j2 | 16 + playbooks/roles/nixos/tasks/destroy.yml | 2 +- playbooks/roles/nixos/tasks/install_deps.yml | 4 +- playbooks/roles/nixos/tasks/libvirt_build.yml | 2 +- playbooks/roles/nixos/tasks/ssh_access.yml | 2 +- .../roles/qemu_system_units/tasks/bringup.yml | 87 + .../roles/qemu_system_units/tasks/console.yml | 23 + .../roles/qemu_system_units/tasks/destroy.yml | 68 + .../qemu_system_units/tasks/generate_configs.yml | 74 + .../qemu_system_units/tasks/imageless_build.yml | 32 + .../tasks/imageless_build_per_vm.yml | 102 + .../roles/qemu_system_units/tasks/install_deps.yml | 31 + playbooks/roles/qemu_system_units/tasks/main.yml | 49 + .../roles/qemu_system_units/tasks/render_units.yml | 55 + .../qemu_system_units/templates/default.nix.j2 | 47 + scripts/provision.Makefile | 4 + scripts/qemu-system-units/.gitignore | 11 + scripts/qemu-system-units/CLAUDE.md | 251 ++ scripts/qemu-system-units/COPYING | 22 + scripts/qemu-system-units/LICENSE | 5 + .../LICENSES/preferred/copyleft-next-0.3.1 | 239 ++ scripts/qemu-system-units/README.md | 81 + scripts/qemu-system-units/docs/design-decisions.md | 404 +++ scripts/qemu-system-units/docs/requirements.md | 56 + scripts/qemu-system-units/docs/transient-units.md | 244 ++ scripts/qemu-system-units/docs/usage.md | 378 +++ scripts/qemu-system-units/docs/vars.md | 381 +++ scripts/qemu-system-units/files/network-config | 18 + scripts/qemu-system-units/files/qmp-powerdown | 2 + scripts/qemu-system-units/files/vfio-pci.conf | 1 + scripts/qemu-system-units/templates/meta-data.j2 | 3 + scripts/qemu-system-units/templates/nvme.env.j2 | 126 + .../templates/qemu-system-override.conf.j2 | 83 + .../templates/[email protected] | 56 + .../templates/transient-run.sh.j2 | 72 + scripts/qemu-system-units/templates/user-data.j2 | 44 + .../templates/[email protected] | 21 + .../qemu-system-units/templates/vfio-udev.rules.j2 | 20 + .../qemu-system-units/templates/virtiofsd.env.j2 | 9 + .../templates/[email protected] | 40 + .../templates/[email protected] | 24 + scripts/qemu-system-units/templates/vm.env.j2 | 144 + scripts/qemu-system-units/vars/example.yaml | 105 + scripts/qemu_system_units.Makefile | 38 + scripts/update_ssh_config_nixos.py | 239 +- workflows/linux/Kconfig | 55 + workflows/linux/Makefile | 7 + 59 files changed, 7103 insertions(+), 100 deletions(-) --- base-commit: 1a61f43dc2bc2e019fc4a35d9673cfa31909272d change-id: 20260423-kdevops-series-c-qemu-system-units-770292250a67 prerequisite-change-id: 20260422-kdevops-series-a-fixes-e26bd7d77c76:v1 prerequisite-patch-id: a92bf777d89e14bd149d1967d2b6dfdd704633a4 prerequisite-patch-id: be22a3a7dbdc0d166677baae73157965556b2bea prerequisite-patch-id: 662a8d56c2343e4ccf954f4b8057a75ca464799d prerequisite-patch-id: 5e6b740ec798a1b645d8a2430c705de2727458d4 prerequisite-patch-id: 4cc5044b243438136ea4307cd3ec06109a5b2498 prerequisite-patch-id: dfb1809d2120d252529ae6d1d49d7d2524141894 prerequisite-patch-id: c51be5cfa9f8024a5e34d72e124cceb66f15218d prerequisite-patch-id: d8d5c310220184db0a6d6f4a15d66b11cc4d301a prerequisite-change-id: 20260423-kdevops-series-b-nixos-qemu-05d26a0416c1:v1 prerequisite-patch-id: a92bf777d89e14bd149d1967d2b6dfdd704633a4 prerequisite-patch-id: be22a3a7dbdc0d166677baae73157965556b2bea prerequisite-patch-id: 662a8d56c2343e4ccf954f4b8057a75ca464799d prerequisite-patch-id: 5e6b740ec798a1b645d8a2430c705de2727458d4 prerequisite-patch-id: 4cc5044b243438136ea4307cd3ec06109a5b2498 prerequisite-patch-id: dfb1809d2120d252529ae6d1d49d7d2524141894 prerequisite-patch-id: c51be5cfa9f8024a5e34d72e124cceb66f15218d prerequisite-patch-id: d8d5c310220184db0a6d6f4a15d66b11cc4d301a prerequisite-patch-id: b33ebda455dfca85610859bb1901a7e36ab4e503 prerequisite-patch-id: a1e0e2b42570fff9d73f215756f42d87ec097331 prerequisite-patch-id: 44164fe2e6733332e0733d850a8c12565ee871e8 prerequisite-patch-id: bd865671061675b651bf3f25276230febdbe5244 prerequisite-patch-id: dbd484a85c7b32193cd8bbf00918f09e2b04f8fc prerequisite-patch-id: f867421f574e57d4f1467ddcf7fa08d7a388e18e prerequisite-patch-id: 3cf9ef97e09fb140dfee08b5340cc9c4e70166c9 prerequisite-patch-id: 0d0154c2cfa7e0baba8f6a32197602a7366c3928 Best regards, -- Daniel Gomez <[email protected]>