[PATCH 07/12] qemu_system_units: add role for imageless NixOS bringup
Daniel Gomez <[email protected]> Thu, 23 Apr 2026 13:30:59 +0200
| Newsgroups | dev.linux.lists.kdevops |
|---|---|
| Message-ID | <20260423-kdevops-series-c-qemu-system-units-v1-7-b7bab3225a36@samsung.com> |
From: Daniel Gomez <[email protected]> Introduce the Ansible side of the QEMU_SYSTEM_UNITS backend as a single play invoking the qemu_system_units role. The role's main.yml dispatches task files by bringup tag (install-deps, generate-configs, build-vms, bringup, console, destroy), same shape as the terraform role. scripts/qemu_system_units.Makefile drives the make targets the provision layer expects (bringup_qemu_system_units, destroy_qemu_system_units) and provision.Makefile sources it when CONFIG_QEMU_SYSTEM_UNITS=y. The role covers the full imageless lifecycle. install_deps.yml resolves the host package requirements for qsu and nix. render_units.yml deploys the shared qemu-system@ and virtiofsd templates into the user-scope systemd config directory, once per host rather than per VM. generate_configs.yml renders the per-node nixos-qemu flake and default.nix: it copies the subtree's imageless template, rewrites the nixos-qemu.url flake input to resolve at the checked-in subtree, and drops in a per-node override module. imageless_build.yml and its companion imageless_build_per_vm.yml nix-build the imageless closure once, then render per-VM vm.env and virtiofsd envs from the bootspec the Nix toolchain writes at <toplevel>/boot.json (init and initrd paths) together with the bootlinux-controller kernel image and modules directory. The per-VM override drop-in receives vsock_cid so machined registers the right CID, and the <destdir>/lib/modules share is pinned at its parent rather than the release subdir so the guest sees /lib/modules/<release>/ modules.dep where modprobe looks for it. bringup.yml starts qemu-system@<vm>.service, restarts the per-share virtiofsd sockets to install fresh listener fds after daemon-reload invalidated them, waits for sshd's SSH-2.0 banner on the forwarded port (a plain TCP accept is not enough under SLIRP because the port opens before sshd binds), and publishes ssh_config entries via update_ssh_config_nixos.py. console.yml prints the three access paths (forwarded ssh, vsock-over-ssh, virtio console socket) and destroy.yml stops the per-VM services and cleans the per-node rendered state. Per-VM ssh_port and vsock_cid derive from the Kconfig bases plus the guest's inventory index, so multiple VMs on one host stay unique without further user input. gen_nodes renders a qsu-shaped nodes yaml under scripts/qemu-system-units/vars/ (qsu's own .gitignore excludes it) that downstream targets like make linux consume. Generated-by: Claude AI Signed-off-by: Daniel Gomez <[email protected]> --- playbooks/qemu_system_units.yml | 7 ++ .../gen_nodes/templates/qemu_system_units_nodes.j2 | 16 ++++ .../roles/qemu_system_units/tasks/bringup.yml | 87 ++++++++++++++++++ .../roles/qemu_system_units/tasks/console.yml | 23 +++++ .../roles/qemu_system_units/tasks/destroy.yml | 68 ++++++++++++++ .../qemu_system_units/tasks/generate_configs.yml | 74 +++++++++++++++ .../qemu_system_units/tasks/imageless_build.yml | 32 +++++++ .../tasks/imageless_build_per_vm.yml | 102 +++++++++++++++++++++ .../roles/qemu_system_units/tasks/install_deps.yml | 31 +++++++ playbooks/roles/qemu_system_units/tasks/main.yml | 49 ++++++++++ .../roles/qemu_system_units/tasks/render_units.yml | 55 +++++++++++ .../qemu_system_units/templates/default.nix.j2 | 47 ++++++++++ scripts/provision.Makefile | 4 + scripts/qemu_system_units.Makefile | 38 ++++++++ 14 files changed, 633 insertions(+) diff --git a/playbooks/qemu_system_units.yml b/playbooks/qemu_system_units.yml new file mode 100644 index 00000000..e71ddc6d --- /dev/null +++ b/playbooks/qemu_system_units.yml @@ -0,0 +1,7 @@ +--- +- name: Manage imageless NixOS VMs with qemu-system-units + hosts: localhost + vars_files: + - "{{ playbook_dir }}/../extra_vars.yaml" + roles: + - role: qemu_system_units diff --git a/playbooks/roles/gen_nodes/templates/qemu_system_units_nodes.j2 b/playbooks/roles/gen_nodes/templates/qemu_system_units_nodes.j2 new file mode 100644 index 00000000..10e3f8db --- /dev/null +++ b/playbooks/roles/gen_nodes/templates/qemu_system_units_nodes.j2 @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 +# +# qemu-system-units vars file for {{ nodes[0] }}. Generated by kdevops. + +vm_name: {{ nodes[0] }} +service_scope: user + +qemu_binary: {{ qemu_system_units_qemu_binary }} +cpu: {{ qemu_system_units_cpu }} +accel: {{ qemu_system_units_accel }} +ram: {{ qemu_system_units_ram }} +cpus: {{ qemu_system_units_cpus }} +machine_type: {{ qemu_system_units_machine_type }} + +ssh_port: {{ qemu_system_units_ssh_port_base }} +vsock_cid: {{ qemu_system_units_vsock_cid_base }} diff --git a/playbooks/roles/qemu_system_units/tasks/bringup.yml b/playbooks/roles/qemu_system_units/tasks/bringup.yml new file mode 100644 index 00000000..e204d0cb --- /dev/null +++ b/playbooks/roles/qemu_system_units/tasks/bringup.yml @@ -0,0 +1,87 @@ +--- +# SPDX-License-Identifier: copyleft-next-0.3.1 +# +# Start each imageless VM through systemd user mode, wait for +# sshd on the forwarded port, and register the guest in +# ~/.ssh/config so subsequent Ansible plays reach it by +# inventory name rather than by raw port number. + +- name: Reload user systemd so new unit files and drop-ins are visible + ansible.builtin.systemd_service: + daemon_reload: true + scope: user + +# qsu docs/usage.md "Upgrading templates": daemon-reload does not +# restart running sockets, so after re-deploying [email protected] +# or [email protected], or after a reload that re-observes them, +# systemd marks any loaded instance "Unit configuration changed ... +# no socket file descriptors are open" - a zombie listener that +# QEMU's later connect sees as refused. Enumerate the per-share env +# files (one per VM x share, rendered in imageless_build_per_vm.yml) +# to recover the <vm>-<tag> instance names, and restart each matching +# socket right before starting the VMs. No-op on the first run and on +# runs where nothing was re-rendered: restart of an already-healthy +# socket is cheap. +- name: Locate deployed virtiofsd per-share env files + ansible.builtin.find: + paths: "{{ ansible_facts.env.HOME }}/.config/systemd/virtiofsd" + patterns: "*.env" + file_type: file + register: qsu_virtiofsd_envs + +- name: Restart virtiofsd sockets so QEMU connects to fresh listeners + ansible.builtin.systemd_service: + name: "virtiofsd@{{ item.path | basename | splitext | first }}.socket" + state: restarted + scope: user + loop: "{{ qsu_virtiofsd_envs.files }}" + loop_control: + label: "virtiofsd@{{ item.path | basename | splitext | first }}.socket" + +- name: Start qemu-system@<vm> instances + ansible.builtin.systemd_service: + name: "qemu-system@{{ item }}.service" + state: started + scope: user + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + +- name: Determine SSH key path + ansible.builtin.command: python3 {{ playbook_dir }}/../scripts/nixos_ssh_key_name.py --path + register: qsu_ssh_key_path_result + changed_when: false + +- name: Wait for sshd to greet on each VM's forwarded port + ansible.builtin.wait_for: + host: 127.0.0.1 + port: "{{ qemu_system_units_ssh_port_base | int + qsu_vm_idx }}" + search_regex: "^SSH-2\\.0" + delay: 5 + timeout: 300 + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + loop_control: + loop_var: qsu_vm_name + index_var: qsu_vm_idx + label: "{{ qsu_vm_name }}:{{ qemu_system_units_ssh_port_base | int + qsu_vm_idx }}" + +- name: Update SSH config for qsu VMs + ansible.builtin.command: + argv: + - python3 + - "{{ playbook_dir }}/../scripts/update_ssh_config_nixos.py" + - update + - "{{ qsu_vm_name }}" + - 127.0.0.1 + - "{{ qemu_system_units_ssh_port_base | int + qsu_vm_idx }}" + - root + - "{{ ansible_facts.env.HOME }}/.ssh/config" + - "{{ qsu_ssh_key_path_result.stdout | trim }}" + - qsu imageless NixOS + - --vsock-cid + - "{{ qemu_system_units_vsock_cid_base | int + qsu_vm_idx }}" + - --default-transport + - "{{ 'tcp' if qemu_system_units_ssh_default_tcp | default(false) | bool else 'vsock' }}" + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + loop_control: + loop_var: qsu_vm_name + index_var: qsu_vm_idx + label: "{{ qsu_vm_name }}" diff --git a/playbooks/roles/qemu_system_units/tasks/console.yml b/playbooks/roles/qemu_system_units/tasks/console.yml new file mode 100644 index 00000000..27526f05 --- /dev/null +++ b/playbooks/roles/qemu_system_units/tasks/console.yml @@ -0,0 +1,23 @@ +--- +# SPDX-License-Identifier: copyleft-next-0.3.1 +# +# Final status banner listing SSH, VSOCK, and console access for +# each imageless VM plus the common systemctl and machinectl +# commands. + +- name: Display qsu VM access information + ansible.builtin.debug: + msg: | + qemu-system-units imageless NixOS VMs are running. + + {% for vm in groups['all'] | reject('equalto', 'localhost') | list %} + {% set idx = loop.index0 %} + {{ vm }}: + ssh: ssh {{ vm }} (or ssh -p {{ qemu_system_units_ssh_port_base | int + idx }} [email protected]) + vsock: ssh root@vsock/{{ qemu_system_units_vsock_cid_base | int + idx }} + console: socat -,raw,echo=0,escape=0x1d UNIX-CONNECT:$XDG_RUNTIME_DIR/qemu-system/{{ vm }}/console.sock + {% endfor %} + + Status: systemctl --user status qemu-system@<vm> + Logs: journalctl --user-unit=qemu-system@<vm>.service + List: machinectl --user list diff --git a/playbooks/roles/qemu_system_units/tasks/destroy.yml b/playbooks/roles/qemu_system_units/tasks/destroy.yml new file mode 100644 index 00000000..9e2631b7 --- /dev/null +++ b/playbooks/roles/qemu_system_units/tasks/destroy.yml @@ -0,0 +1,68 @@ +--- +# SPDX-License-Identifier: copyleft-next-0.3.1 +# +# Stop any running qsu VMs and remove the shared units plus the +# QMP powerdown helper. Per-VM env files and override drop-ins +# are cleaned in a later task file once those renderings exist. + +- name: Stop qemu-system@<vm> instances + ansible.builtin.systemd_service: + name: "qemu-system@{{ item }}.service" + state: stopped + scope: user + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + failed_when: false + +- name: Remove SSH config entries for qsu VMs + ansible.builtin.command: | + python3 {{ playbook_dir }}/../scripts/update_ssh_config_nixos.py remove \ + {{ item }} \ + '' \ + '' \ + '' \ + {{ ansible_facts.env.HOME }}/.ssh/config \ + '' \ + 'qsu imageless NixOS' + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + failed_when: false + +- name: Clean up per-node generated imageless configurations + ansible.builtin.file: + path: "{{ topdir_path }}/scripts/nixos-qemu/configurations/{{ item }}" + state: absent + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + +- name: Remove per-VM vm.env files + ansible.builtin.file: + path: "{{ ansible_facts.env.HOME }}/.config/systemd/qemu-system/{{ item }}.env" + state: absent + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + +- name: Remove per-VM service drop-in directories + ansible.builtin.file: + path: "{{ ansible_facts.env.HOME }}/.config/systemd/user/qemu-system@{{ item }}.service.d" + state: absent + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + +- name: Remove per-share virtiofsd env files + ansible.builtin.file: + path: "{{ ansible_facts.env.HOME }}/.config/systemd/virtiofsd/{{ item.0 }}-{{ item.1 }}.env" + state: absent + loop: "{{ (groups['all'] | reject('equalto', 'localhost') | list) | product(['store', 'modules']) | list }}" + loop_control: + label: "{{ item.0 }}-{{ item.1 }}" + +- name: Remove qsu shared units and helper + ansible.builtin.file: + path: "{{ item }}" + state: absent + loop: + - "{{ ansible_facts.env.HOME }}/.config/systemd/user/[email protected]" + - "{{ ansible_facts.env.HOME }}/.config/systemd/user/[email protected]" + - "{{ ansible_facts.env.HOME }}/.config/systemd/user/[email protected]" + - "{{ ansible_facts.env.HOME }}/.config/systemd/qemu-system/qmp-powerdown" + +- name: Reload user systemd after removal + ansible.builtin.systemd_service: + daemon_reload: true + scope: user diff --git a/playbooks/roles/qemu_system_units/tasks/generate_configs.yml b/playbooks/roles/qemu_system_units/tasks/generate_configs.yml new file mode 100644 index 00000000..a5d35d4b --- /dev/null +++ b/playbooks/roles/qemu_system_units/tasks/generate_configs.yml @@ -0,0 +1,74 @@ +--- +# SPDX-License-Identifier: copyleft-next-0.3.1 +# +# Per-node imageless configuration generation. Creates one +# directory per guest under $QEMU_SYSTEM_UNITS_CONFIG_DIR, seeds +# an SSH keypair for the guest user, copies the nixos-qemu +# imageless template flake into it, rewrites the nixos-qemu.url +# input to resolve at the checked-in subtree, and renders the +# per-node default.nix. + +- name: Create top-level qsu configuration directory + ansible.builtin.file: + path: "{{ topdir_path }}/scripts/nixos-qemu/configurations" + state: directory + mode: '0755' + +- name: Create per-node configuration directories + ansible.builtin.file: + path: "{{ topdir_path }}/scripts/nixos-qemu/configurations/{{ item }}" + state: directory + mode: '0755' + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + +- name: Ensure SSH key exists for configuration + block: + - name: Determine SSH key path based on directory + ansible.builtin.command: python3 {{ playbook_dir }}/../scripts/nixos_ssh_key_name.py --path + register: ssh_key_path_result + changed_when: false + + - name: Set SSH key path + ansible.builtin.set_fact: + qemu_system_units_ssh_key_path: "{{ ssh_key_path_result.stdout | trim }}" + + - name: Generate SSH key for imageless VMs if not exists + openssh_keypair: + path: "{{ qemu_system_units_ssh_key_path }}" + type: rsa + size: 2048 + comment: "kdevops@qsu" + force: false + + - name: Read SSH public key + ansible.builtin.slurp: + src: "{{ qemu_system_units_ssh_key_path }}.pub" + register: ssh_public_key + + - name: Set SSH key in fact + ansible.builtin.set_fact: + qemu_system_units_ssh_authorized_key: "{{ ssh_public_key['content'] | b64decode | trim }}" + +- name: Copy nixos-qemu imageless template flake into each per-node directory + ansible.builtin.copy: + src: "{{ topdir_path }}/scripts/nixos-qemu/templates/imageless/flake.nix" + dest: "{{ topdir_path }}/scripts/nixos-qemu/configurations/{{ item }}/flake.nix" + remote_src: true + mode: '0644' + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + +- name: Point nixos-qemu input at the local subtree in each per-node flake + ansible.builtin.replace: + path: "{{ topdir_path }}/scripts/nixos-qemu/configurations/{{ item }}/flake.nix" + regexp: '^(\s*)nixos-qemu\.url = "path:/path/to/nixos-qemu";' + replace: '\1nixos-qemu.url = "path:{{ topdir_path }}/scripts/nixos-qemu";' + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + +- name: Render per-node default.nix + ansible.builtin.template: + src: default.nix.j2 + dest: "{{ topdir_path }}/scripts/nixos-qemu/configurations/{{ item }}/default.nix" + mode: '0644' + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + vars: + vm_name: "{{ item }}" diff --git a/playbooks/roles/qemu_system_units/tasks/imageless_build.yml b/playbooks/roles/qemu_system_units/tasks/imageless_build.yml new file mode 100644 index 00000000..9c8f3f74 --- /dev/null +++ b/playbooks/roles/qemu_system_units/tasks/imageless_build.yml @@ -0,0 +1,32 @@ +--- +# SPDX-License-Identifier: copyleft-next-0.3.1 +# +# Build the per-node imageless NixOS closure and render qsu's +# per-VM env files using its own upstream templates, so systemd +# can start each VM against a freshly built closure and the +# controller-installed kernel from bootlinux. + +- name: Resolve controller-built kernel release from the linux build tree + community.general.make: + chdir: "{{ bootlinux_controller_tree_path }}" + target: kernelrelease + params: + O: "{{ bootlinux_controller_builddir }}" + environment: + LOCALVERSION: "{{ target_linux_localversion | default('', true) }}" + register: qsu_kernelrelease_make + changed_when: false + +- name: Derive the controller-built kernel artefact paths + ansible.builtin.set_fact: + qsu_kernel_release: "{{ qsu_kernelrelease_make.stdout_lines[-1] }}" + qsu_kernel_image: "{{ bootlinux_controller_destdir }}/boot/vmlinuz-{{ qsu_kernelrelease_make.stdout_lines[-1] }}" + qsu_modules_dir: "{{ bootlinux_controller_destdir }}/lib/modules" + +- name: Prepare qsu imageless artefacts per VM + ansible.builtin.include_tasks: imageless_build_per_vm.yml + loop: "{{ groups['all'] | reject('equalto', 'localhost') | list }}" + loop_control: + loop_var: qsu_vm_name + index_var: qsu_vm_idx + label: "{{ qsu_vm_name }}" diff --git a/playbooks/roles/qemu_system_units/tasks/imageless_build_per_vm.yml b/playbooks/roles/qemu_system_units/tasks/imageless_build_per_vm.yml new file mode 100644 index 00000000..f6f2e6af --- /dev/null +++ b/playbooks/roles/qemu_system_units/tasks/imageless_build_per_vm.yml @@ -0,0 +1,102 @@ +--- +# SPDX-License-Identifier: copyleft-next-0.3.1 +# +# Per-VM imageless closure build and qsu artefact render. Included +# from imageless_build.yml with qsu_vm_name (VM name) and +# qsu_vm_idx (inventory index) as loop variables. + +- name: Build the imageless NixOS closure for {{ qsu_vm_name }} + ansible.builtin.command: + argv: + - nix + - build + - "path:{{ topdir_path }}/scripts/nixos-qemu/configurations/{{ qsu_vm_name }}#toplevel" + - --out-link + - "{{ topdir_path }}/scripts/nixos-qemu/configurations/{{ qsu_vm_name }}/result" + creates: "{{ topdir_path }}/scripts/nixos-qemu/configurations/{{ qsu_vm_name }}/result" + environment: + PATH: "/nix/var/nix/profiles/default/bin:{{ ansible_facts.env.PATH | default('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin') }}" + +- name: Read bootspec from the built closure for {{ qsu_vm_name }} + ansible.builtin.slurp: + src: "{{ topdir_path }}/scripts/nixos-qemu/configurations/{{ qsu_vm_name }}/result/boot.json" + register: qsu_bootspec_raw + +- name: Extract init and initrd paths from bootspec for {{ qsu_vm_name }} + ansible.builtin.set_fact: + qsu_init: "{{ (qsu_bootspec_raw.content | b64decode | from_json)['org.nixos.bootspec.v1'].init }}" + qsu_initrd: "{{ (qsu_bootspec_raw.content | b64decode | from_json)['org.nixos.bootspec.v1'].initrd }}" + +- name: Ensure qsu per-VM directories exist + ansible.builtin.file: + path: "{{ item }}" + state: directory + mode: '0755' + loop: + - "{{ ansible_facts.env.HOME }}/.config/systemd/qemu-system" + - "{{ ansible_facts.env.HOME }}/.config/systemd/virtiofsd" + - "{{ ansible_facts.env.HOME }}/.config/systemd/user/qemu-system@{{ qsu_vm_name }}.service.d" + +- name: Render vm.env for {{ qsu_vm_name }} + ansible.builtin.template: + src: "{{ topdir_path }}/scripts/qemu-system-units/templates/vm.env.j2" + dest: "{{ ansible_facts.env.HOME }}/.config/systemd/qemu-system/{{ qsu_vm_name }}.env" + mode: '0644' + vars: + vm_name: "{{ qsu_vm_name }}" + qemu_binary: "{{ qemu_system_units_qemu_binary }}" + cpu: "{{ qemu_system_units_cpu }}" + accel: "{{ qemu_system_units_accel }}" + ram: "{{ qemu_system_units_ram }}" + cpus: "{{ qemu_system_units_cpus }}" + machine_type: "{{ qemu_system_units_machine_type }}" + ssh_port: "{{ qemu_system_units_ssh_port_base | int + qsu_vm_idx }}" + vsock_cid: "{{ qemu_system_units_vsock_cid_base | int + qsu_vm_idx }}" + kernel: + image: "{{ qsu_kernel_image }}" + initrd: "{{ qsu_initrd }}" + append: "console=ttyS0,115200 console=hvc0 init={{ qsu_init }}" + shares: + - tag: store + dir: /nix/store + mount: /nix/store + - tag: modules + dir: "{{ qsu_modules_dir }}" + mount: /lib/modules + +- name: Render per-VM service drop-in override.conf for {{ qsu_vm_name }} + ansible.builtin.template: + src: "{{ topdir_path }}/scripts/qemu-system-units/templates/qemu-system-override.conf.j2" + dest: "{{ ansible_facts.env.HOME }}/.config/systemd/user/qemu-system@{{ qsu_vm_name }}.service.d/override.conf" + mode: '0644' + vars: + vm_name: "{{ qsu_vm_name }}" + service_scope: user + qemu_binary: "{{ qemu_system_units_qemu_binary }}" + ram: "{{ qemu_system_units_ram }}" + machine_type: "{{ qemu_system_units_machine_type }}" + vsock_cid: "{{ qemu_system_units_vsock_cid_base | int + qsu_vm_idx }}" + shares: + - tag: store + dir: /nix/store + mount: /nix/store + - tag: modules + dir: "{{ qsu_modules_dir }}" + mount: /lib/modules + +- name: Render virtiofsd env files for the imageless shares of {{ qsu_vm_name }} + ansible.builtin.template: + src: "{{ topdir_path }}/scripts/qemu-system-units/templates/virtiofsd.env.j2" + dest: "{{ ansible_facts.env.HOME }}/.config/systemd/virtiofsd/{{ qsu_vm_name }}-{{ item.tag }}.env" + mode: '0644' + vars: + vm_name: "{{ qsu_vm_name }}" + share_tag: "{{ item.tag }}" + shares: + - tag: "{{ item.tag }}" + dir: "{{ item.dir }}" + loop: + - { tag: store, dir: /nix/store } + - { tag: modules, dir: "{{ qsu_modules_dir }}" } + loop_control: + label: "{{ qsu_vm_name }}-{{ item.tag }}" diff --git a/playbooks/roles/qemu_system_units/tasks/install_deps.yml b/playbooks/roles/qemu_system_units/tasks/install_deps.yml new file mode 100644 index 00000000..548e9a07 --- /dev/null +++ b/playbooks/roles/qemu_system_units/tasks/install_deps.yml @@ -0,0 +1,31 @@ +--- +# SPDX-License-Identifier: copyleft-next-0.3.1 +# +# Host dependencies for the QEMU_SYSTEM_UNITS bringup method: the +# QEMU system emulator, virtiofsd, socat for QMP graceful shutdown, +# and systemd-container for machinectl. Nix is handled by the +# existing nixos-qemu path when the imageless guest image is built. + +- name: Ensure qsu dependencies are installed (Debian) + become: true + ansible.builtin.package: + name: + - qemu-system-x86 + - qemu-utils + - socat + - systemd-container + - virtiofsd + state: present + when: ansible_facts.os_family == "Debian" + +- name: Ensure qsu dependencies are installed (RedHat) + become: true + ansible.builtin.package: + name: + - qemu-kvm + - qemu-img + - socat + - systemd-container + - virtiofsd + state: present + when: ansible_facts.os_family == "RedHat" diff --git a/playbooks/roles/qemu_system_units/tasks/main.yml b/playbooks/roles/qemu_system_units/tasks/main.yml new file mode 100644 index 00000000..fad4f7e5 --- /dev/null +++ b/playbooks/roles/qemu_system_units/tasks/main.yml @@ -0,0 +1,49 @@ +--- +# qemu_system_units role entry point. Orchestrates install-deps, +# generate-configs (shared units plus per-node nixos-qemu flakes), +# build-vms (imageless closure and per-VM envs), bringup +# (systemctl start and SSH access), console (access banner), and +# destroy via tag-gated include_tasks. + +- name: Include install_deps tasks + ansible.builtin.import_tasks: + file: "{{ role_path }}/tasks/install_deps.yml" + tags: + - install-deps + +- name: Include render_units tasks + ansible.builtin.import_tasks: + file: "{{ role_path }}/tasks/render_units.yml" + tags: + - generate-configs + +- name: Include generate_configs tasks + ansible.builtin.import_tasks: + file: "{{ role_path }}/tasks/generate_configs.yml" + tags: + - generate-configs + +- name: Include imageless_build tasks + ansible.builtin.import_tasks: + file: "{{ role_path }}/tasks/imageless_build.yml" + tags: + - build-vms + +- name: Include bringup tasks + ansible.builtin.import_tasks: + file: "{{ role_path }}/tasks/bringup.yml" + tags: + - bringup + +- name: Include console tasks + ansible.builtin.import_tasks: + file: "{{ role_path }}/tasks/console.yml" + tags: + - console + +- name: Include destroy tasks + ansible.builtin.import_tasks: + file: "{{ role_path }}/tasks/destroy.yml" + tags: + - destroy + - never diff --git a/playbooks/roles/qemu_system_units/tasks/render_units.yml b/playbooks/roles/qemu_system_units/tasks/render_units.yml new file mode 100644 index 00000000..a55f8afe --- /dev/null +++ b/playbooks/roles/qemu_system_units/tasks/render_units.yml @@ -0,0 +1,55 @@ +--- +# SPDX-License-Identifier: copyleft-next-0.3.1 +# +# Deploy the three qemu-system-units template units (qemu-system@, +# [email protected], [email protected]) and the QMP powerdown +# script that [email protected]'s ExecStop= pipes into the VM. +# These files are shared across all VMs on the host; per-VM state +# lives in %E/systemd/qemu-system/<vm>.env and is rendered in a +# later task file. + +- name: Ensure user systemd and qsu config directories exist + ansible.builtin.file: + path: "{{ item }}" + state: directory + mode: '0755' + loop: + - "{{ ansible_facts.env.HOME }}/.config/systemd/user" + - "{{ ansible_facts.env.HOME }}/.config/systemd/qemu-system" + +- name: Render [email protected] + ansible.builtin.template: + src: "{{ topdir_path }}/scripts/qemu-system-units/templates/[email protected]" + dest: "{{ ansible_facts.env.HOME }}/.config/systemd/user/[email protected]" + mode: '0644' + vars: + qemu_binary: "{{ qemu_system_units_qemu_binary }}" + service_scope: user + +- name: Render [email protected] + ansible.builtin.template: + src: "{{ topdir_path }}/scripts/qemu-system-units/templates/[email protected]" + dest: "{{ ansible_facts.env.HOME }}/.config/systemd/user/[email protected]" + mode: '0644' + vars: + service_scope: user + +- name: Render [email protected] + ansible.builtin.template: + src: "{{ topdir_path }}/scripts/qemu-system-units/templates/[email protected]" + dest: "{{ ansible_facts.env.HOME }}/.config/systemd/user/[email protected]" + mode: '0644' + vars: + service_scope: user + +- name: Install QMP powerdown helper for [email protected] ExecStop= + ansible.builtin.copy: + src: "{{ topdir_path }}/scripts/qemu-system-units/files/qmp-powerdown" + dest: "{{ ansible_facts.env.HOME }}/.config/systemd/qemu-system/qmp-powerdown" + remote_src: true + mode: '0644' + +- name: Reload user systemd so the rendered units are visible + ansible.builtin.systemd_service: + daemon_reload: true + scope: user diff --git a/playbooks/roles/qemu_system_units/templates/default.nix.j2 b/playbooks/roles/qemu_system_units/templates/default.nix.j2 new file mode 100644 index 00000000..7e979372 --- /dev/null +++ b/playbooks/roles/qemu_system_units/templates/default.nix.j2 @@ -0,0 +1,47 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 +# +# Per-node overrides for {{ vm_name }}. Generated by kdevops. +# +# The imageless template flake (copied from +# scripts/nixos-qemu/templates/imageless/flake.nix) passes +# `nixos-qemu` and the flake inputs through specialArgs, so +# workflow modules are imported directly here. +{ config, lib, pkgs, nixos-qemu, inputs, ... }: { + imports = [ + nixos-qemu.nixosModules.devel + nixos-qemu.nixosModules.build-tools +{% if kdevops_workflow_enable_blktests is defined and kdevops_workflow_enable_blktests %} + nixos-qemu.nixosModules.workflows.blktests +{% endif %} +{% if kdevops_workflow_enable_fstests is defined and kdevops_workflow_enable_fstests %} + nixos-qemu.nixosModules.workflows.fstests +{% endif %} +{% if kdevops_workflow_enable_gitr is defined and kdevops_workflow_enable_gitr %} + nixos-qemu.nixosModules.workflows.gitr +{% endif %} +{% if kdevops_workflow_enable_ltp is defined and kdevops_workflow_enable_ltp %} + nixos-qemu.nixosModules.workflows.ltp +{% endif %} +{% if kdevops_workflow_enable_mmtests is defined and kdevops_workflow_enable_mmtests %} + nixos-qemu.nixosModules.workflows.mmtests +{% endif %} +{% if kdevops_workflow_enable_pynfs is defined and kdevops_workflow_enable_pynfs %} + nixos-qemu.nixosModules.workflows.pynfs +{% endif %} +{% if kdevops_workflow_enable_selftests is defined and kdevops_workflow_enable_selftests %} + nixos-qemu.nixosModules.workflows.selftests +{% endif %} +{% if kdevops_workflow_enable_sysbench is defined and kdevops_workflow_enable_sysbench %} + nixos-qemu.nixosModules.workflows.sysbench +{% endif %} + ]; + + networking.hostName = "{{ vm_name }}"; + + nixos-qemu.user.name = "kdevops"; + +{% if qemu_system_units_ssh_authorized_key is defined %} + users.users.root.openssh.authorizedKeys.keys = [ "{{ qemu_system_units_ssh_authorized_key }}" ]; + users.users.kdevops.openssh.authorizedKeys.keys = [ "{{ qemu_system_units_ssh_authorized_key }}" ]; +{% endif %} +} diff --git a/scripts/provision.Makefile b/scripts/provision.Makefile index 378cbde4..01d17e9c 100644 --- a/scripts/provision.Makefile +++ b/scripts/provision.Makefile @@ -64,6 +64,10 @@ ifeq (y,$(CONFIG_NIXOS)) include scripts/nixos.Makefile endif +ifeq (y,$(CONFIG_QEMU_SYSTEM_UNITS)) +include scripts/qemu_system_units.Makefile +endif + KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_SSH) KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_DEVCONFIG) diff --git a/scripts/qemu_system_units.Makefile b/scripts/qemu_system_units.Makefile new file mode 100644 index 00000000..6443768b --- /dev/null +++ b/scripts/qemu_system_units.Makefile @@ -0,0 +1,38 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 + +HOSTS_PREFIX:=$(subst ",,$(CONFIG_KDEVOPS_HOSTS_PREFIX)) + +KDEVOPS_NODES_TEMPLATE := $(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/qemu_system_units_nodes.j2 +KDEVOPS_NODES := scripts/qemu-system-units/vars/$(HOSTS_PREFIX).yaml + +export KDEVOPS_PROVISIONED_SSH := $(KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD) + +KDEVOPS_PROVISION_METHOD := bringup_qemu_system_units +KDEVOPS_PROVISION_DESTROY_METHOD := destroy_qemu_system_units + +install_qemu_system_units_deps: + $(Q)ansible-playbook \ + playbooks/qemu_system_units.yml \ + --extra-vars=@./extra_vars.yaml \ + --tags install-deps + +generate_qemu_system_units_configs: + $(Q)ansible-playbook \ + playbooks/qemu_system_units.yml \ + --extra-vars=@./extra_vars.yaml \ + --tags generate-configs + +bringup_qemu_system_units: install_qemu_system_units_deps generate_qemu_system_units_configs + $(Q)ansible-playbook \ + playbooks/qemu_system_units.yml \ + --extra-vars=@./extra_vars.yaml \ + --tags build-vms,bringup,console + $(Q)touch $(KDEVOPS_PROVISIONED_SSH) +PHONY += bringup_qemu_system_units + +destroy_qemu_system_units: + $(Q)ansible-playbook \ + playbooks/qemu_system_units.yml \ + --extra-vars=@./extra_vars.yaml \ + --tags destroy +PHONY += destroy_qemu_system_units -- 2.53.0