[PATCH 0/8] Force detach PCI devices for ACPI-based and PCIe native hot-unplug
Dongli Zhang <[email protected]>
| Newsgroups | org.nongnu.qemu-devel,org.xenproject.lists.xen-devel |
|---|---|
| Message-ID | <[email protected]> |
Hot-unplugging a PCI device can require cooperation from the guest. For ACPI PCI hotplug, QEMU notifies the guest through ACPI and the guest eventually writes the ACPI PCI eject register. For PCIe native hotplug, QEMU notifies the guest through the PCIe hotplug mechanism and waits for the slot unplug flow to complete. Only after that completion does QEMU unrealize the device and emit DEVICE_DELETED. This can leave a device stuck in the unplug pending state when the guest does not cooperate. Examples include: 1. The guest has panicked, or the relevant ACPI/PCI hotplug driver is unavailable. 2. The guest is stalled and cannot handle the hot-unplug event. For example, stalling the Linux [irq/9-acpi] kernel thread can reproduce this for ACPI-based hot-unplug. 3. The device was attached to a slot that the guest cannot use. For example, a pcie-root-port only supports slot 0. If a device is added to a non-zero slot below a pcie-root-port, the guest may never discover the device and therefore may never complete the unplug request. The non-zero slot case has also been discussed in: hw/pci: warn when PCIe device is plugged into non-zero slot of downstream port https://gitlab.com/qemu-project/qemu/-/commit/ca92eb5defcf9d1c2106341744a73a03cf26e824 hw/pci: add comment to explain checking for available function 0 in pci hotplug https://gitlab.com/qemu-project/qemu/-/commit/67d045a0ef5b9c5f871c3a1d87325a8a42d2b9d5 pci: don't skip function 0 occupancy verification for devfn auto assign https://gitlab.com/qemu-project/qemu/-/commit/e228d62b4af29bca698ec57efdceb46f392f5444 For example, if root-port.1 is a pcie-root-port, the following command adds a vhost-scsi-pci device to an invalid slot: (qemu) device_add vhost-scsi-pci,id=scsi01,wwpn=naa.5001405324af0985,bus=root-port.1,addr=01.0 warning: PCI: slot 1 is not valid for vhost-scsi-pci, parent device only allows plugging into slot 0. In this situation, the device may be impossible to remove through normal guest-cooperative hot-unplug. Production environments also need a host side recovery option when the guest kernel is the reason that unplug does not complete. This series adds a force option to QMP device_del and HMP device_del. When requested, QEMU asks the selected hotplug controller to complete the unplug through a new force_unplug callback. This series implements forced unplug for ACPI PCI hotplug and PCIe native hotplug, which cover common pc/q35/virt cases. SHPC is not implemented by this series. Dongli Zhang (8): qdev: Add force argument to qdev_unplug qdev: hotplug: Add force_unplug handler callback qdev: Support forced device_del in QMP and HMP hw/acpi/pcihp: acpi/pcihp: Add forced slot unplug helper hw/acpi/piix4: Support forced PCI unplug hw/acpi/ich9: Support forced PCI unplug hw/acpi/ged: Support forced PCI unplug hw/pci/pcie: Support forced PCIe native unplug hmp-commands.hx | 11 ++++++----- hw/acpi/acpi-pci-hotplug-stub.c | 6 ++++++ hw/acpi/generic_event_device.c | 15 +++++++++++++++ hw/acpi/ich9.c | 15 +++++++++++++++ hw/acpi/pcihp.c | 17 +++++++++++++++++ hw/acpi/piix4.c | 15 +++++++++++++++ hw/core/hotplug.c | 11 +++++++++++ hw/isa/lpc_ich9.c | 1 + hw/pci/pcie.c | 20 ++++++++++++++++++++ hw/pci/pcie_port.c | 1 + hw/s390x/s390-pci-bus.c | 4 ++-- hw/vfio/ap.c | 2 +- hw/vfio/ccw.c | 2 +- hw/vfio/pci.c | 2 +- hw/xen/xen-legacy-backend.c | 2 +- hw/xen/xen_pvdev.c | 2 +- include/hw/acpi/ich9.h | 2 ++ include/hw/acpi/pcihp.h | 3 +++ include/hw/core/hotplug.h | 12 ++++++++++++ include/hw/core/qdev.h | 2 +- include/hw/pci/pcie.h | 2 ++ qapi/qdev.json | 11 +++++++++-- system/qdev-monitor.c | 23 +++++++++++++++++------ 23 files changed, 160 insertions(+), 21 deletions(-) base-commit: eea8fe61b8be8f3016e522e6af24924a0266ca95 Thank you very much! Dongli Zhang