[RFC PATCH v1 0/1] vfio/pci: Revoke BARs and DMABUFs during sysfs-triggered PCI reset

Introduce PCI .reset_prepare and .reset_done handlers to safely revoke
active userspace mappings and exported DMABUFs during sysfs-triggered
device resets.

We are seeing a situation where system health and monitoring daemons 
(at times erroneously) issue device resets via sysfs for devices bound
to vfio-pci:

  echo 1 > /sys/bus/pci/devices/0000:01:00.0/reset

However, because vfio-pci does not implement the .reset_prepare and
.reset_done error handlers, this hardware reset occurs completely unnoticed
by the VFIO driver.

Consequently, active traditional userspace BAR mappings and exported DMABUFs
are never zapped or revoked. Importers of the DMABUFs (e.g., RDMA drivers)
continue to issue DMAs (such as PCIe Memory Writes) toward the Endpoint. 
These transactions are silently dropped by the root port or trigger CTOs 
while higher-level actions (e.g., RDMA reg_mr) continue to succeed.

We'd like to fix this by implementing the PCI reset ops for vfio-pci
that revoke the DMABUFs and zap the BARs while holding the memory lock
allowing concurrent user accesses to sleep and fault back in once the reset
completes.

Note: I've tried to handle the locking as a first attempt here, there
might've been some cases that were missed. Also, for the RFC, the
drivers that implement their own pci_error_handlers (like nvgrace) are 
not altered for now.

Quick Note about Matt's DMABUF mmap Series
==========================================
While this patch is aimed for the current upstream code, I believe with
Matt's refactor [1] these ops might change slightly. If we have consensus
on this patch, I'd send another patch based to Matt based on their series
for them to include it in their next version.

[1] https://lore.kernel.org/all/[email protected]/

Thanks,
Praan

Pranjal Shrivastava (1):
  vfio/pci: Revoke BARs and DMABUFs during sysfs-triggered PCI reset

 drivers/vfio/pci/vfio_pci_core.c | 88 ++++++++++++++++++++------------
 1 file changed, 56 insertions(+), 32 deletions(-)

-- 
2.55.0.679.g6767b8d81c-goog