Re: [RFC PATCH v1 0/1] vfio/pci: Revoke BARs and DMABUFs during sysfs-triggered PCI reset

On Fri,  7 Aug 2026 20:14:04 +0000
Pranjal Shrivastava <[email protected]> wrote:

> Introduce PCI .reset_prepare and .reset_done handlers to safely revoke
> active userspace mappings and exported DMABUFs during sysfs-triggered
> device resets.
> 
> We are seeing a situation where system health and monitoring daemons 
> (at times erroneously) issue device resets via sysfs for devices bound
> to vfio-pci:
> 
>   echo 1 > /sys/bus/pci/devices/0000:01:00.0/reset
> 
> However, because vfio-pci does not implement the .reset_prepare and
> .reset_done error handlers, this hardware reset occurs completely unnoticed
> by the VFIO driver.
> 
> Consequently, active traditional userspace BAR mappings and exported DMABUFs
> are never zapped or revoked. Importers of the DMABUFs (e.g., RDMA drivers)
> continue to issue DMAs (such as PCIe Memory Writes) toward the Endpoint. 
> These transactions are silently dropped by the root port or trigger CTOs 
> while higher-level actions (e.g., RDMA reg_mr) continue to succeed.
> 
> We'd like to fix this by implementing the PCI reset ops for vfio-pci
> that revoke the DMABUFs and zap the BARs while holding the memory lock
> allowing concurrent user accesses to sleep and fault back in once the reset
> completes.

That sounds like a nice, serene solution, but that's not actually what
happens.  Due to the write vs read memory_lock semaphore, CPU faults
are stalled.  On the other hand, DMA mappings via IOMMUFD/dmabuf are
lost.  They require the userspace driver to be involved to perform the
unmap/remap.

Potentially this is all better than letting the device generate a
machine check as it's still trying to run across the reset, but let's
not pretend this is just a hiccup for the device that will continue
running after the rogue reset.  Thanks,

Alex
 
> Note: I've tried to handle the locking as a first attempt here, there
> might've been some cases that were missed. Also, for the RFC, the
> drivers that implement their own pci_error_handlers (like nvgrace) are 
> not altered for now.
> 
> Quick Note about Matt's DMABUF mmap Series
> ==========================================
> While this patch is aimed for the current upstream code, I believe with
> Matt's refactor [1] these ops might change slightly. If we have consensus
> on this patch, I'd send another patch based to Matt based on their series
> for them to include it in their next version.
> 
> [1] https://lore.kernel.org/all/[email protected]/
> 
> Thanks,
> Praan
> 
> Pranjal Shrivastava (1):
>   vfio/pci: Revoke BARs and DMABUFs during sysfs-triggered PCI reset
> 
>  drivers/vfio/pci/vfio_pci_core.c | 88 ++++++++++++++++++++------------
>  1 file changed, 56 insertions(+), 32 deletions(-)
>