Re: [RFC PATCH 1/5] PCI: Refuse function reset of an SR-IOV PF with enabled VFs

On Tue, Aug 18, 2026 at 02:39:06PM -0600, Alex Williamson wrote:
>On Tue, 18 Aug 2026 11:03:04 -0300
>Jason Gunthorpe <[email protected]> wrote:
>
>> On Tue, Aug 18, 2026 at 07:07:20AM +0000, Tian, Kevin wrote:
>> > 3) resetting a PF due to ad-hoc FW error
>> >
>> > e.g. pdsc_check_pci_health() calls pci_reset_function() when FW
>> > reports broken PCI. Sounds a similar condition to AER.
>> >
>> > but as it calls pci_reset_function() this series causes a regression
>> > in case of active VFs. Is it desired?
>>
>> mlx5 does something similar. RAS flows require detecting the PF has
>> blown up and resetting it to bring it back.
>>
>> This has to work while VFs exist and bound to kernel drivers or vfio.
>>
>> In the VFIO KVM case I would expect the VM to experiance a RAS error
>> on its vPCI device and trigger a VF flr
>
>This would need to be a device level RAS flow as we're sorely lacking
>on forwarding reset notification to VF drivers or providing any
>mechanism for recovery from the vfio error eventfd.
>
>> But we have to sort all this out through the hypervisor to not crash
>> the machine or otherwise :)
>
>In general guarding the high level pci_reset_function() family on
>(pci_num_vfs() == 0) seems to have more pros than cons and the
>restriction matches the scoped intent of the interface.
>
>The low level __pci_reset_function_locked() interface remains
>untouched, and as the prefix implies, relies on the caller to do
>necessary verification and preparation.  A "reset the PF regardless of
>VFs" seems like an underscore prefix sort of operation.  We'll need to
>export the pci_dev_save_and_disable() and pci_dev_restore() functions
>to provide handlers the full suite of tools they need to replace the
>existing function and port those we find (maybe also questioning the
>author's intent on VF reset and recovery).
>
>Of course making the guard parameterized with wrappers to expand this
>family of calls is also an option,
>pci_reset_function_locked_with_vfs()...
>
>The mlx5 driver doesn't actually seem to be affected by these changes,
>doing its own link toggle or a PCI bus reset, not a
>pci_reset_function() call.
>
>On the other improvements, triggering a pci_reset_function_locked() on
>unbind after SR-IOV is disabled, when needs_reset is indicated, looks
>pretty trivial.
>
>The blocking domain operation looks like it might be simplest to
>implement in the IOMMU core.  We can set a flag for a default blocking
>domain on the IOMMU group when we take_dma_ownership of the group.  Then
>release_dma_ownership picks the blocking rather than default domain.
>
>This is then unwound in use_default_domain, called via dma_configure,
>attaching the device to the default domain in probe of the next driver.
>Therefore until probe by another driver, a device used by vfio would
>remain in a blocking domain even while unused and unbound.

The devices are expected to be attached to the default_domain even when
these are unbound and the use_default_domain assumes that, and it only
checks the ownership and doesn't switch the domain to default_domain. I
guess we should add a WARN in use_default_domain() if that is not true.
I will probably send out a patch for that separately.

I think we can move the device back to default_domain after reset after
unbind, maybe it can be done in pci_dma_cleanup() based on
driver_managed_dma?
>
>Does that seem reasonable (Cc +iommu)?  Thanks,
>
>Alex
>

Thanks,
Sami