[XEN][RESEND][PATCH v4 0/3] xen/arm{32,64}: perform IPA-based TLBI when IPA is known

From: Haseeb Ashraf <[email protected]>

This patch series addresses a major performance issue when running Xen as
an L1 hypervisor on KVM, where emulation of VMALLS12E1IS is significantly
more expensive than narrower invalidation sequences and becomes especially
problematic when invoked frequently.

Two common cases are affected:

  (a) vCPU migration on a pCPU, where Xen currently flushes combined
      stage-1+stage-2 guest translations even though the stage-2 regime is
      shared at the domain p2m level.

  (b) teardown of DomU mappings exported into Dom0, where
      XENMEM_remove_from_physmap can trigger repeated full guest TLB
      invalidation even when the affected IPA range is known.

This series reduces reliance on full guest TLB invalidation and prefers
IPA-based invalidation where possible.

Patch 1 introduces IPA-based invalidation in the p2m code when the
affected IPA range is known, and uses stage-1-only invalidation for the
vCPU migration case.

Patch 2 optimizes combined stage-1/stage-2 invalidation in the presence
of FEAT_nTLBPA by avoiding redundant stage-1 invalidation.

Patch 3 adds Arm32 capability detection for TLBI-by-IPA and uses it in
the IPA-range helper when FEAT_nTLBPA is also present. When that
combination is unavailable, Arm32 retains the conservative full-flush
behaviour.

The Arm64 IPA-range helper has also been reworked in v4 to make the TLBI
sequence more explicit:
  - stage-2 invalidation by IPA is performed in a loop,
  - stage-1 invalidation is issued separately only when required,
  - completion of the broadcast TLBI sequence (including the repeat-TLBI
    workaround) is handled explicitly rather than being piggybacked onto a
    stage-1 helper.

This makes the intended semantics clearer and avoids relying on hidden
barrier/workaround side effects.

Measured on Graviton4 with Xen-on-KVM, a full guest TLBI was observed to
be roughly comparable in cost to tens of thousands of single-page
IPA-based invalidations, which motivates using a bounded IPA-based loop
for smaller ranges and falling back to full invalidation above a
threshold.

For Arm64, the 256MB threshold is based on those measurements. For Arm32,
the same cutoff is reused as a conservative practical bound rather than
as an Arm32-specific tuned value.

Haseeb Ashraf (3):
  xen/arm/p2m: perform IPA-based TLBI when IPA is known
  xen/arm: optimize stage-1,2 combined TLBI in presence of FEAT_nTLBPA
  xen/arm32: use IPA-based TLBI when supported with FEAT_nTLBPA

Changes in v4:
- Rebased on master branch
- Reworked the Arm64 IPA-based TLBI sequence to make completion of the
  broadcast TLBI explicit instead of relying on a stage-1 helper for the
  final barriers/workaround.
- Reused __tlb_repeat_sync() for repeat-TLBI workaround handling in the
  Arm64 completion path.
- Fixed the Arm32 IPA-based TLBI helper to fall back to a full guest TLB
  flush when the required capability combination is unavailable, instead
  of becoming a no-op.
- Added a small robustness check in p2m_force_tlb_flush_range_sync().
- Refined comments and commit messages to make the architectural intent
  clearer.

Changes in v3:
- Mainly reworked handling of the repeat-TLBI workaround with IPA-based
  TLBI so that the extra TLBI and DSB are applied only at completion of
  the overall sequence.
- Updated code comments as per feedback. Further details are available in
  each commit's changelog.
- Minor code updates as per feedback. Further details are available in
  each commit's changelog.

Changes in v2:
- Split the original change into three commits. The first commit
  implements the baseline implementation without adding new CPU
  capabilities. CPU capability handling is introduced in follow-up
  commits to make each optimization clearer.
- Moved Arm32 and Arm64 specific TLBI implementations to the respective
  architecture-specific flushtlb.h.
- Added ARM ARM references in code comments.
- Added a threshold to select between IPA-based invalidation and
  fallback to full guest invalidation above that threshold.
- Introduced ARM_HAS_NTLBPA CPU capability for both Arm32 and Arm64.
- Introduced ARM32_HAS_TLB_IPA CPU capability for Arm32 IPA-based TLBI.

-- 
2.43.0