[PATCH v3 0/6] Fix device page migration in low memory fallback

LLMs made my breakfast, lunch, and dinner. Not really. They served as an
assistive tool while I performed the debugging, testing, and analysis
needed to isolate the root cause in core MM while fixing a known DRM SVM
issue involving THP allocation failures in the CPU fault-to-device page
migration path.

When a CPU faults on a device private PMD and the driver cannot allocate
a compound destination folio, the source THP has to be split. That path
is broken: the CPU fault reference makes the split always fail, and it
demotes the PMD only in the faulting VMA, leaving any other VMA mapping
the folio pointing a huge PMD at an order-0 page.
The latter is memory corruption, previously masked by the former.

The DRM side had its own problems in the same fallback: there was no
order-0 fallback at all despite a TODO saying one was needed, the error
path computed folio_order() after put_page(), and once the destination
is demoted to order-0 the source page array has to be populated per
page rather than per folio head, or the copy stops after one page.

Validation was performed using xe_exec_system_allocator. The issue was
initially discovered on systems configured with an artificially
constrained memory footprint (mem=8G), where failures occurred
intermittently. Error injection was then introduced to reliably
reproduce the failure condition, enabling thorough validation of the
fix. Results were confirmed through pass/fail A/B testing.

Matt

v2::
 - Add assert in 'Fix folio allocation fallback and use-after-put' for
   THP placement invariant which Sashiko hallucinated as a bug [1]
 - Add 'Clear MIGRATE_PFN_MIGRATE on all sub-folios of a split THP'
   (Sashiko)
 - Fix checkpatch issues (CI)
 - Swap cache issue flagged by Sashiko [1] not fixed as this code doesn't
   appear reachable (i.e., dead code). Can address in a follow up if
   needed
v3:
 - Noticed Arvind's patch [2] fixes some of core issues attempting to be
   fixed to in v1, v2. Verified Arvind's patch works as expected and
   including it in this rev for Intel's CI which is based on drm-tip and
   does not have Arvind's patch. Dropping unnecessary fixes from v1, v2
   as a result.
 - Include 'mm/migrate_device: Do not write past the end of the src_pfns
   array' unrelated fix flagged by Sashiko in v2
 - Include 'dma-unmap pages before handling migration errors'
   unreleated fix flagged by Sashiko in v2

[1] https://sashiko.dev/#/patchset/20260805113338.3742178-1-matthew.brost%40intel.com
[2] https://patchew.org/linux/[email protected]/

Arvind Yadav (1):
  mm/migrate_device: Clear stale mapping after freeing swapcache

Matthew Brost (5):
  mm/migrate_device: Do not write past the end of the src_pfns array
  mm/migrate_device: Fix THP splitting of a CPU faulted device private
    folio
  drm/pagemap: dma-unmap pages before handling migration errors
  drm/pagemap: Fix folio allocation fallback and use-after-put
  drm/pagemap: Add fault injection for higher-order RAM folio allocation

 drivers/gpu/drm/drm_pagemap.c | 176 +++++++++++++++++++++++++++-------
 mm/migrate_device.c           | 115 +++++++++++++++++++---
 2 files changed, 248 insertions(+), 43 deletions(-)

-- 
2.34.1