[PATCH v2 0/7] nfs: Modernize Direct I/O path
Pranjal Shrivastava <[email protected]>
| Newsgroups | gmane.linux.nfs,gmane.linux.kernel |
|---|---|
| Message-ID | <[email protected]> |
Modernize the NFS Direct I/O path as a preparatory step to enable PCI
Peer-to-Peer DMA (P2PDMA) support. Following feedback on the initial
RFC [1], the modernization and architectural changes are split into
this standalone series.
Currently, NFS O_DIRECT relies on the legacy iov_iter_get_pages_alloc2()
API which does not support the pinning requirements for P2P memory.
The implementation moves NFS to the modern iov_iter_extract_pages() API
and migrates NFS direct I/O away from pages to use folios.
Design
======
1. Pin-Awareness
Standard NFS requests use get_page() and put_page() for memory
management. However, memory extracted via iov_iter_extract_pages()
requires explicit pinning.
Introduce a PG_PINNED flag and a wb_nr_pinned count to struct nfs_page.
This allows the request lifecycle to track ownership of physical pins
and ensure that unpinning is performed only when the I/O is complete.
2. API Migration
Migrate the Direct I/O path to the modern iov_iter_extract_pages()
API. This aligns NFS with the modern extraction model and serves as
the foundation for passing ITER_ALLOW_P2PDMA in a follow-up series.
3. Extraction Helper and Folio Support
Introduce a new extraction helper in direct.c to group contiguous
pages from the same folio into a single struct nfs_page. This
effectively migrates the Direct I/O path from being page-based to being
folio-based.
Note: zone_device_pages_have_same_pgmap() checks are intentionally
omitted in the extraction helper since P2PDMA enablement will be
introduced in a follow-up series.
Bisectability
=============
The series attempts to remain bisectable.
[Patches 1-2] Introduce pin-aware infrastructure and accounting.
[Patch 3] Adds a centralized request release helper.
[Patch 4] Migrates the Direct I/O path to iov_iter_extract_pages().
[Patches 5-6] Implement the extraction helper and folio-based grouping.
[Patch 7] Removes orphaned page-based helpers.
Testing
=======
This series has been tested with xfstests [2] on RDMA & TCP transports:
./check generic/091 generic/130 generic/139 generic/143 generic/154 \
generic/155 generic/183 generic/188 generic/190 generic/196 \
generic/198 generic/203 generic/214 generic/240 generic/263 \
generic/287 generic/290 generic/292 generic/330 generic/444 \
generic/450 generic/451 generic/586 generic/647 generic/708 \
generic/729 generic/760
The following summary was tabulated via a custom script [3] (on github).
python3 display.py results/*/check.log
+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
| testcase | rdma-sys-3 | rdma-sys-4.0 | rdma-sys-4.1 | rdma-sys-4.2 | tcp-sys-3 | tcp-sys-4.0 | tcp-sys-4.1 | tcp-sys-4.2 |
+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
| generic/091 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/130 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/139 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/143 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/154 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/155 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/183 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/188 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/190 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/196 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/198 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/203 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/214 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/240 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/263 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/287 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/290 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/292 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/330 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/444 | skipped | skipped | skipped | skipped | skipped | skipped | skipped | skipped |
| generic/450 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/451 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/586 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/647 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/708 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/729 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/760 | pass | pass | pass | pass | pass | pass | pass | pass |
+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
Thanks,
Praan
[1] https://lore.kernel.org/all/20260401194501.2269200-1-praan-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org/
[2] https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
[v2]
- Fix data corruption in nfs_direct_extract_pages() by correctly
calculating intra-page offsets using offset_in_page().
- Fix requested_bytes accounting in direct read/write paths to only
increment after successful RPC scheduling.
- Add missing kernel-doc descriptions for the @pinned parameter in
nfs_page_create_from_page() and nfs_page_create_from_folio().
- Rebase on fs-next/
[v1] https://lore.kernel.org/all/20260603053033.3300318-1-praan-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org/
Pranjal Shrivastava (7):
nfs: make nfs_page pin-aware
nfs: Track number of pinned pages in nfs_page
nfs: Introduce nfs_release_request_list helper
nfs: migrate direct I/O to iov_iter_extract_pages
nfs: introduce nfs_direct_extract_pages helper
nfs: Optimize direct I/O to use folios for requests
nfs: Cleanup the nfs_page_create_from_page helper
fs/nfs/direct.c | 165 +++++++++++++++++++++++----------------
fs/nfs/pagelist.c | 87 +++++++++++----------
fs/nfs/read.c | 2 +-
fs/nfs/write.c | 2 +-
include/linux/nfs_page.h | 12 ++-
5 files changed, 150 insertions(+), 118 deletions(-)
base-commit: 389bb4a76905771adfa86d21ee0b865247148e9d
--
2.54.0.1136.gdb2ca164c4-goog