Re: [PATCH v10 8/8] x86/string: extend memcpy_flushcache() fixed-size fastpaths

On Mon, Aug 10, 2026 at 08:20:57PM +0800, Li Zhe wrote:
> The x86 memcpy_nontemporal() helper maps to memcpy_flushcache(), and the
> ZONE_DEVICE template-copy path uses it to copy one struct page at a
> time.
> 
> The relevant copy size is sizeof(struct page). On x86_64, the base struct
> page layout is 64 bytes. Adding either the KMSAN metadata pointers or an
> out-of-flags last_cpupid field can make it 80 bytes after alignment, and
> enabling both can make it 96 bytes.
> 
> memcpy_flushcache() currently only has inline fixed-size cases for 4, 8,
> and 16 bytes. As a result, these constant-sized struct page copies fall
> through to __memcpy_flushcache() even though the compiler knows the copy
> size at the call site.
> 
> Add fixed-size MOVNTI cases up to 96 bytes so the ZONE_DEVICE
> template-copy path can keep these struct page copies in the inline
> memcpy_flushcache() path.
> 
> This matters for ZONE_DEVICE memmap initialization because the copy
> happens once per initialized struct page. For a 100 GB fsdax namespace
> with map=dev, this is about 25 million struct page copies during nd_pmem
> binding or rebinding.
> 
> Tested in a VM with a 100 GB fsdax namespace device configured with
> map=dev and a 100 GB devdax namespace (align=2097152) on Intel Ice Lake
> server.
> 
> Test procedure:
> Rebind the nd_pmem and dax_pmem drivers 30 times and collect the memmap
> initialization time from the pr_debug() output of
> memmap_init_zone_device().
> 
> With memcpy_nontemporal() used by the ZONE_DEVICE template-copy path:
>   Average of rebinds for nd_pmem driver: 150.83 ms
>   Average of rebinds for dax_pmem driver: 153.55 ms
> 
> With this x86 fixed-size fastpath patch applied:
>   Average of rebinds for nd_pmem driver: 96.79 ms
>   Average of rebinds for dax_pmem driver: 119.04 ms
> 
> This further reduces the average memmap initialization time measured
> during rebind by about 35.8% for nd_pmem and 22.5% for dax_pmem.
> 
> Suggested-by: Borislav Petkov <[email protected]>
> Signed-off-by: Li Zhe <[email protected]>
> ---
>  arch/x86/include/asm/string_64.h | 71 +++++++++++++++++++++++++-------
>  1 file changed, 56 insertions(+), 15 deletions(-)

Acked-by: Borislav Petkov (AMD) <[email protected]>

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette