Re: [PATCH v2 00/25] mm: introduce numa_memblks

On 23 Jul 2024, at 2:41, Mike Rapoport wrote:

> From: "Mike Rapoport (Microsoft)" <[email protected]>
>
> Hi,
>
> Following the discussion about handling of CXL fixed memory windows on
> arm64 [1] I decided to bite the bullet and move numa_memblks from x86 to
> the generic code so they will be available on arm64/riscv and maybe on
> loongarch sometime later.
>
> While it could be possible to use memblock to describe CXL memory windows,
> it currently lacks notion of unpopulated memory ranges and numa_memblks
> does implement this.
>
> Another reason to make numa_memblks generic is that both arch_numa (arm64
> and riscv) and loongarch use trimmed copy of x86 code although there is no
> fundamental reason why the same code cannot be used on all these platforms.
> Having numa_memblks in mm/ will make it's interaction with ACPI and FDT
> more consistent and I believe will reduce maintenance burden.
>
> And with generic numa_memblks it is (almost) straightforward to enable NUMA
> emulation on arm64 and riscv.
>
> The first 9 commits in this series are cleanups that are not strictly
> related to numa_memblks.
> Commits 10-16 slightly reorder code in x86 to allow extracting numa_memblks
> and NUMA emulation to the generic code.
> Commits 17-19 actually move the code from arch/x86/ to mm/ and commits 20-22
> does some aftermath cleanups.
> Commit 23 switches arch_numa to numa_memblks.
> Commit 24 enables usage of phys_to_target_node() and
> memory_add_physaddr_to_nid() with numa_memblks.
> Commit 25 moves the description for numa=fake from x86 to admin-guide
>
> [1] https://lore.kernel.org/all/[email protected]/
>
> v1: https://lore.kernel.org/all/[email protected]
> * add cleanup for arch_alloc_nodedata and HAVE_ARCH_NODEDATA_EXTENSION
> * add patch that moves description of numa=fake kernel parameter from
>   x86 to admin-guide
> * reduce rounding up of node_data allocations from PAGE_SIZE to
>   SMP_CACHE_BYTES
> * restore single allocation attempt of numa_distance
> * fix several comments
> * added review tags
>
> Mike Rapoport (Microsoft) (25):
>   mm: move kernel/numa.c to mm/
>   MIPS: sgi-ip27: make NODE_DATA() the same as on all other architectures
>   MIPS: sgi-ip27: ensure node_possible_map only contains valid nodes
>   MIPS: sgi-ip27: drop HAVE_ARCH_NODEDATA_EXTENSION
>   MIPS: loongson64: rename __node_data to node_data
>   MIPS: loongson64: drop HAVE_ARCH_NODEDATA_EXTENSION
>   mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
>   arch, mm: move definition of node_data to generic code
>   arch, mm: pull out allocation of NODE_DATA to generic code
>   x86/numa: simplify numa_distance allocation
>   x86/numa: use get_pfn_range_for_nid to verify that node spans memory
>   x86/numa: move FAKE_NODE_* defines to numa_emu
>   x86/numa_emu: simplify allocation of phys_dist
>   x86/numa_emu: split __apicid_to_node update to a helper function
>   x86/numa_emu: use a helper function to get MAX_DMA32_PFN
>   x86/numa: numa_{add,remove}_cpu: make cpu parameter unsigned
>   mm: introduce numa_memblks
>   mm: move numa_distance and related code from x86 to numa_memblks
>   mm: introduce numa_emulation
>   mm: numa_memblks: introduce numa_memblks_init
>   mm: numa_memblks: make several functions and variables static
>   mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing
>     meminfo
>   arch_numa: switch over to numa_memblks
>   mm: make range-to-target_node lookup facility a part of numa_memblks
>   docs: move numa=fake description to kernel-parameters.txt
>
Hi,

I have tested this series on both x86_64 and arm64. It works fine on x86_64.
All numa=fake= options work as they did before the series.

But I am not able to boot the kernel (no printout at all) on arm64 VM
(Mac mini M1 VMWare). By git bisecting, arch_numa: switch over to numa_memblks
is the first patch causing the boot failure. I see the warning:

WARNING: modpost: vmlinux: section mismatch in reference: numa_add_cpu+0x1c (section: .text) -> early_cpu_to_node (section: .init.text)

I am not sure if it is red herring or not, since changing early_cpu_to_node
to cpu_to_node in numa_add_cpu() from mm/numa_emulation.c did get rid of the
warning, but the system still failed to boot.

Please note that you need binutils 2.40 to build the arm64 kernel, since there
is a bug(https://sourceware.org/bugzilla/show_bug.cgi?id=31924) in 2.42 preventing
arm64 kernel from booting as well.

My config is attached.

--
Best Regards,
Yan, Zi

-----BEGIN PGP SIGNATURE-----

iQJDBAEBCgAtFiEE6rR4j8RuQ2XmaZol4n+egRQHKFQFAmahg+cPHHppeUBudmlk
aWEuY29tAAoJEOJ/noEUByhUnJEP+wQgp5y8s+DWBKzngvbc8cZBbgk0AxZhN8E1
M8ych/hIdDOtFHmGA0ncj8R+L6rwIieehibknQU3c71JCfFG44yXB7jHn0tGYKWG
frVxDx3wON5AMJ3pMHIAIVAORIdyuUfAphLhMclej+GZexuVsYreh2Itccp47jS+
GnrKRAKfgtoB77/O2Fq0ui9eupTRTKqO5LUAGQqVUj07yAFKziPDoAcOPzWSlpm3
J9y5evrJSaVm+dQxqhHcFj6AYNdDFKzPatLQ0yAoNf0DJpMrY/5thhNC5xPUHDro
8wrplGPdqmbpjrQlvV8vxwodaJTgiuOsrBrxX99q9wZvbNEsI3LSKHflnFRwhtMW
zDDqL0Dm+CGBXrpnF+hVRmEcCDSomx7gbPiRIZPM3q3Yawg21NVkrCDDMBtOa1H4
gZHhUcO/UXWWlv4y74y379lZ1jLmElaccpz2s5fBVM+gP4cuGonpvZnK7ndQmmBM
JnvSMlZZY9zRIsYdzaQ3y9Tt21rGPfSAHyIzCOGE98CTaxmwj0IEYTksBqwNgk0W
wxse1isSB4cXHxiXz3NxsLYVt7TqigFUx8zfPFvQ2QGJyf2LIDmoB6kPNoURufeP
EFKUn3f6JMFF9IbYCXDOjvsPoEK+VAp7+x9zN5fdb0e+qAGuksFWSvmoke1BVioI
RuFd2bfz
=UPdo
-----END PGP SIGNATURE-----