Re: NUMA page allocation from next Node
Tharindu Rukshan Bamunuarachchi <[email protected]>
| Newsgroups | org.kernel.vger.linux-numa |
|---|---|
| Message-ID | <[email protected]> |
I have gone through BIOS settings and only applicable setting was
memory type : NUMA or Non-NUMA. (current value is NUMA)
I have attached part of dmesg output.
Is there any other tool or way to gather info ?
DMESG
======
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.27.45-0.1-default (geeko@buildhost) (gcc version
4.3.2 [gcc-4_3-branch revision 141291] (SUSE Linux) ) #1 SMP
2010-02-22 16:49:47 +0100
Command line: root=/dev/disk/by-id/scsi-3600605b0023f45a01449ea30199cc9ae-part1
resume=/dev/disk/by-id/scsi-3600605b0023f45a01449ea30199cc9ae-part3
splash=silent crashkernel=256M-:128M@16M vga=0x314
KERNEL supported cpus:
Intel GenuineIntel
AMD AuthenticAMD
Centaur CentaurHauls
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009c400 (usable)
BIOS-e820: 000000000009c400 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000007d151000 (usable)
BIOS-e820: 000000007d151000 - 000000007d215000 (reserved)
BIOS-e820: 000000007d215000 - 000000007d854000 (usable)
BIOS-e820: 000000007d854000 - 000000007d904000 (reserved)
BIOS-e820: 000000007d904000 - 000000007f68f000 (usable)
BIOS-e820: 000000007f68f000 - 000000007f6df000 (reserved)
BIOS-e820: 000000007f6df000 - 000000007f7df000 (ACPI NVS)
BIOS-e820: 000000007f7df000 - 000000007f7ff000 (ACPI data)
BIOS-e820: 000000007f7ff000 - 000000007f800000 (usable)
BIOS-e820: 000000007f800000 - 0000000090000000 (reserved)
BIOS-e820: 00000000fc000000 - 00000000fd000000 (reserved)
BIOS-e820: 00000000fed1c000 - 00000000fed20000 (reserved)
BIOS-e820: 00000000ff800000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000c80000000 (usable)
DMI 2.5 present.
last_pfn = 0xc80000 max_arch_pfn = 0x100000000
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
last_pfn = 0x7f800 max_arch_pfn = 0x100000000
init_memory_mapping
Using GB pages for direct mapping
0000000000 - 0040000000 page 1G
0040000000 - 007f800000 page 2M
kernel direct mapping tables up to 7f800000 @ 8000-a000
last_map_addr: 7f800000 end: 7f800000
init_memory_mapping
Using GB pages for direct mapping
0100000000 - 0c80000000 page 1G
kernel direct mapping tables up to c80000000 @ 9000-a000
last_map_addr: c80000000 end: c80000000
RAMDISK: 37a03000 - 37fef962
ACPI: RSDP 000FDFD0, 0024 (r2 IBM )
ACPI: XSDT 7F7FE120, 0084 (r1 IBM THURLEY 0 1000013)
ACPI: FACP 7F7FB000, 00F4 (r4 IBM THURLEY 0 IBM 1000013)
ACPI: DSDT 7F7F8000, 2BF3 (r1 IBM THURLEY 3 IBM 1000013)
ACPI: FACS 7F6EC000, 0040
ACPI: TCPA 7F7FD000, 0064 (r0 0 0)
ACPI: APIC 7F7F7000, 011E (r2 IBM THURLEY 0 IBM 1000013)
ACPI: MCFG 7F7F6000, 003C (r1 IBM THURLEY 1 IBM 1000013)
ACPI: SLIC 7F7F5000, 0176 (r1 IBM THURLEY 0 IBM 1000013)
ACPI: HPET 7F7F4000, 0038 (r1 IBM THURLEY 1 IBM 1000013)
ACPI: SRAT 7F7F3000, 0168 (r2 IBM THURLEY 1 IBM 1000013)
ACPI: SLIT 7F7F2000, 0030 (r1 IBM THURLEY 0 IBM 1000013)
ACPI: SSDT 7F7F1000, 0183 (r2 IBM CPUSCOPE 4000 IBM 1000013)
ACPI: SSDT 7F7F0000, 0699 (r2 IBM CPUWYVRN 4000 IBM 1000013)
ACPI: ERST 7F7EF000, 0230 (r1 IBM THURLEY 1 IBM 1000013)
ACPI: DMAR 7F7EE000, 00D8 (r1 IBM THURLEY 1 IBM 1000013)
ACPI: Local APIC address 0xfee00000
SRAT: PXM 0 -> APIC 0 -> Node 0
SRAT: PXM 0 -> APIC 2 -> Node 0
SRAT: PXM 0 -> APIC 4 -> Node 0
SRAT: PXM 0 -> APIC 16 -> Node 0
SRAT: PXM 0 -> APIC 18 -> Node 0
SRAT: PXM 0 -> APIC 20 -> Node 0
SRAT: PXM 1 -> APIC 32 -> Node 1
SRAT: PXM 1 -> APIC 34 -> Node 1
SRAT: PXM 1 -> APIC 36 -> Node 1
SRAT: PXM 1 -> APIC 48 -> Node 1
SRAT: PXM 1 -> APIC 50 -> Node 1
SRAT: PXM 1 -> APIC 52 -> Node 1
SRAT: Node 0 PXM 0 0-80000000
SRAT: Node 0 PXM 0 100000000-680000000
SRAT: Node 1 PXM 1 680000000-c80000000
NUMA: Using 31 for the hash shift.
Bootmem setup node 0 0000000000000000-0000000680000000
NODE_DATA [0000000000009000 - 0000000000020fff]
bootmap [0000000000100000 - 00000000001cffff] pages d0
(7 early reservations) ==> bootmem [0000000000 - 0680000000]
#0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000]
#1 [0000006000 - 0000008000] TRAMPOLINE ==> [0000006000 - 0000008000]
#2 [0000200000 - 0000bcc8b8] TEXT DATA BSS ==> [0000200000 - 0000bcc8b8]
#3 [0037a03000 - 0037fef962] RAMDISK ==> [0037a03000 - 0037fef962]
#4 [000009c400 - 0000100000] BIOS reserved ==> [000009c400 - 0000100000]
#5 [0000008000 - 0000009000] PGTABLE ==> [0000008000 - 0000009000]
#6 [0000001000 - 0000001030] ACPI SLIT ==> [0000001000 - 0000001030]
Bootmem setup node 1 0000000680000000-0000000c80000000
NODE_DATA [0000000680000000 - 0000000680017fff]
bootmap [0000000680018000 - 00000006800d7fff] pages c0
(7 early reservations) ==> bootmem [0680000000 - 0c80000000]
#0 [0000000000 - 0000001000] BIOS data page
#1 [0000006000 - 0000008000] TRAMPOLINE
#2 [0000200000 - 0000bcc8b8] TEXT DATA BSS
#3 [0037a03000 - 0037fef962] RAMDISK
#4 [000009c400 - 0000100000] BIOS reserved
#5 [0000008000 - 0000009000] PGTABLE
#6 [0000001000 - 0000001030] ACPI SLIT
found SMP MP-table at [ffff88000009c540] 0009c540
Reserving 128MB of memory at 16MB for crashkernel (System RAM: 51200MB)
[ffffe20000000000-ffffe200117fffff] PMD ->
[ffff880028200000-ffff8800379fffff] on node 0
[ffffe20011800000-ffffe20019ffffff] PMD ->
[ffff880038000000-ffff8800407fffff] on node 0
[ffffe2001a000000-ffffe20031ffffff] PMD ->
[ffff880680200000-ffff8806981fffff] on node 1
Zone PFN ranges:
DMA 0x00000000 -> 0x00001000
DMA32 0x00001000 -> 0x00100000
Normal 0x00100000 -> 0x00c80000
Movable zone start PFN for each node
early_node_map[7] active PFN ranges
0: 0x00000000 -> 0x0000009c
0: 0x00000100 -> 0x0007d151
0: 0x0007d215 -> 0x0007d854
0: 0x0007d904 -> 0x0007f68f
0: 0x0007f7ff -> 0x0007f800
0: 0x00100000 -> 0x00680000
1: 0x00680000 -> 0x00c80000
On node 0 totalpages: 6288568
DMA zone: 1319 pages, LIFO batch:0
DMA32 zone: 501084 pages, LIFO batch:31
Normal zone: 5677056 pages, LIFO batch:31
On node 1 totalpages: 6291456
Normal zone: 6193152 pages, LIFO batch:31
ACPI: PM-Timer IO Port: 0x588
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x10] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x12] enabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x14] enabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x20] enabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x22] enabled)
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x24] enabled)
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x30] enabled)
ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x32] enabled)
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x34] enabled)
ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x01] disabled)
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x03] disabled)
ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x05] disabled)
ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x11] disabled)
ACPI: LAPIC (acpi_id[0x10] lapic_id[0x13] disabled)
ACPI: LAPIC (acpi_id[0x11] lapic_id[0x15] disabled)
ACPI: LAPIC (acpi_id[0x12] lapic_id[0x21] disabled)
ACPI: LAPIC (acpi_id[0x13] lapic_id[0x23] disabled)
ACPI: LAPIC (acpi_id[0x14] lapic_id[0x25] disabled)
ACPI: LAPIC (acpi_id[0x15] lapic_id[0x31] disabled)
ACPI: LAPIC (acpi_id[0x16] lapic_id[0x33] disabled)
ACPI: LAPIC (acpi_id[0x17] lapic_id[0x35] disabled)
ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 8, version 0, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x09] address[0xfec80000] gsi_base[24])
IOAPIC[1]: apic_id 9, version 0, address 0xfec80000, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
ACPI: HPET id: 0x8086a301 base: 0xfed00000
Using ACPI (MADT) for SMP configuration information
SMP: Allowing 24 CPUs, 12 hotplug CPUs
PM: Registered nosave memory: 000000000009c000 - 000000000009d000
PM: Registered nosave memory: 000000000009d000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
PM: Registered nosave memory: 000000007d151000 - 000000007d215000
PM: Registered nosave memory: 000000007d854000 - 000000007d904000
PM: Registered nosave memory: 000000007f68f000 - 000000007f6df000
PM: Registered nosave memory: 000000007f6df000 - 000000007f7df000
PM: Registered nosave memory: 000000007f7df000 - 000000007f7ff000
PM: Registered nosave memory: 000000007f800000 - 0000000090000000
PM: Registered nosave memory: 0000000090000000 - 00000000fc000000
PM: Registered nosave memory: 00000000fc000000 - 00000000fd000000
PM: Registered nosave memory: 00000000fd000000 - 00000000fed1c000
PM: Registered nosave memory: 00000000fed1c000 - 00000000fed20000
PM: Registered nosave memory: 00000000fed20000 - 00000000ff800000
PM: Registered nosave memory: 00000000ff800000 - 0000000100000000
Allocating PCI resources starting at 98000000 (gap: 90000000:6c000000)
PERCPU: Allocating 61472 bytes of per cpu data
NR_CPUS: 512, nr_cpu_ids: 24, nr_node_ids 2
Built 2 zonelists in Zone order, mobility grouping on. Total pages: 12372611
Policy zone: Normal
__
Tharindu R Bamunuarachchi.
On Fri, Oct 29, 2010 at 9:00 AM, Scott Lurndal <[email protected]> wrote:
> On Fri, Oct 29, 2010 at 07:35:35AM +0530, Tharindu Rukshan Bamunuarachchi wrote:
>> Finally I could isolate the issue further.
>> I tried following kernels and hardware.
>> Issue is visible only with IBM + SLES 11.
>
> Check your ACPI settings, make sure the SRAT and SLIT tables
> are being provided by the BIOS to the kernel.
>
> scott
>
>