>32 core support
Andrew Doran <[email protected]> Mon, 23 Mar 2009 22:46:00 +0000
| Newsgroups | gmane.os.netbsd.devel.smp,gmane.os.netbsd.ports.x86-64 |
|---|---|
| Message-ID | <[email protected]> |
For anyone interested in working on >32 core support here is a interesting link. It suggests that `bochs' can be patched to emulate a 48 core system. http://forum.osdev.org/viewtopic.php?f=1&t=19192 x86 currently supports 32 cores/threads out of the box, but it may not be enough for commodity servers within a couple of years. The low-level issues I am aware of are: - There are static arrays that will not scale (MAXCPUS). Some of these could be folded into cpu_data, leaving us with a single array to worry about sizing dynamically. - There are 32-bit masks, the majority of which are in platform-specific code. The kcpuset code needs work to do stuff that the kernel uses, like atomic operations, scans and so on. A few masks are in MI code and some will #error if MAXCPUS > 32. - 32-bit systems don't have enough kernel address space to support large numbers of cores effectively, although i386 could be interesting as a bring-up target with an emulator like bochs. - The x86 apic code currently supports 255 cores max (assuming 255 cores plus 1 IOAPIC). x2apic support may be required in the future. As far as I understand it, future systems may have cores that can only be addressed with the x2apic's 32-bit IDs. - The mutex/rwlock code will not scale to large numbers of cores because it does a linear scan of the CPU list. There are memory order and data structure visibility issues to be overcome. - There are areas where the scheduler could use incremental improvement: it does not know about NUMA, does periodic scans of the CPU list, etc. Clearly this does not cover stuff like scalability of device drivers, networking, vm system and whatnot, does not discuss virtualization / partitioning, it's just the low-level issues that I am aware of given the current set of stuff that we have in the kernel.