Re: Supporting 2nd Processors in Open BSD

I'm going to comment against all things telling me I have better
thing to do with my time.

On Friday, March 21, Robert Davies wrote:
> The issue that has held back development of SMP in Open BSD, is the large 
> amount of changes, in very many and complex areas of the OS kernel.  Some a 
> few years back did have processors booting and initialised (roughly bout the 
> 2.9 release or so), but core OpenBSD development continued a pace without any
> regard to SMP support.

You've almost hit the nail on the head.  The "problem" is that there are
many more important things that need to be done, other than SMP.  Simply
rewriting portions of the interrupt handling code for x86 would already
be a good thing.  Or rewriting the x86 pmap to do PAE is more pressing to
me than having an SMP box.  Tearing appart 


> It would be much more realistic to cut back on the ambitions, and have a 
> project deliver something more self contained with a low impact on current 
> source.  It strikes me that treating the 2nd CPU as a device allows simpler 
> goals like :

Some of this may not be easily possible.  For example:

Using the second (or whatever) cpu on an x86 box means you will have
to support the APIC devices.  These things route interrupts.  In other
words, now you're dealing with converting the old interrupt controller
code to deal with the APIC.  This includes many nasty little things
like code in locore.S, and machdep.c, code which deals with nasty, CPU
dependant bugs.  Not to mention the ugly x86 architecture...


> 1)  Intitialise 2nd Processor as Slave during boot up

In order to do this, usually many other things need to be operating.
For example (and I'm not sure if the sparc64 code deals or not), on
the sparc64 arch I believe you even need to have special code in the
boot code to catch the "extra" CPU's, such that you only run one on
a non SMP system.


> 2)  Locking primitives for synchronisation, between Master & Slave processor

Including TLB shootdown, and other nasty things.  On the x86, you're
going to need the APIC stuff...


> 3)  2nd CPU Crypto driver, where a process can punt calculations to the 
> unused CPU, treating it much like a hardware encryption device.

The cost of a hifn is small, and will trounce your second CPU without even
breaking a sweat.  The cost/perfomance ratio is not really worth it.  IE:
I can spend 1 year trying to get my second CPU to do crypto work, or I can
spend $200 (or whatever), and have a dedicated crypto engine working on the
issue today.


> Actually getting some code contributions into the core Open BSD kernel, 
> without significantly increasing the complexity of that kernel, would be  
> implementable.  As there is already support for hardware devices for such 
> tasks, a "2nd CPU software emulation" ought not to be disruptive.
> 
> Without the support and help of the core kernel developers, there is no 
> chance of things changing.  The objective "Produce a working SMP kernel" is 
> simply too big for a cinderella side-project.

Instead of trying to solve the problem in this way, I would recommend that
you look at the big picture.  Find the places in the kernel that prevent
SMP from working, and that need biglock or other forms of locking.  If at
all possible, rewrite them to be re-entrant, and otherwise lock-free, but
SMP correct.  In this way, you are re-writing older code, will start to
appreciate the complexity of the problem involved, and contribute new and
better working code, hopefully moving things closer to a kernel that can
deal with NCPU > 1.


> The fact is already now with Intel's Hyper Threading and upcoming dual-core 
> Processor dies, the lack of SMP support is going to become more of a problem 
> for OpenBSD, so with time the attitudes are likely to change.  Few ppl are 
> really willing to wait 1.5 years to buy an upgrade to acquire the today's 
> performance, they run FreeBSD or Linux instead. 

Sure.  Pesonally I'd be happy with my OpenBSD machines making more
efficient use of the hardware they are currently running on.  As such,
there is little point to spooling up another CPU...  it will make the
bad, contention parts of the kernel just more congested.  As such, I
will be more excited seeing UVM, UBC, and other things like that going
into the kernel, than SMP.  Of course, each has their own fetish...  :-)

--Toby.