Re: TCR and Stack in the PowerPC implementation

On Feb 5, 2007, at 3:22 PM, Christian, Martin, OPEE45 wrote:

> Maybe someone (Joshua) could bring some light into the stack issue on
> PowerPC. I tried to understand the code, but there are still a lot of
> things hiding in darkness. :-(
>
> 1.) In Appendix C you are using negative offsets for the TCR fields.
> Why? The stack pointer (r1) should point to the lowest address and  
> thus
> only positive offsets should be valid. And sometimes you also use
> negative offsets in inlined assembler (like in tcb_t::switch_to()).  
> That
> confuses me. :-(


The negative offsets for addressing into the UTCB were copied from  
x86, because I made the mistake of not asking the others why they  
were using negative offsets on x86.  PowerPC doesn't need the  
negative offsets.  To make the negative offsets work, the UTCB  
pointer points into the middle of the UTCB region.  Thus adding the  
UTCB pointer (which is r2 if I remember correctly) to a negative  
offset will bring you closer to the beginning of the UTCB.

Your question here is mixing TCR fields and TCB thread switching,  
which are completely unrelated.  I don't remember why I'm using  
negative offsets in the thread switching code from the top of my head.

>
> 2.) Where is the machine state saved on a thread switch? Is the  
> threads
> stack used?

In a cooperative thread switch between two kernel threads, the  
compiler saves most of the state onto the outgoing kernel stack.  The  
amount of state is determined by the compiler, for it analyzes the  
inlined assembler arguments, and determines how much needs to be  
preserved.  My inlined assembler explicitly saves two registers.   
There is a little ugliness there, becomes sometimes two registers  
aren't enough, and causes compilation errors with a complaint of not  
being able to satisfy the inlined assembler constraints.

Note that after benchmarking, I determined that I should change the  
compiler's ABI.  The standard ABI is optimized for typical  
application code, and not microkernel code.  I changed the ABI (via  
gcc's command line parameters) to change the calling conventions, and  
whom should preserve which registers across function calls.  This is  
related to the thread switch, because it reduces the amount of data  
that the compiler has to store across a cooperative thread switch.

>
> 3.) How many stacks are there? The kernel stack per thread is  
> located in
> the KTCB area following the TCB entry, isn't it? And the thread has  
> it's
> own user level stack, I assume? And there must be a stack for kernel
> internal function calls?


The KTCB and TCB are the same thing.  The stack is at the top of the  
TCB, and grows downwards towards the TCB's data members.  I don't  
remember whether I ever determined if the stack has sufficient space  
for all code paths.  Compilers should do that automatically, but  
compilers are broken.

The user code has its own stack.  But that stack isn't located on the  
UTCB.  It is maintained in a different area.

Consider referring to:
http://i30www.ira.uka.de/teaching/coursedocuments/130/thread.pdf


> 4.) Where would I find the saved machine state of a runnable thread?

The kernel state, or the user state?  If the user thread has entered  
the kernel, which might not be the case on multiprocessor, user state  
would be somewhere near the top of the kernel stack in its TCB.  It  
will be located at a predetermined location.  If you want the kernel  
state, then that means you're running on a different kernel thread;  
the kernel thread state will be at a random location on the stack of  
the TCB --- that location is stored in the TCB (I forgot the member's  
name that points to it).


Note that it is quite difficult to inductively learn the concepts of  
a microkernel from its implementation; you'd have to see lots of  
implementations to induce the proper concept.  Porting Pistachio is  
much easier if you start from its concepts, and deduce the  
implementation.  Unfortunately it is hard to find material with the  
concepts.  But all of the Unis (UKa, Dresden, and UNSW) have some  
material that can help.

Josh


>
> I hope someone (Jushua) finds time to answer these questions. ;-)
>
> Best,
>
> Martin.
>
>