G(lobal) bit and ASID management

(Caffeine overdose in sunny comfortable Sunday...)

I don't understand the following comments.   From mipsX_subr.S,

 * mips3_TLBInvalidException --
 * ...
 *      The case of wired TLB entries is special.  The wired TLB entries
 *      are used to keep the u area TLB's valid.  The PTE entries for these
 *      do not have MIPS3_PG_G set; the kernel instead relies 
 *      on the switch_resume function to set these bits.
 *      
 *      To preserve this situation, we set PG_G bits on the "other" TLB entries
 *      when they are wired.

I was puzzled for a while and now am suspicious there is
a confusion on the intent of G(lobal) bit in MIPS TLB
machinery.

The point is, there is nothing to do between TLB G bit
and VM wiring.

Here goes MIPS ASID design explanation.

MIPS is the one of pioneering processors in which ASID plays important
role for TLB management.   It was intended for a kind of address space
tag or extender to distinguish multiple virtual address spaces.  UNIX-like 
protected OS runs multiple processes simultaneously given any moments.
Assigning unique ASID for each process, TLB entries which
belong to distinct processes can comfortably co-exist inside
TLB.  As Processor is aware of current ASID, the right TLB entry is
chosen for VA->PA translation of current process, while leaving entries
tagged with non-current ASIDs safely neglected.  Without ASID, TLB
would be sorely for current VA->PA translation of a single process.  In that
case, TLB contents must be entirely flushed off on every context switch.
This situation is much like VAC (virtual address index with virtual address
tagged cache) headache in which no cache contents can survive across
context switch.  Since TLB is cache for address translation, the effective
use of contents is very critical for whole OS performance.

ASID is maintained in TLBHi register.  It sounds rather peculiar.  Setting
new ASID value in TLBHi register switches processor's current ASID.
The design trick makes clearer when TLB miss occurs.  At the time TLBHi
register has faulting VADDR combined with current ASID.  TLB refill
handler searches the matching PTE for the address, sets the value
to TLBLo register and then induces the pair into TLB (with tlbwr insn).
In this way, ASID management is mostly invisible, but it plays critical role
for efficient TLB management.

The TLB G bit is necessary to maintain kernel address space.  TLB
entries which have G bit are immune from ASID match.  Only VA
address is consulted to TLB lookup.  This is comfortable for KSEG2
which is the single space shared across multiple processes.  Marking
TLB entry as G is done through TLBLo register.  So, given
any time, TLB might be filled of entries with distinct ASIDs, and 
among them, any KSEG2 entries have G bit assigned, while their ASIDs
vary depending on which process brought them into TLB so far.  

NetBSD/mips we have now maintains PTEs in this way;
- user address map in two level tree in 10+10+12 address decode.
- kernel address map in a linear array.
The latter is subject to G bit.

Now, go further than principles.  R4K TLB is re-designed to make
possible larger VA and PA (PFN was shifted right; seems infuenced by
R6000 TLB).   The twist is that TLBLo is now a pair of two, TLBLo0
and TLBLo1.  This is a rather strange arrangement.   As long as my
limited knowledge covers, no other processor runs TLB in that way.
This scheme effectively doubles TLB capacity with smaller hard
circuit cost.

The downside of paired TLBLo[0|1] scheme is it's now awkward to
manipulate a single PTE independently, while TLB refill is OK as long
as PTE[0] and PTE[1]  are induced at a time.  G bit management is
harmed as well since two G bits are necessary in the PTE pair to
mark the TLB entry as G.

Gross effect is; NV (not valid) kernel PTE should be PG_G, not 0 (zero),
all the time and PTE zap must be done by *pte &= PG_G; logic...

I hope this would contribute the popularity and acceptance of NetBSD/mips.

Toru Nishimura/ALKYL Technology