Re: L4Re: MIPS SMP cache initialisation may be broken for some CPUs

On 2025-07-20 20:42, Adam Lackorzynski wrote:
> 
> On Mon Jun 23, 2025 at 00:34:41 +0200, Paul Boddie wrote:
>> 
>> Does anyone have any thoughts or recollections about this cache 
>> initialisation
>> routine?
> 
> I'm afraid, I have not.

MIPS cache initialisation always seems to involve recipes and caveats. 
Perhaps the level 2 cache is the thing that makes this SoC different to 
various other ones.

MIPS is becoming a rather niche architecture now, its one-time corporate 
guardian now having pivoted to RISC-V and coming under the ownership of 
GlobalFoundries, who with one of their more advanced fabs must count as 
some kind of neighbour of yours.

>> On the topic of actually starting secondary cores, this SoC has its 
>> own set of
>> registers for resetting cores, interrupt control, mailboxes, and so 
>> on. I have
>> developed a fairly simple abstraction that performs similar core 
>> bootstrapping
>> work to that of the Cm abstraction (in src/kern/mips/cm.cpp). I then 
>> extend
>> the Platform_control abstraction so that it will call the
>> alloc_secondary_boot_code method, followed by the appropriate method 
>> to start
>> the secondary cores.
> 
> Sounds good. If it's just for this single BSP, all the code could also
> be in there.

I will probably try and consolidate the code for this family of SoCs.

>> Is there any documentation for the SMP architecture in Fiasco and 
>> L4Re? I note
>> that interprocessor interrupts (IPI) are one element that I might need 
>> to
>> support, probably using the mailboxes.
> 
> Yes. some cross-core notification is needed, i.e., some mechanism to
> trigger an interrupt on another code. Cache-coherency should work. And
> then it should be fine I guess.

It seems that the IPI mechanism in src/kern/mips/ipi-mips.cpp works 
fine, or at least sufficiently well for the debugger to use it. I will 
need to reacquaint myself with the reliability of the debugger in the 
next few days because there are some problems with the UART on the board 
I am using that prevent effective testing.

The IPI mechanism as provided appears to use some memory as the channel 
through which information is communicated. It seems that IPI signalling 
in src/kern/mips/gic.cpp involves the MIPS GIC, but that is not provided 
by this particular SoC. Instead, the SoC provides a number of 
core-specific mailboxes with interrupts raised when the mailboxes are 
written to.

I thought that I might replace the implementation in ipi-mips.cpp with 
something that makes use of these mailboxes more effectively. Currently, 
a non-zero value written to a mailbox causes an IPI, but I don't make 
use of this value, instead relying on the existing IPI implementation to 
set the condition in its _rq array, which appears to maintain the status 
of the different IPI conditions (which itself employs words to hold 
boolean values, presumably to facilitate use of the architecture's 
atomic update primitives).

I imagine that I could instead have an IPI implementation that updates 
the mailbox for a given core, setting a bit in the mailbox register 
corresponding to the condition. Since there is some form of rudimentary 
locking provided by the SoC, a read-update-write sequence should be 
fairly safe. This was not a priority when I just wanted to get it all 
working.

The state of this work reached the point where I think I had implemented 
core-specific interrupt conditions correctly, along with the appropriate 
semantics for the set_cpu operation. Here, my mental model is a bit 
cloudy. If I create an IRQ object in L4Re, I wonder if it is bound to a 
particular CPU/core or whether it may be made to schedule any CPU/core 
upon an interrupt condition occurring.

(I have been wondering similar things about the Linux kernel where there 
is a substantial framework for interrupt handling but relatively little 
coherent documentation about how it all fits together.)

>> Thanks in advance for any advice or feedback that may be given!
> 
> Sorry for the delay.

No worries! I haven't been able to look at this for a while, myself. I 
appreciate the reply!

Paul
_______________________________________________
l4-hackers mailing list -- [email protected]
To unsubscribe send an email to [email protected]