Re: L4Re: Boot module copying and the MIPS Creator CI20

On Wednesday, 26 April 2023 17:27:35 CEST Paul Boddie wrote:
> 
> As part of reviewing my earlier efforts getting L4Re running on various
> MIPS- based devices, I have been trying to get the bootstrap module to
> start the kernel on the MIPS Creator CI20, and here I encounter a strange
> problem.

[Invalid data at the kernel entry point]

> Does this seem like recognisable behaviour of some kind of misconfiguration?
> For the most part, I am just rebasing my old patches on the current L4Re
> and Fiasco versions, so I didn't expect a problem at this level.

Further troubleshooting suggests that what happens is as follows...

Boot_modules::_move_module copies data from its original location as part of 
the boot payload to locations relative to the module base address. This 
copying is done using "virtual" addresses, even though the bootstrap module 
outputs "physical" addresses in the log. So, for the offending kernel code:

  moving module 00 { 2f4000-39a537 } -> { 2000000-20a6537 } [681272]

Here, the actual copying is done using these "virtual" addresses:

  802f4000-8039a537 -> 82000000-820a6537

As a reminder, these "virtual" addresses are KSEG0 addresses, meaning that 
they correspond directly to the previously indicated addresses, but the 
address mapping is done by the processor, effectively clearing the top bit of 
each address.

Subsequently, the data is copied again to its final, desired location by 
l4_exec_read_exec. However, this function uses entirely "physical" addresses, 
and it is here that the bad data was being found and copied for execution. For 
example:

  2000000-20a6537 -> d3000-d5097

As another reminder, these "physical" addresses are actually virtual addresses 
that are set up in an identity mapping in the _start routine in the 
appropriate crt0.S file.

What I discovered was that whilst bad data was available via the "physical" 
addresses, valid data was available via the "virtual" addresses. Thus, it was 
possible to get the l4_exec_read_exec function to see the appropriate data by 
employing "virtual" addresses when copying. Alternatively, by making the 
_move_module function use the "physical" addresses, I could get the 
l4_exec_read_exec function to see the appropriate data that way.

An alternative to changing the addressing schemes employed by these functions 
was to instead flush the data cache in _move_module. This appears to 
synchronise the views of memory that the KSEG0 mapping and the configured 
mapping have.

However, after doing this, and demonstrating that I can invoke the kernel code 
(using the low-technology method of changing the colour of the CI20's power 
LED), I find that the instruction clearing the processor's status register 
seems to halt execution. To me, this suggests that something else is wrong 
with the memory mapping, but I can't think of what this is at the moment.

It is very odd that this code doesn't work any more. Furthermore, I use the 
QEMU emulation of the MIPS Malta board all the time, and it doesn't have such 
problems. I can imagine that there are details of the cache management that 
are different between the emulated Malta processor and the genuine CI20 
processor, but it doesn't really explain how this worked on the CI20 before. I 
suppose that all the necessary cached data might have been written to memory 
purely by chance.

Anyway, that is as far as I have managed to get, unfortunately. I did attempt 
to try the older Subversion-based L4Re distribution, but I encountered a long-
forgotten issue with the build system and didn't have the appetite to 
continue. Any insights would be welcome.

Paul