Re: L4Re on Raspi 3

Hi Paul,

On Sat Jun 19, 2021 at 02:00:18 +0200, Paul Boddie wrote:
> On Monday, 14 June 2021 22:21:50 CEST Adam Lackorzynski wrote:
> > 
> > On Mon Jun 14, 2021 at 13:15:22 +0200, Yanneck Geiger wrote:
> > > Hello, unfortunately I can't get L4Re running on the Raspberry Pi 3 / 3+
> > > using the provided Guider (https://l4re.org/rpi.html). I did the 64-Bit
> > > setup accordingly with the provided U-Boot and the ready-to-use images.
> > > U-Boot itself seems to work fine and I can set the IPs ...
> 
> First of all, apologies for interrupting this thread, but I either forgot 
> about or overlooked the above guide to L4Re on the Raspberry Pi, so I decided 
> to return to an attempt I made two years ago to get things working on the Pi 
> Zero. Although reviving the old thread might be an option, maybe some of my 
> new experiences might be informative here.
> 
> In any case, thanks for bringing the above document to my attention! :-)
> 
> > > Unfortunately I don't know the address to load the .elf file so I'm using
> > > the following commands: "tftpboot bootstrap_hello_rpi3.elf" and "bootelf".
> > > After entering "bootelf" the output on serial spews garbage and after a
> > > few seconds the same garbage is "printed" in an interval of around 1
> > > second. Am I doing anything wrong or am I missing something?
> > 
> > That actually does not sound too bad, the periodic output that comes
> > once a second is probably "Hello World!" being printed, i.e., it worked
> > principally. When there is garbage maybe the Baud rate of the UART is
> > wrong. We're using 115200 Baud, is there anything different in your
> > setup?
> 
> The short version of a much longer story whose details might be pertinent to 
> the above is that care must be taken with the UART configuration on these 
> boards. Although the Pi 3 and 3+ are mentioned above, while I am using the Pi 
> Zero, I did initially think that I might set up Fiasco to use the existing 
> definition for the Pi Zero W. Unfortunately, the UART defined for the Pi Zero 
> W is the 16550, whereas the Pi Zero may be more similar to the original Pi (or 
> Pi 1) and use the PL001.

Ok, thanks, bootstrap is currently not handling this, I'll try to add
it.
 
> (See src/fiasco/src/kern/arm/bsp/rpi/Kconfig for the board configuration 
> options.)
> 
> For me, the effect of using the wrong UART involved the initial bootstrap 
> messages being shown normally followed by subsequent messages appearing very, 
> very slowly - one or two characters per second - and with a certain amount of 
> corruption. Once I had realised my mistake, the problem went away and the 
> output was as expected.

The previous' problem was actually a wrong speed seeting that confuses
the UART. It also happens in Linux if wrong.
 
> Rewinding to the beginning, reviewing the whole process, and reflecting on my 
> earlier experiences, what I discovered this time round was that the Debian 
> toolchains for ARM cross-compilation are definitely unsuitable for the Pi Zero 
> and early generation Pi models (with the BCM2835). Although the CPU options 
> can be given to the toolchains, the Debian toolchains provide built-in 
> function implementations that are compiled for ARMv7. Thus, booting a payload 
> compiled with these toolchains will eventually reach code that will not work 
> on these early SoCs, whereas models using the later SoCs will be fine.

Yes, common problem.

> (Here, the use of U-Boot is helpful because a register dump is produced that 
> indicates where the problem lies. In the first instance, the compiler was 
> generating ARMv7 code because I had not set up the appropriate configuration 
> options in L4Re. Subsequently, the unsuitable built-in or bundled code was 
> encountered and identified. I don't know why U-Boot wasn't this helpful 
> before, though.)
> 
> So, I found that building an ARMv6 toolchain using Buildroot 2020.11, with C++ 
> support enabled, was sufficient, specifically these options:
> 
>     Target options  --->
>         Target Architecture (ARM (little endian))
>         Target Binary Format (ELF)
>         Target Architecture Variant (arm1176jzf-s)
>         Target ABI (EABIhf)
>         Floating point strategy (VFPv2)
>         ARM instruction set (ARM)
> 
>     Toolchain  --->
>         Enable C++ support
> 
> I had to get my head around the build systems of L4Re, Fiasco and this 
> snapshot to then set everything up appropriately. This seemed to involve 
> defining the following files:
> 
> src/fiasco/src/templates/globalconfig.out.arm-rpiz
> 
> ----
> CONFIG_ARM=y
> CONFIG_PF_RPI=y
> CONFIG_PF_RPI_RPI1=y
> CONFIG_MP=y
> CONFIG_ARM_1176=y
> CONFIG_ARM_1176_CACHE_ALIAS_FIX=y
> CONFIG_ARM_CPU_ERRATA=y
> ----
> 
> src/l4/mk/defconfig/config.arm-rpiz
> 
> ----
> CONFIG_BUILD_ARCH_arm=y
> CONFIG_CPU_ARM_ARMV6ZK=y
> CONFIG_PLATFORM_TYPE_rv=y
> ----
> 
> Note the ARMv6 options appropriate for the SoC. My earlier thread went into 
> these details a bit more.

Thanks, I'll try to make this wopr.
 
> One problem I found that maybe Adam might have an opinion about, given that it 
> is his code, is a change I made to the following file:
 
> src/l4/pkg/bootstrap/server/src/platform/rpi.cc
> 
> Here, I removed statements of this form:
> 
> asm volatile("dsb sy");
> 
> This instruction is not supported by this SoC, but it seems that the 
> accompanying instructions might be sufficient by themselves. I was shown the 
> code for the Inferno port to the Raspberry Pi where the "dsb" and "isb" 
> instructions are defined in terms of other instructions which also seem to 
> appear in the above file. So it isn't clear what "dsb sy" adds to what else is 
> already there.

Yes, definetly, dsb sy is a ARMv7 instruction. I'll have this fixed.

> Finally, I had to change the setup script:
> 
> bin/setup.d/04-setup
> 
> These changes are relatively trivial but too verbose to be put inline, so I 
> have attached them to this message. If the attachment gets removed, I can 
> always post it separately or upload it somewhere. These just make sure that 
> the above configuration templates are used, but I also disabled the building 
> of L4Linux since it doesn't interest me and also seemed to produce build 
> errors with the Buildroot toolchain.
> 
> Doing the described steps (make setup, make, deploying the appropriate 
> bootstrap image file, u-boot.bin file, and editing the config.txt file) seemed 
> to yield the desired end result. So it was a good thing to be able to 
> rediscover this and to be able to work through to the end.
> 
> Thanks for helping to finish this unfinished business! :-)

Thanks for the addition too :)



Adam