[Bug 292494] powerpc64: booting qemu + mac99 results in crash (kernel DSI write trap) after openfirmware call once MMU/PMAP is setup

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=292494

--- Comment #6 from Adrian Chadd <[email protected]> ---
ok, so the tl;dr is its a bug in openbios (git.qemu.org/openbios.git /
https://gitlab.com/qemu-project/openbios).

Specifically, in arch/ppc/qemu/start.S:

```
#ifdef __powerpc64__
#define STKOFF STACKFRAME_MINSIZE
#define SAVE_SPACE 320
#else
#define STKOFF 8
#define SAVE_SPACE 144
#endif
```

However, the ofw entry point is doing this before calling its client code:

```
14:45 <@adrian_>    0x00000000fff02678:  stw     r28,140(r1)
14:45 <@adrian_>    0x00000000fff0267c:  stw     r29,144(r1)
14:45 <@adrian_>    0x00000000fff02680:  stw     r30,148(r1)
14:45 <@adrian_> => 0x00000000fff02684:  stw     r31,152(r1)
14:45 <@adrian_>    0x00000000fff02688:  bl      0xfff0e6cc
```

So it's smashing its own stack. Its stack here gets intialised to 0, and they
reserve 144 bytes, and in 32 bit mode the pointer wraps around and overwrites
the first three 32 bit words at address 0:

```
15:08 <@adrian_> (kgdb) x/32x 0
15:08 <@adrian_> 0x0:    0x60000000      0x60000000      0x4bfffffc     
0x00000000
15:08 <@adrian_> (kgdb) printf "0x%lx\n", $r30
15:08 <@adrian_> 0x3458b34
15:08 <@adrian_> a couple si steps later and
15:09 <@adrian_> 0x00000000fff02688 in ?? ()
15:09 <@adrian_> (kgdb) x/32x 0
15:09 <@adrian_> 0x0:    0x00000000      0x03458b34      0x00000000     
0x00000000
```

Now, this isn't an immediate crash as long as there's memory mapped there, but
once we setup the pmap via pmap_bootstrap() and the direct map removes mappings
to the first page so NULL pointer dereferences -> trap, that wrapping will
fail. That's why it was succeeding on some openfirmware calls early in boot,
but eventually they'd fail.

The fix is adding extra space in openbios:

```
diff --git a/arch/ppc/qemu/start.S b/arch/ppc/qemu/start.S
index c679230..407d498 100644
--- a/arch/ppc/qemu/start.S
+++ b/arch/ppc/qemu/start.S
@@ -503,7 +503,7 @@ _GLOBAL(saved_stack):
 #define SAVE_SPACE 320
 #else
 #define STKOFF 8
-#define SAVE_SPACE 144
+#define SAVE_SPACE 160
 #endif

 GLOBL(of_client_callback):
```

Which stops FreeBSD panicing at boot and then, well, we get a bit further and
panic for a couple of other reasons.

So there's a few things that have popped out of here:

* I need to go fix the endless faulting due to the stackframe being invalid
during panic() -> kdb -> backtrace -> invalid stackframe -> trap -> fault ->
panic() -> kdb -> backtrace. ..

* I need to go fix the qemu openbios upstream so (a) it builds on freebsd
without local changes to find our gcc14 cross compiler, and (b) add that space

* .. then fix up the rest of the fun panics I'm seeing in the init paths.

-- 
You are receiving this mail because:
You are the assignee for the bug.