mkroot quickstart (was RE: Testing on lots of architecture targets with mkroot.)

Hey Rob,

I just tried the mkroot quickstart instructions in toybox/mkroot/README, an=
d the image failed
to start, due to missing init program.

Maybe I missed something.  What is the appropriate mailing list to use for =
mkroot
discussion?  It is on the toybox list or somewhere else?

I'm thinking a mkroot environment might make a good baseline test system fo=
r short
boot-times (kind of an extreme case to establish a minimum boot time value,
at least for the kernel).

Below was the output.

I checked the root filesystem in toybox/root/i686/fs/bin and there's no 'in=
it' or 'sh' link or program
present.  Does toybox use the same configuration in mkroot as the base toyb=
ox build?
I modified the configuration for toybox to include the 'sh' toy, rebuilt it=
, and verified that the
x86_64 toybox had a working 'sh' toy.

Then I rebuilt the mkroot environment with:
$ mkroot/mkroot.sh CROSS=3Di686 LINUX=3Dlinux
and ran it with:
$ root/i686/run-qemu.sh

I got the same error (missing 'init') on running qemu.
bin/init and bin/sh are still not in root/i686/fs.

Also, the build rebuilt the entire kernel, even though I had just built it =
a few minutes earlier.
Not sure what's going on there.

I note that the mkroot.sh script has 'make clean defconfig toybox install_a=
irlock ...'
which I'm not sure includes 'sh'.  Note that root/build/airlock has 'sh', b=
ut
root/i686/fs/bin does not have 'sh'.

Anyways, thanks for any tips you can provide to help me get this working...
 -- Tim

--------- output from run-qemu.sh
Linux version 7.1.0-rc5 (tbird@timdesk) (i686-linux-musl-gcc (GCC) 11.2.0, =
GNU 6
x86/CPU: Model not found in latest microcode list
BIOS-provided physical RAM map:
BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff]  System RAM
BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff]  device reserved
BIOS-e820: [gap 0x00000000000a0000-0x00000000000effff]
BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff]  device reserved
BIOS-e820: [mem 0x0000000000100000-0x000000000ffdffff]  System RAM
BIOS-e820: [mem 0x000000000ffe0000-0x000000000fffffff]  device reserved
BIOS-e820: [gap 0x0000000010000000-0x00000000fffbffff]
BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff]  device reserved
Notice: NX (Execute Disable) protection missing in CPU!
DMI: SMBIOS 3.0.0 present.
DMI: QEMU Ubuntu 24.04 PC v2 (i440FX + PIIX, arch_caps fix, 1996), BIOS 1.1=
6.3-4
DMI: Memory slots populated: 1/1
tsc: Fast TSC calibration using PIT
tsc: Detected 2095.043 MHz processor
last_pfn =3D 0xffe0 max_arch_pfn =3D 0x100000
MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable M=
TRRs
x86/PAT: PAT not supported by the CPU.
x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WB  WT  UC- UC =20
RAMDISK: [mem 0x0ff6b000-0x0ffdffff]
255MB LOWMEM available.
  mapped low ram: 0 - 0ffe0000
  low ram: 0 - 0ffe0000
[gap 0x10000000-0xfffbffff] available for PCI devices
clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_=
idles
Zone ranges:
  DMA      [mem 0x0000000000001000-0x0000000000ffffff]
  Normal   [mem 0x0000000001000000-0x000000000ffdffff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000001000-0x000000000009efff]
  node   0: [mem 0x0000000000100000-0x000000000ffdffff]
Initmem setup node 0 [mem 0x0000000000001000-0x000000000ffdffff]
On node 0, zone DMA: 1 pages in unavailable ranges
On node 0, zone DMA: 97 pages in unavailable ranges
On node 0, zone Normal: 32 pages in unavailable ranges
Kernel command line: HOST=3Di686 console=3DttyS0=20
Unknown kernel command line parameters "HOST=3Di686", will be passed to use=
r spac.
printk: log buffer data + meta data: 131072 + 409600 =3D 540672 bytes
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes, linear)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes, linear)
Built 1 zonelists, mobility grouping on.  Total pages: 65406
mem auto-init: stack:off, heap alloc:off, heap free:off
Checking if this processor honours the WP bit even in supervisor mode...Ok.
SLUB: HWalign=3D32, Order=3D0-3, MinObjects=3D0, CPUs=3D1, Nodes=3D1
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
Console: colour VGA+ 80x25
printk: legacy console [ttyS0] enabled
clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x1e32e5fed37,=
 maxs
Calibrating delay loop (skipped), value calculated using timer frequency.. =
4190)
Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
CPU: Intel Pentium III (Katmai) (family: 0x6, model: 0x7, stepping: 0x3)
mitigations: Enabled attack vectors: SMT mitigations: off
Speculative Store Bypass: Vulnerable
Spectre V2 : Vulnerable
ITS: Vulnerable
MDS: Vulnerable
Spectre V1 : Vulnerable: __user pointer sanitization and usercopy barriers =
onlys
L1TF: Kernel not compiled for PAE. No mitigation for L1TF
x86/fpu: x87 FPU will use FXSAVE
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
VFS: Finished mounting rootfs on nullfs
Performance Events: PMU not available due to virtualization, using software=
 eve.
signal: max sigframe size: 1440
Memory: 250396K/261624K available (4793K kernel code, 608K rwdata, 968K rod=
ata,)
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns:=
 764s
posixtimers hash table entries: 512 (order: 0, 2048 bytes, linear)
futex hash table entries: 256 (4096 bytes on 1 NUMA nodes, total 4 KiB, lin=
ear).
NET: Registered PF_NETLINK/PF_ROUTE protocol family
clocksource: pit: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 160=
1818s
PCI: PCI BIOS revision 2.10 entry at 0xfd1d9, last bus=3D0
PCI: Using configuration type 1 for base access
SCSI subsystem initialized
PCI: Probing PCI hardware
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
pci_bus 0000:00: root bus resource [mem 0x00000000-0xffffffff]
pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpo=
int
pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpo=
int
pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpo=
int
pci 0000:00:01.1: BAR 4 [io  0xc040-0xc04f]
pci 0000:00:01.1: BAR 0 [io  0x01f0-0x01f7]: legacy IDE quirk
pci 0000:00:01.1: BAR 1 [io  0x03f6]: legacy IDE quirk
pci 0000:00:01.1: BAR 2 [io  0x0170-0x0177]: legacy IDE quirk
pci 0000:00:01.1: BAR 3 [io  0x0376]: legacy IDE quirk
pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpo=
int
pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI
pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB
pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpo=
int
pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]
pci 0000:00:02.0: BAR 2 [mem 0xfebb0000-0xfebb0fff]
pci 0000:00:02.0: ROM [mem 0xfeba0000-0xfebaffff pref]
pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000df=
fff]
pci 0000:00:03.0: [8086:100e] type 00 class 0x020000 conventional PCI endpo=
int
pci 0000:00:03.0: BAR 0 [mem 0xfeb80000-0xfeb9ffff]
pci 0000:00:03.0: BAR 1 [io  0xc000-0xc03f]
pci 0000:00:03.0: ROM [mem 0xfeb00000-0xfeb7ffff pref]
pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00
pci 0000:00:01.0: PIIX/ICH IRQ router [8086:7000]
e820: register RAM buffer resource [mem 0x0009fc00-0x0009ffff]
e820: register RAM buffer resource [mem 0x0ffe0000-0x0fffffff]
pci 0000:00:02.0: vgaarb: setting as boot VGA device
pci 0000:00:02.0: vgaarb: bridge control possible
pci 0000:00:02.0: vgaarb: VGA device added: decodes=3Dio+mem,owns=3Dio+mem,=
locks=3Dnoe
vgaarb: loaded
clocksource: Switched to clocksource tsc-early
NET: Registered PF_INET protocol family
IP idents hash table entries: 4096 (order: 3, 32768 bytes, linear)
tcp_listen_portaddr_hash hash table entries: 1024 (order: 0, 4096 bytes, li=
near)
Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
TCP established hash table entries: 2048 (order: 1, 8192 bytes, linear)
TCP bind hash table entries: 2048 (order: 2, 16384 bytes, linear)
TCP: Hash tables configured (established 2048 bind 2048)
UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
NET: Registered PF_UNIX/PF_LOCAL protocol family
pci_bus 0000:00: resource 4 [io  0x0000-0xffff]
pci_bus 0000:00: resource 5 [mem 0x00000000-0xffffffff]
pci 0000:00:01.0: PIIX3: Enabling Passive Release
pci 0000:00:00.0: Limiting direct PCI/PCI transfers
pci 0000:00:01.0: Activating ISA DMA hang workarounds
PCI: CLS 0 bytes, default 32
platform rtc_cmos: registered fallback platform RTC device
workingset: timestamp_bits=3D30 (anon: 26) max_order=3D16 bucket_order=3D0 =
(anon: 0)
squashfs: version 4.0 (2009/01/31) Phillip Lougher
Unpacking initramfs...
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
serial8250: ttyS0 at I/O 0x3f8 (irq =3D 4, base_baud =3D 115200) is a 16550A
loop: module loaded
scsi host0: ata_piix
scsi host1: ata_piix
ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc040 irq 14 lpm-pol 0
ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc048 irq 15 lpm-pol 0
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
e1000 0000:00:03.0: found PCI INT A -> IRQ 11
Freeing initrd memory: 468K
ata2: found unknown device (class 0)
ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5
e1000 0000:00:03.0 eth0: (PCI:33MHz:32-bit) 52:54:00:12:34:56
e1000 0000:00:03.0 eth0: Intel(R) PRO/1000 Network Connection
NET: Registered PF_INET6 protocol family
Segment Routing with IPv6
In-situ OAM (IOAM) with IPv6
NET: Registered PF_PACKET protocol family
sched_clock: Marking stable (958101015, 36815557)->(1006698485, -11781913)
netconsole: network logging started
Freeing unused kernel image (initmem) memory: 412K
Write protecting kernel text and read-only data: 5764k
Run /init as init process
Failed to execute /init (error -2)
Run /sbin/init as init process
Run /etc/init as init process
Run /bin/init as init process
Run /bin/sh as init process
Kernel panic - not syncing: No working init found.  Try passing init=3D opt=
ion to.
CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 7.1.0-rc5 #1 PREEMPTLAZY=20
Hardware name: QEMU Ubuntu 24.04 PC v2 (i440FX + PIIX, arch_caps fix, 1996)=
, BI4
Call Trace:
 dump_stack_lvl+0x52/0x5c
 dump_stack+0x12/0x18
 vpanic+0x284/0x28c
 ? rest_init+0xa8/0xa8
 panic+0xe/0x10
 kernel_init+0x15b/0x1b8
 ret_from_fork+0x1c1/0x1fc
 ? rest_init+0xa8/0xa8
 ret_from_fork_asm+0x12/0x18
 entry_INT80_32+0xef/0xef
Kernel Offset: disabled
Rebooting in 1 seconds..

> -----Original Message-----
> From: Rob Landley <[email protected]>
> Sent: Tuesday, May 26, 2026 12:52 PM
> To: [email protected]
> Subject: Testing on lots of architecture targets with mkroot.
>=20
> I promised Tim I'd explain to him how I regression test on s390x and sh4 =
and mips and powerpc and so on in parallel, and he was in the
> audience for my wednesday talk at ELC, but I'm not sure that gave him eve=
rything he needed to do proper automated
>=20
> I promised Tim I'd explain to him how I regression test on s390x and sh4
> and mips and powerpc and so on in parallel, and he was in the audience
> for my wednesday talk at ELC, but I'm not sure that gave him everything
> he needed to do proper automated regression testing under qemu on lots
> of architectures in parallel.
>=20
> The source repo I used was: https://codeberg.org/landley/toybox
>=20
> Prebuilt binaries of all the outputs are at https://landley.net/bin/
> with several releases worth of history. (Although it'll probably go down
> at the end of the year to avoid Gavin Newsom's $7500/download fines.)
>=20
> There's a README in the toybox/mkroot directory, but also instructions
> at https://landley.net/toybox/faq.html#mkroot and
> https://landley.net/talks/mkroot-2023.txt (which was the previous time I
> gave a mkroot talk).
>=20
> You'll need cross compilers: my musl+gcc cross (and native) compilers
> are built with Rich Felker's
> https://github.com/richfelker/musl-cross-make using a wrapper script in
> scripts/mcm-buildall.sh in the toybox source. The binary tarballs are in
> https://landley.net/bin/toolchains but you'll probably want to build
> them from source.
>=20
> If you run that mcm-buildall.sh script in a fresh musl-cross-make
> checkout directory with no arguments it builds toolchains for all the
> targets I've managed to get working so far.
>=20
> If you want to rebuild just one, you can feed it any of the strings from
> https://codeberg.org/landley/toybox/src/branch/master/scripts/mcm-buildal=
l.sh#L18
> as an argument, ala:
>=20
> $ ~/toybox/mcm-buildall.sh m68k::
>=20
> Note that these toolchains are statically linked with a host compiler to
> be portable, so the script will first build a musl host compiler (see
> line 38 in the buildall script) because glibc is incompetent at static
> linking.
>=20
> In theory this means you can just grab a tarball and extract it
> somewhere and add it to your $PATH and it should just work. There's also
> a https://landley.net/bin/toolchains/25-03-2024/musl-cross-make.tar.xz
> with everything (all the source and all the targets it built) to shut
> Bradley Stallman up.
>=20
> The test harness script is mkroot/testroot.sh again in the toybox
> source. Run it from the top level (toybox source) directory and it'll
> iterate through all the images under root/ and try to run them under
> qemu (you need qemu-system-* and mksquashfs in your $PATH, it'll
> complain if either is missing). It provides a squashfs filesystem with
> controls scripts and test data, so instead of booting to a shell prompt
> the qemu instance runs a test script and then exits on its own. It
> prints out various "=3D=3D=3D thingy" lines for successful tests, and the
> pass/fail thing at the end says whether it got enough =3D=3D=3D lines of
> output. If not, look at root/build/log/*-test.txt to see what output it
> DID get.
>=20
> I did the start of a busybox-based version at
> https://codeberg.org/landley/toybox/src/branch/master/mkroot/packages/bus=
ybox
> a while back but unfortunately building busybox defconfig is INSANE
> these days. You'll notice rather extensive patching with sed to remove
> build dependencies toybox doesn't have, and even then it was mad about
> something (I forget what).
>=20
> That said, you can also just do:
>=20
> mkroot/mkroot.sh NOAIRLOCK=3D1 LINUX=3D~/linux CROSS=3Dallnonstop mybusyb=
ox
>=20
> To skip the airlock step and run your own much simpler busybox build
> script (just stick it in mkroot/packages), and as long as your script
> sets NOTOYBOX=3D1 before it exits you should get the directory layout, the
> init script, the kernel build and packaging, and the run-qemu.sh for the
> target, with whatever additional files YOU installed (under $ROOT/). So
> a busybox based filesystem instead of toybox shouldn't be a big lift,
> just wasn't a priority for me.
>=20
> But since you're not testing _toybox_, and toybox isn't 1.0 yet, that
> would probably give you an environment much closer to what you want. :)
>=20
> Note that mkroot.sh is almost entirely self-contained, merging it into
> toybox and having LINUX=3D be specified on the command line was just so I
> didn't have to download any packages in the default build. (There's
> download plumbing in scripts/plumbing, I think I went over that in the
> talk. The "dropbear" package is another one to read to see how that
> stuff works.)
>=20
> Rob