Re: How to make FreeBSD's kernel boot a RPi4B with modern RPi* firmware

On Dec 24, 2022, at 19:15, Mark Millard <[email protected]> wrote:

> I finally ran into EARLY_DRIVER_MODULE, BUS_PASS_RESOURCE,
> BUS_PASS_ORDER_MIDDLE and the like and they allow being
> sure that the brcm,bcm2835-dma related setup has been done
> before any use of it is made, despite the order in the
> Device Tree: use an earlier pass for brcm,bcm2835-dma
> related attach. This avoids the kernel crashing during
> boot.
> 
> The example context used below has: serial console with
> USB3 SSD boot media (not requiring a usb_pgood_delay
> setting), booting a stable/13. The RPI4B is a C0T one (no
> 3 GiByte limitation, for example: the PCIe wrapper logic
> has been corrected).
> 
> stable/13's source code changes ( similarly for
> releng/13.1 ):
> 
> diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c b/sys/arm/broadcom/bcm2835/bcm2835_dma.c
> index cab8639bb607..d8b49acfe332 100644
> --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c
> +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c
> @@ -766,5 +766,6 @@ static driver_t bcm_dma_driver = {
> 
> static devclass_t bcm_dma_devclass;
> 
> -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, bcm_dma_devclass, 0, 0);
> +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, bcm_dma_devclass,
> +    0, 0, BUS_PASS_RESOURCE + BUS_PASS_ORDER_MIDDLE);
> MODULE_VERSION(bcm_dma, 1);
> 
> 
> For reference, a 13S snapshot with my kernel build replacing
> the snapshot's kernel, was booted with:
> 
> # strings /boot/msdos/start4.elf | grep VC_BUILD_ID_
> VC_BUILD_ID_USER: dom
> VC_BUILD_ID_TIME: 11:09:05
> VC_BUILD_ID_VARIANT: start
> VC_BUILD_ID_TIME: Oct 26 2022
> VC_BUILD_ID_BRANCH: bcm2711_2
> VC_BUILD_ID_HOSTNAME: buildbot
> VC_BUILD_ID_PLATFORM: raspberrypi_linux
> VC_BUILD_ID_VERSION: c72ad6b26ff40c91ef776b847436094ee63fabee (clean)
> 
> There are new things present that FreeBSD reports
> but ignores, producing messages like:
> 
> clk_fixed4: <Fixed clock> disabled on ofwbus0
> clk_fixed4: Cannot FDT parameters.
> device_attach: clk_fixed4 attach returned 6
> 
> over and over during part of the boot. It seems to
> retry as it goes and thus produce so many messages.
> 
> There was also:
> 
> fb0: <BCM2835 VT framebuffer driver> on simplebus0
> fb0: changing fb bpp from 0 to 24
> mbox0: mbox response error
> fb0: bcm2835_mbox_fb_init failed, err=5
> device_attach: fb0 attach returned 6
> 
> genet0 is working.
> 
> I've not checked if the microsd card slot can be used.
> 
> I used the normal FreeBSD U-Boot since I was not booting
> the NVM3 media that requires extra time (usb_pgood_delay
> would be assigned in my own U-Boot builds).
> 
> For reference, I used my typical sort of config.txt :
> 
> # more /boot/msdos/config.txt
> [all]
> arm_64bit=1
> dtparam=audio=on,i2c_arm=on,spi=on
> dtoverlay=mmc
> dtoverlay=disable-bt
> device_tree_address=0x4000
> kernel=u-boot.bin
> 
> [pi4]
> hdmi_safe=1
> armstub=armstub8-gic.bin
> 
> #
> [all]
> #
> # Local addition that avoids USB3 SSD boot failures that look like:
> #   uhub_reattach_port: port ? reset failed, error=USB_ERR_TIMEOUT
> #   uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port ?
> initial_turbo=60
> # U-Boot that has, for example, a built-in usb_pgood_delay assignment
> # for a media specific issue added:
> #kernel=u-boot.bin.2022.10.arm64
> #
> # Local additions:
> enable_uart=1
> uart_2ndstage=1
> dtdebug=1
> disable_commandline_tags=1
> disable_overscan=1
> #gpu_mem_1024=32
> #
> #program_usb_boot_mode=1
> #program_usb_boot_timeout=1
> 
> # Old RPi3's/RPi2Bv1.2's may ignore [pi4] and the like.
> # That would make the below inappropriate for such contexts.
> [pi4]
> # Locally avoid hdmi_safe's dislay scaling:
> hdmi_safe=0
> #
> armstub=armstub8-gic.bin
> #
> # Local additions:
> over_voltage=6
> arm_freq=2000
> sdram_freq_min=3200
> force_turbo=1
> #
> #total_mem=1024
> #total_mem=991
> [all]
> 
> [pi3] 
> armstub=armstub8.bin
> dtoverlay=pwm
> audio_pwm_mode=0
> [all]
> 
> 
> As for main [so: 14], the devclass_t use is gone, thus
> the source code changes are:
> 
> diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c b/sys/arm/broadcom/bcm2835/bcm2835_dma.c
> index 5f9ecb0b7981..83c4c493a66b 100644
> --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c
> +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c
> @@ -764,5 +764,6 @@ static driver_t bcm_dma_driver = {
>        sizeof(struct bcm_dma_softc),
> };
> 
> -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0);
> +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0,
> +    BUS_PASS_RESOURCE + BUS_PASS_ORDER_MIDDLE);
> MODULE_VERSION(bcm_dma, 1);
> 
> 

I should note that the above is not likely to be
the most appropriate in detail. The boot reports:

# dmesg -a | grep bcm_dma0
bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq 31,32,33,34,35,36,37,38,39,40,41 on simplebus0
bcm_dma0: cannot allocate interrupt
device_attach: bcm_dma0 attach returned 6
bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq 31,32,33,34,35,36,37,38,39,40,41 on simplebus0
bcm_dma0: cannot allocate interrupt
device_attach: bcm_dma0 attach returned 6
bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq 31,32,33,34,35,36,37,38,39,40,41 on simplebus0
bcm_dma0: cannot allocate interrupt
device_attach: bcm_dma0 attach returned 6
bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq 31,32,33,34,35,36,37,38,39,40,41 on simplebus0
bcm_dma0: cannot allocate interrupt
device_attach: bcm_dma0 attach returned 6
bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq 31,32,33,34,35,36,37,38,39,40,41 on simplebus0

where that last (working) one has the message
context:

gic0: <ARM Generic Interrupt Controller> mem 0x40041000-0x40041fff,0x40042000-0x40043fff,0x40044000-0x40045fff,0x40046000-0x40047fff irq 30 on simplebus0
gic0: pn 0x2, arch 0x2, rev 0x1, implementer 0x43b irqs 256
bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq 31,32,33,34,35,36,37,38,39,40,41 on simplebus0

So something involving BUS_PASS_INTERRUPT or later
(but before, say, BUS_PASS_SUPPORTDEV) may be more
appropriate. Possibly:

BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE

(so after gic0).


===
Mark Millard
marklmi at yahoo.com