Re: 13.1-RELEASE image fails to boot Rev 1.4 8 GiByte RPi4B (B0T vintage SOC) when dd'd to the example USB3 media then used to try to boot via USB3 port

On 2022-Jul-9, at 21:23, Mark Millard <[email protected]> wrote:

> On 2022-Jul-9, at 18:59, Mark Millard <[email protected]> wrote:
> 
>> On 2022-Jul-5, at 15:45, Mark Millard <[email protected]> wrote:
>> 
>>> On 2022-Jul-5, at 14:17, Mark Millard <[email protected]> wrote:
>>> 
>>>> Summary for Rev 1.4 8 GiByte RPi4B (B0T vintage SOC) context:
>>>> 
>>>> A) One type of USB3 media (USB2 compatible) works for booting via USB2-port.
>>>> B) The same media fails for USB3-port based boot attempts.
>>>> C) I boot the same type of media for main [so: 14] instead of 13.1-RELEASE.
>>>> D) An alternate type of USB3 media (USB2 compatible) booted via USB3 just
>>>> fine via 13.1-RELEASE.
>>>> 
>>>> The details . . .
>>>> 
>>>> I intended for for use in the Rev 1.4 8 GiByte RPi4B's
>>>> by first going onto USB3 media (of the same kind I
>>>> normally use on the RPi4B's with main and the like).
>>>> So, I tried: 
>>>> 
>>>> # dd if=FreeBSD-13.1-RELEASE-arm64-aarch64-RPI.img of=/dev/da0 \
>>>> bs=1m conv=sync status=progress
>>>> 
>>>> and then tried to boot the RPi4B via the media and it gets
>>>> the following (the kernel loads and runs but . . .):
>>>> 
>>>> . . .
>>>> Trying to mount root from ufs:/dev/ufs/rootfs [rw]...
>>>> uhub0: 5 ports with 4 removable, self powered
>>>> ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0
>>>> uhub1 on uhub0
>>>> uhub1: <vendor 0x2109 USB2.0 Hub, class 9/0, rev 2.10/4.21, addr 1> on usbus0
>>>> Root mount waiting for: usbus0
>>>> uhub1: 4 ports with 4 removable, self powered
>>>> Root mount waiting for: usbus0
>>>> uhub_reattach_port: port 3 reset failed, error=USB_ERR_TIMEOUT
>>>> uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 3
>>>> mountroot: waiting for device /dev/ufs/rootfs...
>>>> Mounting from ufs:/dev/ufs/rootfs failed with error 19.
>>>> 
>>>> Loader variables:
>>>> vfs.root.mountfrom=ufs:/dev/ufs/rootfs
>>>> vfs.root.mountfrom.options=rw
>>>> 
>>>> Manual root filesystem specification:
>>>> <fstype>:<device> [options]
>>>>   Mount <device> using filesystem <fstype>
>>>>   and with the specified (optional) option list.
>>>> 
>>>> eg. ufs:/dev/da0s1a
>>>>     zfs:zroot/ROOT/default
>>>>     cd9660:/dev/cd0 ro
>>>>       (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)
>>>> 
>>>> ?               List valid disk boot devices
>>>> .               Yield 1 second (for background tasks)
>>>> <empty line>    Abort manual input
>>>> 
>>>> mountroot> 
>>>> 
>>>> Plugging into the other USB3 port and attempting the
>>>> power on boot just changes the port numbers from 3 to 2
>>>> in the sequence.
>>>> 
>>>> Plugging into a USB2 port does not have the problem and
>>>> it completes the boot and operates. (The media is USB3
>>>> but USB2 compatible as well.)
>>>> 
>>>> Adding to /boot/loader.conf :
>>>> 
>>>> # First, 3 additions:
>>>> kern.cam.boot_delay=10000
>>>> vfs.mountroot.timeout=10
>>>> vfs.root_mount_always_wait=1
>>>> 
>>>> and retrying via a USB3 port just results in:
>>>> 
>>>> . . .
>>>> Root mount waiting for: usbus0 CAM
>>>> uhub_reattach_port: port 3 reset failed, error=USB_ERR_TIMEOUT
>>>> uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 3
>>>> Root mount waiting for: CAM
>>>> Root mount waiting for: CAM
>>>> Root mount waiting for: CAM
>>>> Root mount waiting for: CAM
>>>> Root mount waiting for: CAM
>>>> Root mount waiting for: CAM
>>>> Mounting from ufs:/dev/ufs/rootfs failed with error 2; retrying for 10 more seconds
>>>> Mounting from ufs:/dev/ufs/rootfs failed with error 2.
>>>> 
>>>> Loader variables:
>>>> vfs.root.mountfrom=ufs:/dev/ufs/rootfs
>>>> vfs.root.mountfrom.options=rw
>>>> 
>>>> Manual root filesystem specification:
>>>> <fstype>:<device> [options]
>>>>   Mount <device> using filesystem <fstype>
>>>>   and with the specified (optional) option list.
>>>> 
>>>> eg. ufs:/dev/da0s1a
>>>>     zfs:zroot/ROOT/default
>>>>     cd9660:/dev/cd0 ro
>>>>       (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)
>>>> 
>>>> ?               List valid disk boot devices
>>>> .               Yield 1 second (for background tasks)
>>>> <empty line>    Abort manual input
>>>> 
>>>> mountroot> 
>>>> 
>>>> So: same problem.
>>>> 
>>>> For reference, from the media being plugged into a different
>>>> aarch64 FeeBSD machine running main:
>>>> 
>>>> usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device Samsung PSSD T7 Touch (0x04e8:0x4001)
>>>> ugen1.5: <Samsung PSSD T7 Touch> at usbus1
>>>> umass0 on uhub4
>>>> umass0: <Samsung PSSD T7 Touch, class 0/0, rev 3.20/1.00, addr 9> on usbus1
>>>> umass0:  SCSI over Bulk-Only; quirks = 0x0100
>>>> umass0:6:0: Attached to scbus6
>>>> da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
>>>> da0: <Samsung PSSD T7 Touch 0> Fixed Direct Access SPC-4 SCSI device
>>>> da0: Serial Number REDACTED
>>>> da0: 400.000MB/s transfers
>>>> da0: 953869MB (1953525168 512 byte sectors)
>>>> da0: quirks=0x2<NO_6_BYTE>
>>>> 
>>>> The RPi4B is of a vintage that has the "3 GiByte DMA" issue
>>>> present, not that it would be likely to be contributing here.
>>>> 
>>>> 
>>>> So I then tried the sequence using a different type of USB3
>>>> media (also USB2 compatible), placed in a USB3 port to boot:
>>>> 
>>>> usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device OWC Envoy Pro mini (0x1e91:0xa2a5)
>>>> ugen1.5: <OWC Envoy Pro mini> at usbus1
>>>> umass0 on uhub4
>>>> umass0: <OWC Envoy Pro mini, class 0/0, rev 3.00/1.00, addr 11> on usbus1
>>>> umass0:  SCSI over Bulk-Only; quirks = 0x0100
>>>> umass0:6:0: Attached to scbus6
>>>> da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
>>>> da0: <OWC Envoy Pro mini 0> Fixed Direct Access SPC-4 SCSI device
>>>> da0: Serial Number REDACTED
>>>> da0: 400.000MB/s transfers
>>>> da0: 228936MB (468862128 512 byte sectors)
>>>> da0: quirks=0x2<NO_6_BYTE>
>>>> 
>>>> For this media, the sequence worked and booted successfully.
>>>> 
>>>> 
>>>> So the issue is somehow specific to some USB3 media 
>>>> used in the USB3 ports but not to others. "reset failed,
>>>> error=USB_ERR_TIMEOUT" may need more time or better
>>>> recovery if a wider range of boot devices are to be
>>>> supported. May be the T7 Touch needs more than the
>>>> standard amount of time to reset as a USB3 device or
>>>> some such. (I've no clue about the details.)
>>> 
>>> I substituted a 13-STABLE kernel.txz expansion and got the
>>> same result using that kernel. (There are not .img files
>>> for this currently.)
>> 
>> I took a different path of setting up a stable/13 based
>> media, based on my own build context but upgrading a
>> 13.1-RELEASE starting point. This stable/13 variant boots
>> the RPi4B's just fine via the USB3 media that 13.1-RELEASE
>> failed to boot with. I'm not sure what the differences
>> that matter are.
>> 
> 
> Well, the kernel.txz was based on:
> 
> Thu, 30 Jun 2022
> 	• git: 70fd40edb869 - stable/13 - debugnet: Fix an error handling bug in the DDB command tokenizer Mark Johnston
> 
> via:
> 
> Fri, 01 Jul 2022
> 	• New FreeBSD snapshots available: stable/13 (20220701 70fd40edb86) Glen Barber
> 
> while my build was based on:
> 
> stable/13-n251684-815db559eccc-dirty
> (815db559eccc is from Wed, 06 Jul 2022)
> 
> Between those are:
> 
> Fri, 01 Jul 2022
> 	• git: 39cd7aa134df - stable/13 - USB: add quirks to XHCI Bjoern A. Zeeb
> 	• git: 66754c01ff95 - stable/13 - XHCI: clear warm and port reset Bjoern A. Zeeb
> 
> and that last mentions port reset handling explicitly.
> It is, at least, suggestive.

Nope.

Turns out that the problem still exists but my config.txt
content was hiding the problem. Once I have "force_turbo=1"
in config.txt , the problem stops. Without it, the problem
occurs. This is for stable/13 based and releng/13.1 based.
main [so: 14] does not show the problem wither way. (The
boot contexts being U-Boot style, like is normal for
FreeBSD.)

See: https://lists.freebsd.org/archives/freebsd-arm/2022-July/001585.html

>> For reference for the working stable/13 context and
>> other content involved:
>> 
>> # strings -a /boot/efi/start4.elf | grep VC_BUILD_ID_ 
>> VC_BUILD_ID_USER: dom
>> VC_BUILD_ID_TIME: 12:10:40
>> VC_BUILD_ID_VARIANT: start
>> VC_BUILD_ID_TIME: Feb 25 2021
>> VC_BUILD_ID_BRANCH: bcm2711_2
>> VC_BUILD_ID_HOSTNAME: buildbot
>> VC_BUILD_ID_PLATFORM: raspberrypi_linux
>> VC_BUILD_ID_VERSION: 564e5f9b852b23a330b1764bcf0b2d022a20afd0 (clean)
>> 
>> (/boot/efi/armstub8-gic.bin does not have very useful
>> version-string information.)
>> 
>> # strings -a /boot/efi/u-boot.bin | grep 'U-Boot 20'
>> U-Boot 2021.07 (May 12 2022 - 07:00:33 +0000)
>> 
>> Note: To my knowledge, the RPi* firmware, armstub8-gic.bin
>> file, and u-boot.bin file above are all as they were for
>> 13.1-RELEASE's image, other that for 13.1-RELEASE I had
>> also tried adding /boot/efi/timeout and I've left that
>> in place since then and I've added my usual
>> /boot/efi/config.txt material (that had also not made a
>> difference for 13.1-RELEASE).
>> 
>> (/boot/efi/EFI/BOOT/bootaa64.efi and
>> /boot/efi/EFI/freebsd/loader.efi do not have very useful
>> version-string information.)
>> 
>> # uname -apKU # manually line split
>> FreeBSD 13S-ufs 13.1-STABLE FreeBSD 13.1-STABLE #40
>> stable/13-n251684-815db559eccc-dirty: Sat Jul  9 14:02:23 PDT 2022
>> root@CA72_16Gp_ZFS:/usr/obj/BUILDs/13S-CA72-nodbg-clang/usr/13S-src/arm64.aarch64/sys/GENERIC-NODBG-CA72
>> arm64 aarch64 1301505 1301505
>> 
>> 
>> Unfortunately, it has been some time since a
>> 
>> FreeBSD-13.1-STABLE-arm64-aarch64-RPI-*.img
>> 
>> (a snapshot) has managed to build and be distributed.
>> I'm not sure what the result would be like for such at
>> this point.
>> 
>> Once such is available, I may do an experiment based on
>> just a stable/13 snapshot.

Done, using:

FreeBSD-13.1-STABLE-arm64-aarch64-RPI-20220715-831c6b8edda-251792.img

That is when I figured out the "force_turbo=1" issue for config.txt
being involved in hiding the problem.

>>> But using:
>>> 
>>> FreeBSD-14.0-CURRENT-arm64-aarch64-RPI-20220701-9aa02d5120a-256480.img
>>> 
>>> does not have the problem.
>> 

I still do not see the problem in main [so: 14].


===
Mark Millard
marklmi at yahoo.com