[SOLVED] Thanks Mark! Re: USB Flash drive booting from June 22,2023 RPI Snapshot FreeBSD 14.0

On 7/3/23 7:48 AM, Mark Millard wrote:
> On Jul 3, 2023, at 07:16, Mark Millard <[email protected]> wrote:
>
>> On Jul 3, 2023, at 05:31, Fred G. Finster <[email protected]> wrote:
>>
>>> I downloaded this June 22, 2023 version of the  RPI snapshot and burned to a USB flash drive to test on my Raspberry Pi 4B with 8 GB
>>>
>>> https://www.freebsd.org/where/
>>>
>>> https://download.freebsd.org/snapshots/arm64/aarch64/ISO-IMAGES/14.0/
>>>
>>> https://download.freebsd.org/snapshots/arm64/aarch64/ISO-IMAGES/14.0/FreeBSD-14.0-CURRENT-arm64-aarch64-RPI-20230622-b95d2237af40-263748.img.xz
>>>
>>> Is there a recipe that some machine follow to build this image? Can I have that recipe to verify the build process?  What should I do about
>>>
>>> the
>>>
>>> It try to TFTP  boot an image,  I was not setup for yet.  I will google and find some docs and webpages to help me out.
>>>
>>> EFI boot manager: Cannot load any image
>>> Found EFI removable media binary efi/boot/bootaa64.efi
>>> ** Reading file would overwrite reserved memory **
>> That message is from the u-boot.bin stage, before
>> FreeBSD itself is involved but after the RPi*
>> firmwmare. FreeBSD is currently using a vintage of
>> U-Boot that is broken for 8 GiByte RPi4's. FreeBSD
>> has not updated to use the more recent fixed
>> vintaeg of U-Boot. This is a known issue that has
>> been reported on multiple times on the lists.
>>
>> (Note: the message content itself is misleading
>> compared to the actual internal problem in U-Boot.)
>>
>> You can replace the u-boot.bin with a copy of the
>> older one in:
>>
>> http://ftp3.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/13.1/FreeBSD-13.1-RELEASE-arm64-aarch64-RPI.img.xz
>>
>> Similarly if you have other older copies of the
>> arm64-aarch64 RPI u-boot.bin on other of your
>> existing media.
>>
>>> Failed to load 'efi/boot/bootaa64.efi'
>>> No UEFI binary known at 0x00080000
>>> EFI LOAD FAILED: continuing...
>>>
>>>
>>> Yes, I can take a working  bootaa64.efi file and replace this one version  Then see if I can boot my raspberry pi 4B.
>> Replacing bootaa64.efi will not fix anything.
>>
>>> What else do you suggest to do to check or verify.  My system was booting the older version of FreeBSD 14.0 from my
>>>
>>> 500GB SSD.  See my blogpost for details.  Yes, snapshots of 14.0-CURRENT can have problems, so I just wish to share my experience.
>>>
>>> I am afraid I over looked some small simple detail that I did not change or setup.  My apology in advance.
>> You did not overlook anything. FreeBSD is just
>> bundling a bad version of u-boot.bin in its
>> modern images, broken specifically for 8 GiByte
>> RPi4 variants.
>>
>>> I am interested to learn how to use this nice 2023.01 U-BOOT and learn to debug by PXE booting the Raspberry Pi.
>> 2023.01 is not nice for 8 GiByte RPi4 variants.
>> It is broken.
>>
>>> Is there a  website tutorial about using U-BOOT>  to test and learn about your ARM64 Hardware?  RTFM manual
>> You can not make 2023.01 work. Older or newer.
>> Older is easier to get copies of.
>>
>>> Anyone else encountering this particular issue from a snapshot image?
>> Multiple people have hit this issue in the past
>> and the freebsd-arm list history has the records
>> about it.
>>
>>> https://ghostbsd-arm64.blogspot.com/2022/02/booting-500-gb-ssd-on-freebsd-arm64-140.html
>>>
>>> https://ghostbsd-arm64.blogspot.com/2022/09/freebsd-140-compiling-kernel-for.html
>>>
>>>
>>>
>>> Sorry to share this long listing with details:
>>>
>>> Here is a a partial listing of the board dump  bdinfo command issued to 'U-BOOT>'  prompt
>>>
>>> lmb_dump_all:
>>> memory.cnt  = 0x3
>>> memory[0]      [0x0-0x3b2fffff], 0x3b300000 bytes flags: 0
>>> memory[1]      [0x40000000-0xfbffffff], 0xbEFI boot manager: Cannot load any image
>>> Found EFI removable media binary efi/boot/bootaa64.efi
>>> ** Reading file would overwrite reserved memory **
>>> Failed to load 'efi/boot/bootaa64.efi'
>>> No UEFI binary known at 0x00080000
>>> EFI LOAD FAILED: continuing...c000000 bytes flags: 0
>>> memory[2]      [0x100000000-0x1ffffffff], 0x100000000 bytes flags: 0
>>> reserved.cnt  = 0x8
>>> reserved[0]    [0x0-0xfff], 0x00001000 bytes flags: 0
>>> reserved[1]    [0x7ef0000-0x7f0ffff], 0x00020000 bytes flags: 0
>>> reserved[2]    [0x39c28000-0x3b2fffff], 0x016d8000 bytes flags: 0
>>> reserved[3]    [0x3ac3c380-0x3b0fffff], 0x004c3c80 bytes flags: 0
>>> reserved[4]    [0x3ee5c0a0-0x3ee5c164], 0x000000c5 bytes flags: 4
>>> reserved[5]    [0x40000000-0xfbffffff], 0xbc000000 bytes flags: 0
>>> reserved[6]    [0xfe100000-0xfe100fff], 0x00001000 bytes flags: 0
>>> reserved[7]    [0x100000000-0x1ffffffff], 0x100000000 bytes flags: 0
>> Turns out the problem in u-boot.bin it tied to
>> how many of the reserved[?] are actually needed
>> at one stage: it ran out but needed more. They
>> forgot to increase the allowed count when then
>> made other changes that caused usage of more
>> reserved address ranges.
>>
>>> devicetree  = board
>>> arch_number = 0x0000000000000000
>>> TLB addr    = 0x000000003b0f0000
>>> irq_sp      = 0x000000003ac40820
>>> sp start    = 0x000000003ac40820
>>> Early malloc usage: 878 / 2000
>>> U-Boot> boot
>>> Card did not respond to voltage select! : -110
>>> MMC Device 1 not found
>>> no mmc device at slot 1
>>> MMC Device 2 not found
>>> no mmc device at slot 2
>>>
>>> Device 0: Vendor: Verbatim Rev: PMAP Prod: ClickUSB
>>>             Type: Removable Hard Disk
>>>             Capacity: 14776.0 MB = 14.4 GB (30261248 x 512)
>>> ... is now current device
>>> Scanning usb 0:1...
>>> BootOrder not defined
>>> EFI boot manager: Cannot load any image
>>> Found EFI removable media binary efi/boot/bootaa64.efi
>>> ** Reading file would overwrite reserved memory **
>> This is the misleading message that is actually
>> caused by running out of reserved[?] but needing
>> more.
>>
>>> Failed to load 'efi/boot/bootaa64.efi'
>>> No UEFI binary known at 0x00080000
>>> EFI LOAD FAILED: continuing...
>>> BOOTP broadcast 1
>>> DHCP client bound to address 192.168.1.7 (8 ms)
>>> *** Warning: no boot file name; using 'C0A80107.img'
>>> Using ethernet@7d580000 device
>>> TFTP from server 192.168.1.1; our IP address is 192.168.1.7
>>> Filename 'C0A80107.img'.
>>> Load address: 0x1000000
>>> Loading: T T T T T
>>> Abort
>>> missing environment variable: pxeuuid
> Separately from u-boot.bin I also use a modified
> config.txt (at least for my type of boot devices).
> See the comments about use of
> initial_turbo/force_turbo. force_turbo required
> use of over_voltage for reliability. You may
> or may not have the issue that leads to these
> changes (additions).
>
> Part of the reason for various settings is
> FreeBSD's use of older RPi* firmware. That in
> turn means that the RPi* firmware documentation
> is inaccurate about various defaults for the
> vintage that FreeBSD uses. One has to look up
> older documentation to see the accurate defaults
> for what FreeBSD is using.
>
> # more /boot/efi/config.txt
> [all]
> arm_64bit=1
> dtparam=audio=on,i2c_arm=on,spi=on
> dtoverlay=mmc
> dtoverlay=disable-bt
> device_tree_address=0x4000
> kernel=u-boot.bin
>
> [pi4]
> hdmi_safe=1
> armstub=armstub8-gic.bin
>
> [all]
> #
> # Local addition that avoids USB3 SSD boot failures that look like:
> # uhub_reattach_port: port ? reset failed, error=USB_ERR_TIMEOUT
> # uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port ?
> # WARNING, not sufficient for "boot -s": that needs the full force_turbo=1
> initial_turbo=60
> [pi4]
> over_voltage=6
> arm_freq=2000
> sdram_freq_min=3200
> force_turbo=1
> #
> hdmi_safe=0
>
>
> ===
> Mark Millard
> marklmi at yahoo.com
>
Thank you Mark, for config.txt file settings, and for pointing me to 
change U-BOOT.BIN file to older version.  This worked. Thanks again.

I am happy camper today, because of your email help, Mark. FreeBSD is 
booting from the USB Flash drive, from replacing

u-boot.bin file in the June-22 Snapshot image with an older October 21, 
2022 version  568960 Oct 21  2021 u-boot.bin

I modified  the  config.txt  file also.

https://ghostbsd-arm64.blogspot.com/2021/05/audit-your-boot-files-with-md5deep.html 
Audit your boot files with md5deep tool.

 1.

    Calculate hashes of the required directory on Server A:

    |md5deep -r /opt/foo/ > file_hashes.txt|

 2.

    Copy the file |file_hashes.txt| file onto Server B for comparison.

 3.

    Calculate hashes of the required directory on Server B, but taking
    the file hashes from Server A as an input file by using the |-x|
    flag to only show files that are different:

    |md5deep -x file_hashes.txt -r /opt/foo/|

root@Fred_RasPi4B:/mnt/rpifat32 # cp -p ../msdos/u* .
root@Fred_RasPi4B:/mnt/rpifat32 # ls -l u*
-rwxr-xr-x  1 root  wheel  568960 Oct 21  2021 u-boot.bin
-rwxr-xr-x  1 root  wheel  602552 Jun 22 07:08 u-bootJune22_broken.bin
root@Fred_RasPi4B:/mnt/rpifat32 #

root@Fred_RasPi4B:/mnt/msdos # md5 u-boot.bin   # 568960 Oct 21 2021 
u-boot.bin  Working Older Copy
MD5 (u-boot.bin) = 4c2998a9bd108775ada821a56e6362a7    # Working Older 
Copy of u-boot.bin

-rwxr-xr-x  1 root  wheel   602552 Jun 22 07:08 u-boot.bin
root@Fred_RasPi4B:/mnt/rpifat32 # md5 u-boot.bin    #
MD5 (u-boot.bin) = be6c1637a6c775fad754efa723eff516      # Not booting 
June 22 2023 u-boot.bin replaced



### Login Information

login: root
Password:
Jul  3 19:40:40 generic login[1460]: ROOT LOGIN (root) ON ttyu0
FreeBSD 14.0-CURRENT (GENERIC) #0 main-n263748-b95d2237af40: Thu Jun 22 
11:10:50 UTC 2023

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List:        https://www.FreeBSD.org/lists/questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

To change this login announcement, see motd(5).
root@generic:~ # df -h
Filesystem          Size    Used   Avail Capacity  Mounted on
/dev/ufs/rootfs      13G    2.7G    8.8G    24%    /
devfs               1.0K      0B    1.0K     0%    /dev
/dev/msdosfs/EFI     50M     25M     25M    51%    /boot/efi
tmpfs               8.9G    4.0K    8.9G     0%    /tmp
root@generic:~ #




### Boot information for Raspberry Pi 4G with older U-BOOT.BIN file

CPU  0: ARM Cortex-A72 r0p3 affinity:  0
                    Cache Type = <64 byte D-cacheline,64 byte 
I-cacheline,PIPT ICache,64 byte ERG,64 byte CWG>
  Instruction Set Attributes 0 = <CRC32>
  Instruction Set Attributes 1 = <>
  Instruction Set Attributes 2 = <>
          Processor Features 0 = <AdvSIMD,FP,EL3 32,EL2 32,EL1 32,EL0 32>
          Processor Features 1 = <>
       Memory Model Features 0 = <TGran4,TGran64,SNSMem,BigEnd,16bit 
ASID,16TB PA>
Trying to mount root from ufs:/dev/ufs/rootfs [rw]...
       Memory Model Features 1 = <8bit VMID>
       Memory Model Features 2 = <32bit CCIDX,48bit VA>
              Debug Features 0 = <DoubleLock,2 CTX BKPTs,4 Watchpoints,6 
Breakpoints,PMUv3,Debugv8>
              Debug Features 1 = <>
          Auxiliary Features 0 = <>
          Auxiliary Features 1 = <>
AArch32 Instruction Set Attributes 5 = <CRC32,SEVL>
AArch32 Media and VFP Features 0 = <FPRound,FPSqrt,FPDivide,DP 
VFPv3+v4,SP VFPv3+v4,AdvSIMD>
AArch32 Media and VFP Features 1 = <SIMDFMAC,FPHP DP Conv,SIMDHP SP 
Conv,SIMDSP,SIMDInt,SIMDLS,FPDNaN,FPFtZ>
CPU  1: ARM Cortex-A72 r0p3 affinity:  1
CPU  2: ARM Cortex-A72 r0p3 affinity:  2
CPU  3: ARM Cortex-A72 r0p3 affinity:  3
Release APs...done
TCP_ratelimit: Is now initialized
WARNING: WITNESS option enabled, expect reduced performance.
uhub0: 5 ports with 4 removable, self powered
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0
uhub1 on uhub0
uhub1: <vendor 0x2109 USB2.0 Hub, class 9/0, rev 2.10/4.21, addr 1> on 
usbus0
Root mount waiting for: usbus0
uhub1: 4 ports with 4 removable, self powered
Root mount waiting for: usbus0
usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device 
Verbatim ClickUSB (0x18a5:0x025e)
usb_msc_auto_quirk: UQ_MSC_NO_TEST_UNIT_READY set for USB mass storage 
device Verbatim ClickUSB (0x18a5:0x025e)
usb_msc_auto_quirk: UQ_MSC_NO_PREVENT_ALLOW set for USB mass storage 
device Verbatim ClickUSB (0x18a5:0x025e)
ugen0.3: <Verbatim ClickUSB> at usbus0
umass0 on uhub1
umass0: <Verbatim ClickUSB, class 0/0, rev 2.00/1.00, addr 2> on usbus0
umass0:  SCSI over Bulk-Only; quirks = 0x8101
umass0:0:0: Attached to scbus0
da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <Verbatim ClickUSB PMAP> Removable Direct Access SPC-2 SCSI device
da0: Serial Number 8010B1DD0E4223CD
da0: 40.000MB/s transfers
da0: 14776MB (30261248 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
Root mount waiting for: usbus0
ugen0.4: <Logitech Logitech USB Optical Mouse> at usbus0
Warning: no time-of-day clock registered, system time will not be set 
accurately
Dual Console: Serial Primary, Video Secondary
No suitable dump device was found.
Setting hostuuid: 30303031-3030-3030-6633-303631333234.
Setting hostid: 0xc34c5039.
Starting file system checks:
/dev/ufs/rootfs: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ufs/rootfs: clean, 537141 free (325 frags, 67102 blocks, 0.0% 
fragmentation)
Growing root partition to fill device
random: randomdev_wait_until_seeded unblock wait
random: unblocking device.
Adding swap partition
GEOM_PART: da0s2 was automatically resized.
   Use `gpart commit da0s2` to save changes or `gpart undo da0s2` to 
revert them.
da0s2 resized
da0s2b added
da0s2a resized
super-block backups (for fsck_ffs -b #) at:
  11524224, 12804672, 14085120, 15365568, 16646016, 17926464, 19206912,
  20487360, 21767808, 23048256, 24328704, 25609152, 26889600
Metadata value stored on da0s2b.
Done.
Mounting local filesystems:.
Building /boot/kernel/linker.hints
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
Setting hostname: generic.
Setting up harvesting: 
[CALLOUT],[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: .
genet0: link state changed to UP
lo0: link state changed to UP
genet0: link state changed to DOWN
genet0: link state changed to UP
Starting Network: lo0 genet0.
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 
16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
         inet 127.0.0.1 netmask 0xff000000
         inet6 ::1 prefixlen 128
         inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
         groups: lo
         nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
genet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> 
metric 0 mtu 1500
options=68000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
         ether dc:a6:32:d5:3d:76
         inet6 fe80::dea6:32ff:fed5:3d76%genet0 prefixlen 64 scopeid 0x1
         media: Ethernet autoselect (100baseTX <full-duplex>)
         status: active
         nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
Starting devd.
Autoloading module: uhid
Autoloading module: ums
Autoloading module: usbhid
ums0 on uhub1
ums0: <Logitech Logitech USB Optical Mouse, class 0/0, rev 2.00/43.00, 
addr 3> on usbus0
ums0: 3 buttons and [XYZ] coordinates ID=0
Starting dhclient.
DHCPDISCOVER on genet0 to 255.255.255.255 port 67 interval 7
DHCPOFFER from 192.168.1.1
DHCPREQUEST on genet0 to 255.255.255.255 port 67
DHCPACK from 192.168.1.1
bound to 192.168.1.7 -- renewal in 43200 seconds.
Starting ums0 moused.
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Updating motd:.
Updating /var/run/os-release done.
Clearing /tmp (X related).
Creating and/or trimming log files.
Starting syslogd.
Mounting late filesystems:.
Generating RSA host key.
3072 SHA256:fP0ev28UgyUk6jjDbzBUZ2qifwbNwYUWDsXrYyGfxkw root@generic (RSA)
Generating ECDSA host key.
256 SHA256:S0y+ltd+hUIjzTa+VmY9eMpqcwdzR4SKUv/tySOT/nc root@generic (ECDSA)
Generating ED25519 host key.
256 SHA256:4o3+dlAlP1HrXKcU08oBqVvTgwGBkoIXlRdCQAxYS8E root@generic 
(ED25519)
Performing sanity check on sshd configuration.
Starting sshd.
Starting cron.
Starting background file system checks in 60 seconds.

Mon Jul  3 19:26:56 UTC
FreeBSD/arm64 (generic) (ttyu0)

login: