Re: rpi4 zfs-on-root boot-to-usb3 [Example sequence that lead to booting zfs-on-root under GPT partitioning]

On 2021-May-6, at 12:12, Mark Millard <marklmi at yahoo.com> wrote:

> On 2021-May-6, at 05:55, Mark Millard <marklmi at yahoo.com> wrote:
> 
>> On 2021-May-6, at 04:12, tech-lists <tech-lists at zyxst.net> wrote:
>> 
>>> How can zfs-on-root boot-to-usb3 on rpi4 be accomplished?
>>> 
>>> I've tried bsdinstall from a mmcsd-booted rpi4 but there seems to be
>>> problems with it that I can't work around. What's really needed is an
>>> installer, but these aren't made for arm64.aarch64 rpi4 from what I can
>>> see (I'm no expert though, it's entirely feasible i've missed
>>> something).
>>> 
>>> Maybe one way of doing it would be to have a usb key (as ufs2) for the
>>> system to boot on, then have /home /usr/obj and other larger dirs on the
>>> usb3-zfs disk.
>> 
>> I used bsdinstall from booting a releng/13.0's release/13.0.0.0
>> microsd card in a 8 GiBYte RPi4B to produce the:
>> . . .
> 
> Various details shown will just be my specific
> choices. (The RPi4B's that I have access to have
> the 2021-Apr-29 default(/critical) EEPROM image.)
> 
> Taking notes as I go (and readjusting as I
> progress and figure things out, eliminating
> failing attempts as well) . . .
> 
> Booting based on a microsd card with releng/13.0 's
> release/13.0.0 as its basis. The context has a
> working network with internet access.
> 
> # uname -apKU
> FreeBSD generic 13.0-RELEASE FreeBSD 13.0-RELEASE #0 releng/13.0-n244733-ea31abc261f: Fri Apr  9 06:06:55 UTC 2021     [email protected]:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC  arm64 aarch64 1300139 1300139
> 
> Plug in USB3 SSD. Ends up as da0.
> 
> # /bin/sh  # Just for my familiarity
> # set -o vi
> 
> # mkdir -p /usr/freebsd-dist
> # cd /usr/freebsd-dist
> # fetch http://ftp3.freebsd.org/pub/FreeBSD/releases/arm64/13.0-RELEASE/MANIFEST
> MANIFEST                                               782  B 6147 kBps    00s
> # cd ~
> 
> # bsdinstall
> 
> Continue with default keymap : Select
> 
> Enter hostname as ZFStest : OK
> 
> [*] base-dbg
> [*] kernel-dbg
> [ ] ports
> [ ] src
> [*] tests
> then: OK
> 
> (Note: I use git for src and ports.)
> 
> Main Site : OK
> 
> Auto (ZFS) : OK
> 
> Pool Name : Select
> 
> Enter name for zpool ztstp : OK
> 
> Swap Size : Select
> 
> Enter swap size 24g : OK
> 
> Proceed with Installation : Select
> 
> Stripe - No Redundancy : OK
> 
> [*] da0 : OK
> 
> Last Chance for da0 : YES
> 
> Downloads . . .
> Extracts . . .
> 
> New Password: . . .
> Retype New Password: . . .
> 
> genet0 : OK
> 
> configure IPv4 : YES
> configure DHCP : YES
> configure IPv6 : YES
> try SLAAC : YES
> Resolver Configuration : OK
> 
> time is UTC? : YES
> 
> America : OK
> United States of America : OK
> Pacific : OK
> Does PDT look reasonable? : Yes
> May 2021 6 : Set Date
> 11 07 00 : Set Time
> 
> [ ] local_unbound
> [*] sshd
> [ ] moused
> [ ] ntpdate
> [*] ntpd
> [*] powerd
> [*] dumpdev
> Then : OK
> 
> No hardening options enabled : OK
> 
> Add uses? : Yes
> . . . details omitted . . .
> OK ? yes
> Add another user? no
> 
> Handbook : OK
> [*] en : OK
> 
> Apply configuration and exit installer : OK
> open a shell : No

I suggested an inappropriate later stage if one
wants to try the same vintage of rpi-firmware
and u-boot as releng/13.0 itself uses. One
can copy over materials before the shutdown
by getting them from /boot/msdos/ .

Note that you might not want to copy over
/boot/msdos/EFI/... as the install will
already have such.

So, something like:

# mount -onoatime -tmsdosfs /dev/da0p1 /mnt
# cp -aRx /boot/msdos/[LRa-z]* /mnt/
# umount /mnt

then do the shutdown and remove the microsd card.

> # shutdown -p now

The following is only if one did not copy from
/boot/msdosfs/ or one needs more recent
materials, such as U-Boot for *C0T revision
processors.

> At this point it still can not boot an RPi4B
> for lack of rpi firmware and U-Boot.
> 
> I have such available on other machine based
> on the latest ports instead of quarterly. There
> are RPi4B's in the world that need the more
> modern U-Boot compared to the quarterly that
> releng/13.0 is tied to by default. But you
> likely could install rpi-firmware and
> u-boot-rpi-arm64 on the microsd card and
> then copy over materials from there.

The above is dumb suggestion unless newer
material is needed. See before the shutdown
above.

The below is me getting more recent materials
(well, U-Boot) from another context. You might
not do similarly.

> In my context . . .
> 
> # gpart show -p da1
> =>       40  468862048    da1  GPT  (224G)
>         40     532480  da1p1  efi  (260M)
>     532520       2008         - free -  (1.0M)
>     534528   50331648  da1p2  freebsd-swap  (24G)
>   50866176  417994752  da1p3  freebsd-zfs  (199G)
>  468860928       1160         - free -  (580K)
> 
> # mount -onoatime -tmsdosfs /dev/da1p1 /mnt
> # cp -aRx /usr/local/share/rpi-firmware/* /mnt/
> # cp -aRx /mnt/config_arm64.txt /mnt/config.txt 
> # cp -aRx /usr/local/share/u-boot/u-boot-rpi-arm64/u-boot.bin /mnt/
> # umount /mnt

(Note: It is possible to be more selective in
what to copy over. I did not complicate the
sequence with such handling.)


Back to the normal flow on the RPi4B given
appropriate RPi* materials copied over to the
msdos file system . . .

> Back to the RPi4B, no microsd card but plugging in the
> USB3 SSD and booting and logging in:
> 
> Dec 31 16:00:48 ZFStest login[1351]: ROOT LOGIN (root) ON ttyu0
> FreeBSD 13.0-RELEASE (GENERIC) #0 releng/13.0-n244733-ea31abc261f: Fri Apr  9 03:54:53 UTC 2021
> 
> # gpart show -p
> =>       40  468862048    da0  GPT  (224G)
>         40     532480  da0p1  efi  (260M)
>     532520       2008         - free -  (1.0M)
>     534528   50331648  da0p2  freebsd-swap  (24G)
>   50866176  417994752  da0p3  freebsd-zfs  (199G)
>  468860928       1160         - free -  (580K)
> 
> # uname -apKU
> FreeBSD ZFStest 13.0-RELEASE FreeBSD 13.0-RELEASE #0 releng/13.0-n244733-ea31abc261f: Fri Apr  9 03:54:53 UTC 2021     [email protected]:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC  arm64 aarch64 1300139 1300139
> 
> I end up adding to /etc/rc.conf:
> 
> #
> ntpd_sync_on_start="YES"
> ntpd_user="root"
> 
> The first boot's time will be messed up for
> lack of the ntpd_sync_on_start="YES" .
> 
> # shutdown -r now
> 
> After login:
> 
> # ls -Tld /etc/rc.conf
> -rw-r--r--  1 root  wheel  279 Dec 31 16:12:37 1969 /etc/rc.conf
> # touch /etc/rc.conf
> 
> There are other files around with such an odd timestamp.
> 
> # zpool list
> NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
> ztstp   199G  1.09G   198G        -         -     0%     0%  1.00x    ONLINE  -
> 
> # zfs list
> NAME                 USED  AVAIL     REFER  MOUNTPOINT
> ztstp               1.09G   192G       96K  /ztstp
> ztstp/ROOT          1.08G   192G       96K  none
> ztstp/ROOT/default  1.08G   192G     1.08G  /
> ztstp/tmp             96K   192G       96K  /tmp
> ztstp/usr            416K   192G       96K  /usr
> ztstp/usr/home       128K   192G      128K  /usr/home
> ztstp/usr/ports       96K   192G       96K  /usr/ports
> ztstp/usr/src         96K   192G       96K  /usr/src
> ztstp/var            680K   192G       96K  /var
> ztstp/var/audit       96K   192G       96K  /var/audit
> ztstp/var/crash       96K   192G       96K  /var/crash
> ztstp/var/log        200K   192G      200K  /var/log
> ztstp/var/mail        96K   192G       96K  /var/mail
> ztstp/var/tmp         96K   192G       96K  /var/tmp
> 
> # more /etc/sysctl.conf 
> # $FreeBSD$
> #
> #  This file is read when going to multi-user and its contents piped thru
> #  ``sysctl'' to adjust kernel values.  ``man 5 sysctl.conf'' for details.
> #
> 
> # Uncomment this to prevent users from seeing information about processes that
> # are being run under another UID.
> #security.bsd.see_other_uids=0
> vfs.zfs.min_auto_ashift=12
> 
> I'll note that in:
> 
> # more /boot/efi/config.txt 
> [all]
> arm_64bit=1
> dtparam=audio=on,i2c_arm=on,spi=on
> dtoverlay=mmc
> dtoverlay=disable-bt
> device_tree_address=0x4000
> kernel=u-boot.bin
> 
> [pi4]
> hdmi_safe=1
> armstub=armstub8-gic.bin
> 
> The hdmi)safe=1 line restricts the HDMI display
> resolution/scaling. Any of the following
> replacements for that line will avoid that but
> in some contexts one could end up in a "blind
> display" context instead, which is why hdmi_safe
> is enabled by default.
> 
> hdmi_safe=0
> or:
> #hdmi_safe=1
> or just delete the line.
> 


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

_______________________________________________
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[email protected]"