Re: System not booting after upgrade 14.3->14.4 with "ZFS: unsupported feature: com.klarasystems: vdev_zaps_v2"

Hi Mark,

Great instructions! Thanks!

	Julf

On 02/05/2026 10:57 pm, Mark G. wrote:
> On 4/30/26 02:28, Johan Helsingius wrote:
>> Hi all,
>>
>> Was doing an upgrade from 14.3-R7 to 14.4, and after the freebsd-update
>> install, the reboot fails with "ZFS: unsupported feature:
>> com.klarasystems: vdev_zaps_v2"
>>
>> Any idea what that feature is, and how to recover?
>>
>>       Julf
>>
> 
> Here are my notes for when I had to solve the klara issue:
> 
> Disclaimer:  MAKE SURE TO USE YOUR DRIVE NAMES AND PARTITION NUMBERS
> 
> 6.1 Updating Boot Loaders (ZFS)
> 
> I had a failure to boot after an update from FreeBSD 14.2-p3 to 14.2-p4, which had an
> upgraded zfs.ko kernel module. Here is the error and boot failure.
> 
> =====
> Consoles: EFI console
> ZFS: unsupported feature: com.klarasystems:vdev_zaps_v2
> FreeBSD/amd64 EFI loader, Revision 1.1
> ...
> Failed to find bootable partition
> Press any key to interrupt reboot in 3 seconds
> can’t load ’kernel’
> Type ’?’ for a list of commands, ’help’ for more detailed help.
> OK
> =====
> 
> The fixes required boot into single user mode via an USB bootable drive.
> The drive contained FreeBSD 14.2 as an installer with single user
> mode as well.
> 
> Once the USB is booted into single user mode, set the root partition to be mounted
> read-write:
> 
> # mount -u /
> 
> The change from FreeBSD version 13.x to 14.x required updating the boot loaders (in
> various ways) based on zfs-on-root installations. I had failed to do this at the time and
> got bit for it.
> 
> The FreeBSD 14 release notes 1 document this as quoted below.
> 
> [1] https://www.freebsd.org/releases/14.0R/relnotes/
> 
> =======
> "There have been a number of improvements in the boot loaders, and up-
> grading the boot loader on the boot partition is recommended in most cases,
> in particular if the system boots via EFI. If the root is on a ZFS file system,
> updating the boot loader is mandatory if the pool is to be upgraded, and
> the boot loader update must be done first. Note that ZFS pool upgrades are
> not recommended for root file systems in most cases, but updating the boot
> loader can avoid making the system unbootable if the pool is upgraded in
> the future. The bootstrap update procedure depends on the boot method
> (EFI or BIOS), and also on the disk partitioning scheme. The next several
> sections address each in turn.
> 
> Notes for systems that boot via EFI, using either binary or source up-
> grades: There are one or more copies of the boot loader on the MS-DOS
> EFI System Partition (ESP), used by the firmware to boot the kernel. The
> location of the boot loader in use can be determined using the command
> efibootmgr -v. The value displayed for BootCurrent should be the number of
> the current boot configuration used to boot the system. The corresponding
> entry of the output should begin with a + sign, such as
> 
> +Boot0000* FreeBSD HD(1,GPT,f859c46d-19ee-4e40-8975-3ad1ab00ac09,
> 0x800,0x82000)/File(\EFI\freebsd\loader.efi)
> nda0p1:/EFI/freebsd/loader.efi (null)
> 
> The ESP may already be mounted on /boot/efi. Otherwise, the parti-
> tion may be mounted manually, using the partition listed in the efibootmgr
> output (nda0p1 in this case): mount msdosfs /dev/nda0p1 /boot/efi. See
> loader.efi(8) for another example.
> 
> The value in the File field in the efibootmgr -v output, \EFI \freebsd\loader.efi
> in this case, is the MS-DOS name for the boot loader in use on the ESP. If the
> mount point is /boot/efi, this file will translate to /boot/efi/efi/freebsd/loader.efi.
> (Case does not matter on MS-DOSFS file sytems; FreeBSD uses lower case.)
> Another common value for File would be \EFI\boot\bootXXX.efi, where
> XXX is x64 for amd64, aa64 for aarch64, or riscv64 for riscv64; this is the
> default bootstrap if none is configured. Both the configured and default boot
> loaders should be updated by copying from /boot/loader.efi to the correct
> path in /boot/efi.
> 
> For systems that boot via BIOS and use the GPT partition scheme, boot-
> strap upgrades are optional unless a ZFS root pool will be upgraded (which
> is discouraged). Upgrades are only possible if the freebsd-boot partition is
> at least 180K. (512K is now standard.) The device name and the partition
> index can be identified using the command gpart show, such as:
> 
>       =>  34  246162605  ada0 GPT   (224G)
>       34  1024     1 freebsd-boot   (512K)
> 
> When using ZFS, the bootcode can be updated by running the command
> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0. If up-
> dating with a UFS root, the bootcode can be updated by running the com-
> mand gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada0, al-
> though this should not be necessary. Note that the freebsd-boot partition
> size is running up against a hard limit, and this scheme may not be support-
> able in the future. Users of such systems should consider an upgrade to EFI,
> possibly by carving space from an existing swap partition.
> 
> If the system boots via BIOS and uses the MBR partition scheme, or has
> a GPT freebsd-boot partition smaller than 180K, then it is not possible to
> update the bootcode, and therefore to upgrade the root ZFS storage pool."
> =======
> 
> There are two instances that I had to deal with.
> 
> 6.1.1 UEFI Boot Loader
> 
> Most of my newer mini PCs used a UEFI layout for booting and thus have disks
> partitioned similar to this:
> 
> user@think:~ % gpart show
> =>      40  3907029088  nda0  GPT  (1.8T)
>           40      532480     1  efi  (260M)
>       532520        1024     2  freebsd-boot  (512K)
>       533544         984        - free -  (492K)
>       534528    33554432     3  freebsd-swap  (16G)
>     34088960  3872940032     4  freebsd-zfs  (1.8T)
> 3907028992         136        - free -  (68K)
> 
> 
> The partition of interest is the first one (1) with a type of efi.
> This is actually an msdos based partition and can be mounted as such.
> 
> user@think:~ % cat /etc/fstab
> # Device Mountpoint FStype Options Dump Pass#
> /dev/gpt/efiboot0 /boot/efi msdosfs rw 2 2
> 
> The /dev/gpt/efiboot0 device is equivalent to /dev/nda0p1, in this case.
> 
> root@dmz1:~ # mount /boot/efi
> root@dmz1:~ # mount
> ...
> /dev/gpt/efiboot0 on /boot/efi (msdosfs, local)
> 
> To update the boot loader to a FreeBSD 14.x version, the above file system should be
> mounted and its contents will look like:
> 
> user@think:~ % ls -laR /boot/efi
> user@think:~ % ls -laR /boot/efi
> total 25
> drwxr-xr-x   1 root wheel 16384 Dec 31  1979 .
> drwxr-xr-x  15 root wheel    71 Jul  6 20:22 ..
> drwxr-xr-x   1 root wheel 16384 Feb 12 18:49 efi
> 
> /boot/efi/efi:
> total 48
> drwxr-xr-x  1 root wheel 16384 Feb 12 18:49 .
> drwxr-xr-x  1 root wheel 16384 Dec 31  1979 ..
> drwxr-xr-x  1 root wheel 16384 Feb 12 18:49 boot
> drwxr-xr-x  1 root wheel 16384 Feb 12 18:49 freebsd
> 
> /boot/efi/efi/boot:
> total 688
> drwxr-xr-x  1 root wheel  16384 Feb 12 18:49 .
> drwxr-xr-x  1 root wheel  16384 Feb 12 18:49 ..
> -rwxr-xr-x  1 root wheel 660992 Feb 12 18:49 bootx64.efi
> 
> /boot/efi/efi/freebsd:
> total 688
> drwxr-xr-x  1 root wheel  16384 Feb 12 18:49 .
> drwxr-xr-x  1 root wheel  16384 Feb 12 18:49 ..
> -rwxr-xr-x  1 root wheel 660992 Feb 12 18:49 loader.efi
> 
> We have to update both paths:
> 
> /boot/efi/efi/boot/bootx64.efi
> /boot/efi/efi/freebsd/loader.efi
> 
> with the file found in /boot/loader.efi. To whit (make some copies of the old ones
> first):
> 
> root@metal:~ # cp /boot/efi/efi/freebsd/loader.efi loader.efi.from13.x
> root@metal:~ # cp /boot/efi/efi/boot/bootx64.efi bootx64.efi.from13.x
> root@metal:~ # cp /boot/loader.efi /boot/efi/efi/freebsd/loader.efi
> root@metal:~ # cp /boot/loader.efi /boot/efi/efi/boot/bootx64.efi
> 
> 
> There is a command called efibootmgr which can supply some information:
> 
> root@dmz1:~ # efibootmgr -v
> Boot to FW : false
> BootCurrent: 0000
> Timeout    : 1 seconds
> BootOrder  : 0000, 0001
> +Boot0000* FreeBSD HD(1,GPT,9ccacd8f-5c7a-11f0-bfc4-5847ca753c56,
>     0x28,0x82000)/File(\EFI\FREEBSD\LOADER.EFI)
>     ada0p1:/EFI/FREEBSD/LOADER.EFI (null)
> 
> Boot0001* UEFI OS HD(1,GPT,9ccacd8f-5c7a-11f0-bfc4-5847ca753c56,
>     0x28,0x82000)/File(\EFI\BOOT\BOOTX64.EFI)
>     ada0p1:/EFI/BOOT/BOOTX64.EFI (null)
> 
> Boot0001* UEFI OS HD(1,GPT,9ccacd8f-5c7a-11f0-bfc4-5847ca753c56,
> 0x28,0x82000)/File(\EFI\BOOT\BOOTX64.EFI)
> ada0p1:/EFI/BOOT/BOOTX64.EFI (null)
> 
> The plus (+) sign indicates the file currently in use.
> 
> 
> 6.1.2 BIOS GPT Boot Loader
> 
> The second variant I have is an old Lenovo non-UEFI BIOS based system which has a
> partition scheme similar to this:
> 
> root@gate:~ % gpart show
> =>     40  468862048  ada0  GPT  (224G)
>          40       1024     1  freebsd-boot  (512K)
>        1064        984        - free -  (492K)
>        2048    4194304     2  freebsd-swap  (2.0G)
>     4196352  464664576     3  freebsd-zfs  (222G)
> 468860928       1160        - free -  (580K)
> 
> 
> Note the lack of an efi partition.
> 
> The update here is:
> # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
> 
> If drives are mirrored and have the same gpart setup, use the same command for each
> drive: ada1, ada2, and so on.
> 
> 
> 
> 
> 
> 
>