Re: How to boot FreeBSD for arm 32 bit as DomU with u-boot on my ARM Chromebook

Hello to everyone.

I have compiled the needed u-boot.bin from scratch using this procedure :

# git clone https://github.com/u-boot/u-boot.git
# cd u-boot
# ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make snow_defconfig : this
line generates the file .config
# nano .config and I've added these parameters :

CONFIG_ARMV7_NONSEC=n
CONFIG_EFI_GRUB_ARM32_WORKAROUND=y

the uboot-bin file is generated with this command :

# ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make

At this point,I took a look inside the .config file and I saw that the
parameter "CONFIG_ARMV7_NONSEC=n" has been removed. So,for some reason,it
is not accepted and this could be a problem....

These are the xen config files that I've used :

nano freebsd.cfg

name="test"
kernel="u-boot.bin"
extra = "console=hvc0"
memory=256
vcpus=1
disk = [ 'FreeBSD-13.2-RELEASE-armv7.img,raw,xvda' ]

nano start-freebsd

xl create freebsd.cfg
xl console freebsd

This is what happens when I launch the vm :

# ./start-freebsd

Parsing config from freebsd.cfg
xc: error: panic: xg_dom_core.c:689: xc_dom_find_loader: no loader found:
Invalid kernel
libxl: error: libxl_dom.c:571:libxl__build_dom: xc_dom_parse_image failed
libxl: error: libxl_create.c:1640:domcreate_rebuild_done: Domain 1:cannot
(re-)build domain: -3
libxl: error: libxl_domain.c:1183:libxl__destroy_domid: Domain
1:Non-existent domain
libxl: error: libxl_domain.c:1137:domain_destroy_callback: Domain 1:Unable
to destroy guest
libxl: error: libxl_domain.c:1064:domain_destroy_cb: Domain 1:Destruction
of domain failed
freebsd is an invalid domain identifier (rc=-6)


On Mon, Dec 18, 2023 at 12:39 PM Mario Marietto <marietto2008@gmailcom>
wrote:

> So,ok,I should have said "the second u-boot" ; since the first u-boot
> binary is the "u-boot binary located in the RO memory" of the Chromebook".
> Sorry for the confusion.
>
> On Mon, Dec 18, 2023 at 12:35 PM Mario Marietto <[email protected]>
> wrote:
>
>> ---> There are no specific options in u-boot devoted to FreeBSD
>>
>> This is an important factor. So,what about if,instead of compiling a new
>> version of u-boot on the partition 2,I will recompile the u-boot customized
>> version created by the virtual open system in 2014,that should be installed
>> on the first partition ? It could work if there are no differences between
>> the u-boot that should boot Linux and the u-boot that should boot FreeBSD.
>>
>> Can you give a look at the u-boot source code created by virtual open
>> systems ? You can find it on my google drive :
>>
>>
>> https://drive.google.com/file/d/1eAaZMfd6CU0xiqQfH7sq5wGVzzO09BRm/view?usp=sharing
>>
>> I need to understand if I can recompile it without problem so that it can
>> satisfy my needs (the ability of the file u-boot.bin to boot FreeBSD as
>> domU under Xen,as explained by Stefano Stabellini,the xen developer that
>> suggested to me what I could do to have FreeBSD virtualized under Xen on my
>> Arm Chromebook) ; otherwise the risk is to find later problems that will
>> make me troubles and that I will not able to fix.
>>
>> I gave a look at the virtual open system u-boot and I didn't see any arndale_defconfig
>> inside. So,If I have understood correctly,I should put that file inside the
>> root of the u-boot source code,let's say here :
>>
>> marietto:/home/marietto/Desktop/Files/u-boot_FreeBSD/u-boot-vos # ls
>>
>> .checkpatch.conf        README                  doc
>>                     net
>> .git                    api                     drivers
>>                 onenand_ipl
>> .gitignore              arch                    dts
>>                     post
>> COPYING                 board                   examples
>> rules.mk
>> CREDITS                 boards.cfg              fs
>>                      scripts
>> MAINTAINERS             common                  include
>>                 snapshot.commit
>> MAKEALL                 config.mk               lib
>>                     spl
>> Makefile                cros                    mkconfig
>>                test
>> PRESUBMIT.cfg           disk                    nand_spl
>>                tools
>>
>> and I should do : make and make install ? and the file I need,u-boot.bin
>> will be generated ?
>>
>> I didn't find any pre made configuration file inside :
>>
>> u-boot-vos # find . -type f -name "exynos*"
>>
>> ./include/exynos-fb.h
>> ./include/configs/exynos5-common.h
>> ./doc/device-tree-bindings/spi/exynos-spi.txt
>> ./doc/device-tree-bindings/usb/exynos-usb.txt
>> ./drivers/power/exynos-tmu.c
>> ./drivers/power/exynos-cpufreq.c
>> ./drivers/video/exynos-fb.c
>> ./drivers/spi/exynos_spi.c
>> ./board/samsung/dts/exynos5250-spring.dts
>> ./board/samsung/dts/exynos5250-smdk5250.dts
>> ./board/samsung/dts/exynos5250-snow.dts
>> ./board/samsung/dts/exynos5250-daisy.dts
>> ./arch/arm/include/asm/arch-exynos5/exynos-cpufreq.h
>> ./arch/arm/include/asm/arch-exynos5/exynos-tmu.h
>> ./arch/arm/dts/exynos5250.dtsi
>> ./arch/arm/dts/exynos-periph-id.dtsi
>> ./arch/arm/cpu/armv7/exynos5/exynos_cache.c
>>
>> u-boot-vos # find . -type f -name "arndale*"
>>
>> For sure I can't use a newer version of u-boot because otherwise the
>> patches needed to bypass the bootloader protections of the Arm Chromebook
>> (such as a lot of different patches needed to boot correctly Linux) will be
>> broken ; anyway,since it works,I don't need to use an updated version of
>> u-boot.
>>
>> ----> As per my experience, you have to respect these two options,
>> compiling u-boot for FreeBSD:
>> https://github.com/freebsd/freebsd-ports/blob/main/sysutils/u-boot-master/files/FreeBSD_Fragment
>>
>> It says that I should use these parameters :
>>
>> CONFIG_ARMV7_NONSEC=n
>> CONFIG_EFI_GRUB_ARM32_WORKAROUND=y
>>
>> These are the parameters used to configure a Linux kernel. I don't
>> understand what's the relation between the compilation of a linux kernel
>> and u-boot. In the past I tried to recompile u-boot,but I didn't have the
>> need to set up those parameters,so I don't know how to do it (but I know
>> how to recompile a Linux kernel).
>>
>>
>> ---> I'm not sure that I'm getting you right, as I don't understand what
>> you mean under "the first u-boot".
>>
>>
>> I'm talking about first u-boot because the whole procedure to boot Linux
>> on the ARM Chromebook,that's explained here :
>>
>> http://www.virtualopensystems.com/en/solutions/guides/kvm-on-chromebook/
>>
>>
>> at some point they say :
>>
>>
>> To be able to run KVM on ARM platforms, the kernel has to be booted in
>> hypervisor mode. Because of this relatively recent requirement (due to the
>> introduction of the virtualization extensions), up until now all booting
>> methods would boot the kernel in the standard Supervisor mode.
>>
>> For the ARM Chromebook the default boot procedure doesn't allow us to
>> boot in hypervisor mode. Although the laptop's boot mechanism is based on
>> the frequently used u-boot, the binary is located in RO memory.
>> Fortunately, a chained u-boot mechanism can be used (i.e. starting another
>> u-boot after the original). We can then enter hypervisor mode from our
>> custom iteration of u-boot and subsequently load our kernel and userspace.
>>
>> So,the first u-boot is the u-boot provided by virtual open systems,that's
>> able to chainload the "u-boot binary located in RO memory" , that does not
>> boot Chrome OS in hypervisor mode. We don't need it if we want to boot
>> Linux with kvm or xen enabled.
>>
>> On Sun, Dec 17, 2023 at 1:28 AM Stanislav Silnicki <
>> [email protected]> wrote:
>>
>>> I'm not an expert in the topic, I only know, that ARM has divided
>>> hardware into two worlds - Secure and Not-So, strictly limiting any
>>> software, running in non-secure world with access to functions and
>>> resources.
>>> https://developer.arm.com/documentation/den0013/d/Security/TrustZone-hardware-architecture?lang=en
>>>
>>> I'm not sure, that I'm getting you right, as I don't understand what you
>>> mean under "the first u-boot".
>>>
>>> As I understand, virtualization (HYP) is running in non-secure world (
>>> https://developer.arm.com/documentation/ddi0406/c/System-Level-Architecture/The-System-Level-Programmers--Model/The-Virtualization-Extensions),
>>> so my guess (only guess!!!), virtualization software has to prepare
>>> (configure) HW platform in the way, that FreeBSD kernel will not lack any
>>> resources, required to configure MPU, VA, etc.
>>> So, if you lucky to boot virtualizer, which is aware of target OS, that
>>> maybe you can boot the kernel. Although, I doubt, that you need to boot
>>> 'second' u-boot to boot the kernel - there is simply ubldr, which you can
>>> hook somehow from virtualizer....
>>>
>>> Stan
>>>
>>>
>>>
>>> Mario Marietto wrote:
>>>
>>>
>>> ---> As I understand, it makes sure that u-boot keeps in secure mode
>>> during boot and passes control to ubldr, which boots FreeBSD kernel, in
>>> that mode.
>>>
>>> Can you elaborate your sentence more ? I know that the bootloader secure
>>> mode is bypassed by the virtual open systems u-boot. Are you saying that
>>> when the control passes to the second u-boot,it will happen in secure
>>> mode,so that the bypass that happened loading the first u-boot,is annulled
>>> ? If this is true,maybe can I boot FreeBSD using the virtual-open-system
>>> custom u-boot ? Is this compatible with FreeBSD ? Where can I find the
>>> u-boot.bin that the xen developer talked about ? thanks bro'.
>>>
>>>
>>>
>>> On Sun, Dec 17, 2023 at 12:35 AM Stanislav Silnicki <
>>> [email protected]> wrote:
>>>
>>>> Hi Mario,
>>>>
>>>> U-Boot  beast is hiding in this den:
>>>> https://source.denx.de/u-boot/u-boot.git
>>>> I took a brief look at your post and it seems to me, that option
>>>> CONFIG_CMO_BY_VA_ONLY is irrelevant to your target armv7 32 bit
>>>> platform:
>>>> https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/cpu/armv8/Kconfig?ref_type=heads#L3
>>>>
>>>> As for compiling the u-boot, it is a doable task, given that you
>>>> understand what you are doing. There are no specific options in u-boot
>>>> devoted to FreeBSD. It is a boot loader, whose mission to make basic
>>>> hardware initialization, read you kernel file from some media into RAM and
>>>> then pass it control.
>>>>
>>>> Basically, you can grab some defconfig, prepared for any other
>>>> Exynos5250 based board  (say, this one:
>>>> https://source.denx.de/u-boot/u-boot/-/blob/master/configs/arndale_defconfig?ref_type=heads)
>>>> and adopt it somehow.
>>>>
>>>> As per my experience, you have to respect these two options, compiling
>>>> u-boot for FreeBSD:
>>>> https://github.com/freebsd/freebsd-ports/blob/main/sysutils/u-boot-master/files/FreeBSD_Fragment
>>>>
>>>> As I understand, it makes sure, that u-boot keeps in secure mode during
>>>> boot and passes control to ubldr, which boots FreBSD kernel, in that mode.
>>>> Otherwise, there a lot of surprises you may realize.
>>>>
>>>> Hope, this will help to progress you tasks
>>>> Stan
>>>>
>>>> Mario Marietto wrote:
>>>>
>>>>
>>>> Hello.
>>>>
>>>> I'm trying to boot FreeBSD for arm32 bit as DomU on my ARM Chromebook.
>>>> Basically there are two ways to accomplish this task :
>>>>
>>>> 1) to write a patch that allows the FreeBSD kernel to boot as a zImage
>>>> file. This could be accomplished applying this patch to a specific file
>>>> that's on the source code of FreeBSD :
>>>>
>>>>
>>>>
>>>> https://xenbits.xen.org/gitweb/?p=p...8;hb=0782e25d98cc1391472717035f986c979edef0c9
>>>> <https://xenbits.xen.org/gitweb/?p=people/julieng/freebsd.git;a=blob_plain;f=sys/arm/conf/XENVIRT;h=ff6433392e6fc2d5fa7c1ad1f7c40155003af9a8;hb=0782e25d98cc1391472717035f986c979edef0c9>
>>>>
>>>>
>>>> This patch was written by Julien Grall a lot of time ago and now it
>>>> does not work anymore. This is the reason :
>>>>
>>>>
>>>> It appears FreeBSD-CURRENT removed the last step converting the kernel
>>>> file to kernel.bin. The patch can be readily rebased, but without
>>>> kernel.bin that doesn't do too much.
>>>>
>>>>
>>>>
>>>> So,without a rebase of that patch the first option is not applicable.
>>>> And I'm not able to fix it.
>>>>
>>>> 2) booting FreeBSD using U-Boot,as explained to me by a xen developer :
>>>>
>>>>
>>>> I was trying to explain why and how Julien's patch works so that you
>>>> could be the one to re-do something similar or fix the patch on the FreeBSD
>>>> kernel that you are working with. I am happy to help review and write
>>>> patches but I don't work with the FreeBSD kernel so I wouldn't be able to
>>>> help you quickly. However, I might have a suggestion. Do you know if
>>>> FreeBSD can be booted by U-Boot ? Because U-Boot definitely boots as Xen on
>>>> ARM guest firmware/bootloader. You should be able to build U-Boot and use
>>>> the U-Boot binary as Xen guest kernel, then U-Boot could load FreeBSD from
>>>> disk or network and start it. For instance as domU config file:
>>>>
>>>> kernel="/home/petalinux/u-boot.bin"
>>>> disk = [ '/home/petalinux/test.img,raw,xvda' ]
>>>>
>>>> I know it is important to build u-boot with the following config to
>>>> make it work on Xen.
>>>>
>>>> CONFIG_CMO_BY_VA_ONLY=y
>>>>
>>>>
>>>>
>>>> This option seems more doable to me according to my knowledge. But I
>>>> need to understand how to do it.
>>>>
>>>> Well,let's say that on the ARM Chromebook I'm forced to use and install
>>>> a customized version of u-boot,created by virtual open systems,because it
>>>> is the only one that allows bypassing its bootloader protection. You can
>>>> find more information here :
>>>>
>>>>
>>>> http://www.virtualopensystems.com/en/solutions/guides/kvm-on-chromebook/?vos=tech
>>>>
>>>> This is the relevant section to read :
>>>>
>>>>
>>>> Bootloader :
>>>>
>>>> If you wish to skip this chapter you can download a pre-compiled binary
>>>> of the bootloader:
>>>>
>>>>
>>>> $ wget
>>>> http://www.virtualopensystems.com/downloads/guides/kvm_on_chromebook/nv_u-boot-snow.kpart
>>>>
>>>>
>>>> To be able to run KVM on ARM platforms, the kernel has to be booted in
>>>> hypervisor mode. Because of this relatively recent requirement (due to the
>>>> introduction of the virtualization extensions), up until now all booting
>>>> methods would boot the kernel in the standard Supervisor mode. For the ARM
>>>> Chromebook the default boot procedure doesn't allow us to boot in
>>>> hypervisor mode. Although the laptop's boot mechanism is based on the
>>>> frequently used u-boot, the binary is located in RO memory. Fortunately, a
>>>> chained u-boot mechanism can be used (i.e. starting another u-boot after
>>>> the original). We can then enter hypervisor mode from our custom iteration
>>>> of u-boot and subsequently load our kernel and userspace.
>>>>
>>>> Checkout the needed u-boot code :
>>>>
>>>>
>>>> $ git clone git://github.com/virtualopensystems/u-boot.git$ cd u-boot$
>>>> ./scripts/build.sh
>>>>
>>>>
>>>> If successful, a message about how to copy the bootloader on the USB
>>>> flash disk or SD card will appear. We will use it later when preparing the
>>>> boot medium to start our system. If you have followed the Setting up the
>>>> boot medium chapter and you have a prepared boot device, then you can
>>>> update u-boot by running :
>>>>
>>>>
>>>> $ sudo dd if=nv_uboot-snow.kpart of=/dev/sdX1
>>>>
>>>>
>>>>
>>>> so,the needed u-boot that we must use should be installed on the first
>>>> partition of the sd card.
>>>>
>>>> There is another relevant section to read :
>>>>
>>>>
>>>> Setting up the boot medium
>>>>
>>>> Now it is time to copy all the relevant files that we created in the
>>>> previous chapters,and use them to boot Chromebook with a different kernel
>>>> and OS. In all these examples the device /dev/sdX is used. Take extra care
>>>> to change the examples to the device that you have attached. Insert the
>>>> boot medium on your workstation and carefully execute the following step.
>>>> First we need to properly format the boot medium.
>>>>
>>>> In the uboot source directory :
>>>>
>>>>
>>>> $ sudo ./scripts/sdcard.sh /dev/sdX
>>>>
>>>>
>>>> This will erase all data and create 4 partitions in the medium, along
>>>> with copying the u-boot binary to the first partition:
>>>>
>>>>
>>>> Partition 1 = ChromeOS signed binary (V.O.S chained u-boot)
>>>> Partition 2 = not used
>>>> Partition 3 = EXT2 partition for u-boot files (uImage and
>>>> exynos5250-snow.dtb)
>>>> Partition 4 = EXT4 partition for userspace files
>>>>
>>>>
>>>> With u-boot being copied, next is the kernel image and DTB file. From
>>>> the kernel source execute :
>>>>
>>>>
>>>> $ mkdir ../mnt/
>>>> $ sudo mount /dev/sdX3 ../mnt/
>>>> $ sudo cp arch/arm/boot/uImage ../mnt/
>>>> $ sudo cp arch/arm/boot/dts/exynos5250-snow.dtb ../mnt/
>>>> $ sudo umount /dev/sdX3
>>>>
>>>>
>>>> Finally, we have to copy the Ubuntu userspace filesystem that we
>>>> created earlier:
>>>>
>>>>
>>>> $ sudo mount /dev/sdX4 mnt/$ sudo cp -a ./precise/* mnt/$ sudo umount
>>>> /dev/sdX4
>>>>
>>>>
>>>>
>>>> Now,my idea is to chainload the already chain loaded u-boot created by
>>>> V.O.S to the new u-boot that we need for booting FreeBSD and that can be
>>>> installed in the partition n.2,as shown in this scheme,because it is not
>>>> used :
>>>>
>>>>
>>>> Partition 1 = ChromeOS signed binary (V.O.S chained u-boot)
>>>> Partition 2 = not used (maybe we can install the u-boot for arm 32
>>>> bit,compatible with FreeBSD on this partition)
>>>> Partition 3 = EXT2 partition for u-boot files (uImage and
>>>> exynos5250-snow.dtb)
>>>> Partition 4 = EXT4 partition for userspace files
>>>>
>>>>
>>>> Take in consideration that default boot string is hardcoded here,in the
>>>> snow.h file of the custom u-boot created by VOS :
>>>>
>>>>
>>>>
>>>> https://github.com/virtualopensyste...18a39b6c177dff58a/include/configs/snow.h#L101
>>>> <https://github.com/virtualopensystems/u-boot/blob/eecfeb578e296ef3b739ac918a39b6c177dff58a/include/configs/snow.h#L101>
>>>>
>>>>
>>>> and it needs to be recompiled because it should point to the partition
>>>> n.2,where I will install the u-boot files as explained here :
>>>>
>>>>
>>>> https://wiki.freebsd.org/arm/Chromebook
>>>>
>>>>
>>>> I have some questions to ask before I start working on this.
>>>>
>>>> 1) The xen developer said :
>>>>
>>>>
>>>> You should be able to build U-Boot and use the U-Boot binary as Xen
>>>> guest kernel...
>>>>
>>>>
>>>>
>>>> where is the u-boot binary,according to this document ?
>>>>
>>>> https://wiki.freebsd.org/arm/Chromebook
>>>>
>>>> I don't see it.
>>>>
>>>>
>>>> 2) where is the source code of the file that I can get here :
>>>>
>>>>
>>>> http://commondatastorage.googleapis.com/chromeos-localmirror/distfiles/nv_uboot-snow-simplefb.kpart.bz2
>>>>
>>>> I need the source code if I want to recompile u-boot so that it can
>>>> point to the partition 4.
>>>>
>>>> Maybe it can be found on this link :
>>>>
>>>> http://linux-exynos.org/dist/chromebook/nv_uboot/
>>>>
>>>> but it can't be opened....
>>>>
>>>>
>>>> 3) in this specific scenario the source code of u-boot should run on
>>>> arm 32 bit,not on arm 64,because I have the Samsung Chromebook "SNOW" model
>>>> XE303C12,that's powered by a Samsung Exynos 5250 (ARMv7 32 bit Cortex A15)
>>>> Soc.
>>>>
>>>>
>>>> 4) I'm not sure if I can chainload the customized u-boot created by
>>>> V.O.S that should be installed on the first partition with the u-boot
>>>> tailored for booting FreeBSD that should be installed on the partition 2....
>>>>
>>>>
>>>> 5) the xen developer said that u-boot should be compiled enabling this
>>>> option :
>>>>
>>>>
>>>> Code:
>>>>
>>>> CONFIG_CMO_BY_VA_ONLY=y
>>>>
>>>>
>>>>
>>>> Well,can you provide some good source that can help me to understand
>>>> how I can recompile u-boot for FreeBSD ? thanks.
>>>>
>>>> --
>>>> Mario.
>>>>
>>>>
>>>
>>> --
>>> Mario.
>>>
>>>
>>
>> --
>> Mario.
>>
>
>
> --
> Mario.
>


-- 
Mario.