Re: vmm (bhyve) on GICv2

Can someone confirm that bhyve works ok on the RockPro64 (RK3399) ? Has
bhyve been patched in relation to this bug :

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277559

?

I read that there is a huge amount of work to do on the RPI5 before it
works with FreeBSD. And anyway,it does not support bhyve. So if I will buy
a soc,it will be the RockPro64....



On Wed, May 22, 2024 at 6:50 PM Mark Millard <[email protected]> wrote:

> On May 22, 2024, at 08:43, Ronald Klop <[email protected]> wrote:
>
>
> >> Van: Mark Millard <[email protected]>
> >> Datum: woensdag, 22 mei 2024 17:34
> >> Aan: Lexi Winter <[email protected]>
> >> CC: Mario Marietto <[email protected]>, [email protected]
> >> Onderwerp: Re: vmm (bhyve) on GICv2
> >> On May 22, 2024, at 00:25, Lexi Winter <[email protected]> wrote:
> >>
> >> > Mario Marietto:
> >> >> On Wed, May 22, 2024 at 9:03AM Lexi Winter <[email protected]> wrote:
> >> >>> i was pleased to see ARM64 vmm has been merged to main recently.
> >> >>>
> >> >>> i have several use-cases to run bhyve on RPi4, but unfortunately
> this
> >> >>> hardware only supports GICv2 while vmm requires GICv3.
> >> >>>
> >> >>> does anyone have an idea if GICv2 is likely to be supported in the
> >> >>> future?
> >> >>
> >> >> Does RPi5 support GICv3 ? I would like to know which kind of
> relatively
> >> >> cheap arm hardware can run bhyve.
> >> >
> >> > i don't know off hand, but i do know FreeBSD doesn't support RPi5 at
> >> > all right now, so this is not an immediate solution :-)
> >>
> >> I boot and run a RPi5 via EDK2 in the microsd card slot and a USB3
> >> UFS system media (that has lots of different swap partition sizes
> >> for use on various machines with widely varying amounts of RAM):
> >>
> >> # gpart show -pl
> >> =>       40  249737136    mmcsd0  GPT  (119G)
> >>          40       2008            - free -  (1.0M)
> >>        2048  249733120  mmcsd0p1  RPi5-edk2  (119G)
> >>   249735168       2008            - free -  (1.0M)
> >>
> >> =>        34  1875384941     da0  GPT  (894G)
> >>           34       32734          - free -  (16M)
> >>        32768      501760   da0p1  PkgBaseEFI  (245M)
> >>       534528    20971520   da0p2  PkgBaseSwp10  (10G)
> >>     21506048    29360128   da0p3  PkgBaseSwp14  (14G)
> >>     50866176    33554432   da0p4  PkgBaseSwp16  (16G)
> >>     84420608    67108864   da0p5  PkgBaseSwp32  (32G)
> >>    151529472    96468992   da0p6  PkgBaseSwp46  (46G)
> >>    247998464   268435456   da0p7  PkgBaseSwp128  (128G)
> >>    516433920     7340032   da0p8  PkgBaseSwp3p5  (3.5G)
> >>    523773952    13631488  da0p10  PkgBaseSwp6p5  (6.5G)
> >>    537405440  1337979528   da0p9  PkgBaseUFS  (638G)
> >>   1875384968           7          - free -  (3.5K)
> >>
> >> See: https://github.com/worproject/rpi5-uefi
> >>
> >> # uname -apKU
> >> FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT #5
> main-n269589-9dcf39575efb-dirty: Sun Apr 21 01:42:00 PDT 2024
>  root@aarch64-main-pbase:/usr/obj/BUILDs/main-CA76-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA76
> arm64 aarch64 1500018 1500018
> >>
> >> That kernel is at: /boot/kernel.CA76-NODBG/kernel in my context.
> >>
> >> But I also have kernels that I got from:
> >>
> >> https://pkg.freebsd.org/FreeBSD:15:aarch64/base_latest/
> >>
> >> that boot it as well:
> >>
> >> # strings /boot/kernel.GENERIC-NODEBUG/kernel | grep 15.0-
> >> @(#)FreeBSD 15.0-CURRENT main-n269581-dfa39133b333 GENERIC-NODEBUG
> >> FreeBSD 15.0-CURRENT main-n269581-dfa39133b333 GENERIC-NODEBUG
> >> 15.0-CURRENT
> >>
> >> And the debug version (witness and such):
> >>
> >> # strings /boot/kernel/kernel | grep 15.0-
> >> @(#)FreeBSD 15.0-CURRENT main-n269581-dfa39133b333 GENERIC
> >> FreeBSD 15.0-CURRENT main-n269581-dfa39133b333 GENERIC
> >> 15.0-CURRENT
> >>
> >> (It has beem a while since I updated.)
> >>
> >> All these kernels boot a world that I got from the same:
> >>
> >> https://pkg.freebsd.org/FreeBSD:15:aarch64/base_latest/
> >>
> >> I do have another world in a directory tree that I built that I
> >> can chroot to.
> >>
> >> ===
> >> Mark Millard
> >> marklmi at yahoo.com
> >>
> >
> >
> > Hi,
> >
> > This looks nice. Are there any FreeBSD patches needed to boot a RPI5? Or
> only an up-to-date EDK2?
>
> The kernels and world installed via use of:
>
> https://pkg.freebsd.org/FreeBSD:15:aarch64/base_latest/
>
> that I referenced were/are used unmodified. Those materials are
> as built by the FreeBSD project, not built by me.
>
> I build my other kernels and the world that I can chroot to
> in order to experiment with builds that explicitly target
> Cortext-A76 instruction set compatible aarch64 contexts. Such
> is not required. The same Cortex-A76 kernels and world can be
> used on the Windows DevKit 2023's combination of Cortex-X1C's
> and Cortex-A78C's.
>
> I have only tried EDK2 as distributed via:
>
> https://github.com/worproject/rpi5-uefi/
>
> I have not built it myself.
>
> > And if it works well can we collaborate in documenting the RPI5 on
> https://wiki.freebsd.org/arm/Raspberry%20Pi?
>
> I'll note that my normal use is headless: serial console and
> ssh-over-ethernet. The EtherNet use is via a USB3 dongle: the
> built-in EtherNet is not supported via EDK2.
>
> Also, I leave the microsd card that has EDK2 on it in the
> microsd card slot and do not normally have it mounted. But
> the /etc/fstab that I use has a line:
>
> /dev/gpt/RPi5-edk2              /RPi5-edk2      msdosfs rw,noatime,noauto
>      0       0
>
> to mount it when desired. Presuming the /RPi5-edk2/ was
> created previously:
>
> # mount /RPi5-edk2
>
> # ls -loaT /RPi5-edk2
> total 2180
> drwxr-xr-x   1 root wheel -       32768 Dec 31 23:00:00 1979 .
> drwxr-xr-x  26 root wheel -         512 May 22 09:29:16 2024 ..
> -rwxr-xr-x   1 root wheel uarch 2031616 Mar 16 03:43:48 2024 RPI_EFI.fd
> -rwxr-xr-x   1 root wheel -       76038 Mar 15 20:46:12 2024
> bcm2712-rpi-5-b.dtb
> -rwxr-xr-x   1 root wheel uarch     474 Apr 19 18:38:24 2024 config.txt
> -rwxr-xr-x   1 root wheel uarch     474 Apr 19 18:38:24 2024
> config.txt.m_m_edk2_rpi5
>
> # cat /RPi5-edk2/config.txt
> armstub=RPI_EFI.fd
> device_tree_address=0x1f0000
> device_tree_end=0x210000
>
> # Force 32 bpp framebuffer allocation.
> framebuffer_depth=32
>
> # Disable compensation for displays with overscan.
> disable_overscan=1
>
> # Force maximum USB power regardless of the power supply.
> usb_max_current_enable=1
>
> # Force maximum CPU speed.
> force_turbo=1
>
> #
> # Local additions:
> enable_uart=1
> uart_2ndstage=1
> dtdebug=1
> disable_commandline_tags=1
> #
> [pi5]
> over_voltage_delta=100000
> arm_freq=2600
> [all]
>
> The local additions are not required.
>
> I'll note that:
>
> https://github.com/worproject/rpi5-uefi/blob/master/README.md
>
> documents ACPI mode as supporting (extracted from a table):
>
> OS:                        FreeBSD
> Version:                   13.2
> Tested/supported hardware: Display, UART, USB, SD, PCIe
> Notes:                     * SD is limited to HS.
>
> DeviceTree mode is documented with just:
>
> QUOTE
> The included DTB is meant for the RPi downstream 6.1.y kernel.
> END QUOTE
>
> And, so, FreeBSD is likely not a useful match to DeviceTree
> mode at this time.
>
> ===
> Mark Millard
> marklmi at yahoo.com
>
>

-- 
Mario.