Re: How to use the framebuffer as primary video device instead of the nvidia passed-through graphic card in a bhyve/linux vm

Hello.

I know,but it does not work. If I use ONLY the framebuffer argument,it
works : the desktop manager is loaded within the vm window,but if between
the bhyve parameters I declare the framebuffer AND the nvidia slots,on the
vm window I see the blinking pointer because the output is redirected to
the screen that I have attached to the nvidia HDMI connector. The problem
is that at the moment I can't use that monitor,since I'm using my PC in a
different room.
I made an experiment : I have excluded all the parameters that may be used
in my old Ubuntu installation by installing Ubuntu (22.10) from scratch and
between the bhyve parameters I have added both the framebuffer and the 4
nVidia slots.

bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s
1,ahci-cd,/mnt/$vmdisk1'p2'/bhyve/iso/Linux/ubuntu-22.10-desktop-amd64.iso
\
-s 7:0,fbuf,tcp=0.0.0.0:5916,w=1600,h=950,wait \
-s 8:0,passthru,2/0/0 \
-s 8:1,passthru,2/0/1 \
-s 8:2,passthru,2/0/2 \
-s 8:3,passthru,2/0/3 \
-s 10,virtio-net,tap16 \
-s 11,virtio-9p,sharename=/ \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
vm0:16 < /dev/null & sleep 2 && vncviewer 0:16

When I choose "try and install Ubuntu" I see the blinking pointer because
the output is redirected to the screen that I can't use. But if I choose
"safe graphics" maybe it makes the magic. Below you can see what are the
parameters used by Ubuntu to display the installer even if between the
bhyve parameters I have used the framebuffer and the nVidia adapters.

https://ibb.co/L9JqQbL

Instead below you can see what are the parameters used by Ubuntu to display
the installer if between the bhyve parameters I use both the framebuffer
and the nVidia adapters but without choosing "safe graphics" on the ubuntu
boot menu.

https://ibb.co/wNJxxkV

As you can see,the parameter that makes the difference is called
"nomodeset". So,I presume that in a certain ubuntu configuration file I
should add that parameter....


Il giorno dom 4 dic 2022 alle ore 10:41 Corvin Köhne <[email protected]>
ha scritto:

> On December 3, 2022 7:14:29 PM GMT+01:00, Mario Marietto <
> [email protected]> wrote:
>>
>> ok. I tried like this,but it didn't work :
>>
>> bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
>> -s 0,hostbridge \
>> -s 2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootindex=1 \
>> -s 3,virtio-blk,/dev/$vmdisk4 \
>> -s 4,virtio-blk,/dev/$vmdisk2 \-s 7,fbuf,tcp=0.0.0.0:5919,w=1600,h=950,wait \
>> -s 8:0,passthru,2/0/0 \
>> -s 8:1,passthru,2/0/1 \
>> -s 8:2,passthru,2/0/2 \
>> -s 8:3,passthru,2/0/3 \
>> -s 10,virtio-net,tap19 \
>> -s 11,virtio-9p,sharename=/ \
>> -s 30,xhci,tablet \
>> -s 31,lpc \
>> -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
>> vm0:19 < /dev/null & sleep 2 && vncviewer 0:19
>>
>> I tried specifying the bus ID of the framebuffer and I have used this
>> xorg.conf file :
>>
>> Section "Files"
>>     ModulePath   "/usr/lib/xorg/modules"
>>     FontPath     "/usr/share/fonts/X11/misc"
>>     FontPath     "/usr/share/fonts/X11/cyrillic"
>>     FontPath     "/usr/share/fonts/X11/100dpi/:unscaled"
>>     FontPath     "/usr/share/fonts/X11/75dpi/:unscaled"
>>     FontPath     "/usr/share/fonts/X11/Type1"
>>     FontPath     "/usr/share/fonts/X11/100dpi"
>>     FontPath     "/usr/share/fonts/X11/75dpi"
>>     FontPath     "built-ins"
>> EndSection
>>
>> Section "Module"
>>     Load  "vnc"
>>     Load  "glx"
>> EndSection
>>
>>
>> Section "InputDevice"
>>     Identifier  "Keyboard0"
>>     Driver      "kbd"
>> EndSection
>>
>> Section "InputDevice"
>>     Identifier  "Mouse0"
>>     Driver      "mouse"
>>     Option        "Protocol" "auto"
>>     Option        "Device" "/dev/input/mice"
>>     Option        "ZAxisMapping" "4 5 6 7"
>>
>> EndSection
>>
>> Section "Monitor"
>>     Identifier   "Monitor0"
>>     VendorName   "Monitor Vendor"
>>     ModelName    "Monitor Model"
>> EndSection
>>
>> Section "Device"
>>    Identifier  "Card0"
>>    Driver      "modesetting"
>>    BusID       "PCI:0:7:0"
>>
>> EndSection
>>
>> Section "Screen"
>>     Identifier "Screen0"
>>     Device     "Card0"
>>     Monitor    "Monitor0"
>>     SubSection "Display"
>>         Viewport   0 0
>>         Depth     1
>>     EndSubSection
>>     SubSection "Display"
>>         Viewport   0 0
>>         Depth     4
>>     EndSubSection
>>     SubSection "Display"
>>         Viewport   0 0
>>         Depth     8
>>     EndSubSection
>>     SubSection "Display"
>>         Viewport   0 0
>>         Depth     15
>>     EndSubSection
>>     SubSection "Display"
>>         Viewport   0 0
>>         Depth     16
>>     EndSubSection
>>     SubSection "Display"
>>         Viewport   0 0
>>         Depth     24
>>     EndSubSection
>> EndSection
>>
>> The error reported has been :
>>
>> https://ibb.co/1KX2h26
>> https://ibb.co/Cv5FffB
>>
>> thanks.
>>
>> Il giorno sab 3 dic 2022 alle ore 17:34 Corvin Köhne <[email protected]>
>> ha scritto:
>>
>>> On December 3, 2022 4:49:46 PM GMT+01:00, Mario Marietto <
>>> [email protected]> wrote:
>>>>
>>>> Hello to everyone.
>>>>
>>>> what Im trying to do is to set the framebuffer video adapter as primary
>>>> graphic card on my bhyve-ubuntu vm instead of the nvidia RTX 2080 ti card
>>>> that I have passed through. What I want to do really is to use both the
>>>> graphic adapters,but the primary should be the framebuffer and the
>>>> secondary the nvidia. I tried different Xorg configurations,but what I've
>>>> got is that Xorg failed to display some errors. So,the controller that you
>>>> see below should be used as primary inside the ubuntu vm :
>>>>
>>>> 00:1d.0 VGA compatible controller: Device fb5d:40fb
>>>>
>>>> while the ones you see below as secondary :
>>>>
>>>> 08:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)
>>>> 08:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1)
>>>> 08:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1)
>>>> 08:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)
>>>>
>>>>
>>>> The script that I use to launch the vm is the following :
>>>>
>>>> #!/bin/sh
>>>> setxkbmap it
>>>> vms="$(ls /dev/vmm/*)"
>>>> vncs="$(ps ax | awk '/vncviewer [0]/{print $6}')"
>>>>
>>>> for vm in $vms; do
>>>>                 session="${vm##*/}"
>>>>                 echo "bhyve session = $session"
>>>>                 echo "vnc session = $vncs"
>>>>                 if ! printf '%s\n' "${vncs}" | grep "${session}"; then
>>>>                                 printf 'VNC session not found,destroying ghost vms\n'
>>>>                                 bhyvectl --vm=$session --destroy
>>>>                 else
>>>>                                 printf 'Found VNC session %s\n' "${session},no ghost vms found,not destroying them"
>>>>                 fi
>>>> done
>>>>
>>>> vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (2015020204055E)/ && d{print d}'`
>>>> echo "TOSHIBA External USB 3.0 1.8 TB ; $vmdisk1"
>>>>
>>>> mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'
>>>>
>>>> bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
>>>> -s 0,hostbridge \
>>>> -s 2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootindex=1 \
>>>> -s 3,virtio-blk,/dev/$vmdisk4 \
>>>> -s 4,virtio-blk,/dev/$vmdisk2 \
>>>> -s 8:0,passthru,2/0/0 \
>>>> -s 8:1,passthru,2/0/1 \
>>>> -s 8:2,passthru,2/0/2 \
>>>> -s 8:3,passthru,2/0/3 \
>>>> -s 10,virtio-net,tap19 \
>>>> -s 11,virtio-9p,sharename=/ \
>>>> -s 29,fbuf,tcp=0.0.0.0:5919,w=1600,h=950,wait \
>>>> -s 30,xhci,tablet \
>>>> -s 31,lpc \
>>>> -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
>>>> vm0:19 < /dev/null & sleep 2 && vncviewer 0:19
>>>>
>>>> For sure ,on /boot/loader.conf I've added :
>>>>
>>>> /boot/loader.conf
>>>>
>>>> pptdevs="2/0/0 2/0/1 2/0/2 2/0/3"
>>>>
>>>> As I said before,I tried two xorg conf files to achieve the goal. On the
>>>> first one I tried to add only the framebuffer,like this :
>>>>
>>>>
>>>> Section "Files"
>>>>     ModulePath   "/usr/lib/xorg/modules"
>>>>     FontPath     "/usr/share/fonts/X11/misc"
>>>>     FontPath     "/usr/share/fonts/X11/cyrillic"
>>>>     FontPath     "/usr/share/fonts/X11/100dpi/:unscaled"
>>>>     FontPath     "/usr/share/fonts/X11/75dpi/:unscaled"
>>>>     FontPath     "/usr/share/fonts/X11/Type1"
>>>>     FontPath     "/usr/share/fonts/X11/100dpi"
>>>>     FontPath     "/usr/share/fonts/X11/75dpi"
>>>>     FontPath     "built-ins"
>>>> EndSection
>>>>
>>>> Section "Module"
>>>>     Load  "vnc"
>>>>     Load  "glx"
>>>> EndSection
>>>>
>>>>
>>>> Section "InputDevice"
>>>>     Identifier  "Keyboard0"
>>>>     Driver      "kbd"
>>>> EndSection
>>>>
>>>> Section "InputDevice"
>>>>     Identifier  "Mouse0"
>>>>     Driver      "mouse"
>>>>     Option        "Protocol" "auto"
>>>>     Option        "Device" "/dev/input/mice"
>>>>     Option        "ZAxisMapping" "4 5 6 7"
>>>>
>>>> EndSection
>>>>
>>>> Section "Monitor"
>>>>     Identifier   "Monitor0"
>>>>     VendorName   "Monitor Vendor"
>>>>     ModelName    "Monitor Model"
>>>> EndSection
>>>>
>>>> Section "Device"
>>>>    Identifier  "Card0"
>>>>    Driver      "modesetting"
>>>>    BusID       "PCI:0:29:0"
>>>>
>>>> EndSection
>>>>
>>>> Section "Screen"
>>>>     Identifier "Screen0"
>>>>     Device     "Card0"
>>>>     Monitor    "Monitor0"
>>>>     SubSection "Display"
>>>>         Viewport   0 0
>>>>         Depth     1
>>>>     EndSubSection
>>>>     SubSection "Display"
>>>>         Viewport   0 0
>>>>         Depth     4
>>>>     EndSubSection
>>>>     SubSection "Display"
>>>>         Viewport   0 0
>>>>         Depth     8
>>>>     EndSubSection
>>>>     SubSection "Display"
>>>>         Viewport   0 0
>>>>         Depth     15
>>>>     EndSubSection
>>>>     SubSection "Display"
>>>>         Viewport   0 0
>>>>         Depth     16
>>>>     EndSubSection
>>>>     SubSection "Display"
>>>>         Viewport   0 0
>>>>         Depth     24
>>>>     EndSubSection
>>>> EndSection
>>>>
>>>>
>>>> but it didn't work. This is the log file that shows the errors reported : https://pastebin.ubuntu.com/p/Gv7wgsDR4K/
>>>> I have also removed the xorg.conf file,but it didn't work either. This is the log file that shows the errors reported :
>>>>
>>>>
>>>> https://pastebin.ubuntu.com/p/wNcfQTByQm/
>>>>
>>>> Can someone give me some suggestions that can help me to understand where the mistake is,please,thanks.
>>>>
>>>> --
>>>> Mario.
>>>>
>>>
>>> Try to assign a lower pci slot number to the framebuffer device than to
>>> the nvidia gpu in your bhyve command.
>>> --
>>> Best regards,
>>> Corvin
>>>
>>
>>
> I'm unfamiliar with X but according to your logs it looks like it selects
> the framebuffer device as primary graphics in the first place:
>
>
> (--) PCI:*(0@0:7:0) ...
> (--) PCI: (0@0:8:0) ...
>
> --
> Best regards,
> Corvin
>


-- 
Mario.