Re: Multiple CPU support for uvmm on virt-arm64 qemu machine

Philipp Eppelt <[email protected]> Thu, 31 Oct 2024 18:47:47 +0100
Newsgroups	gmane.comp.micro-kernel.l4.devel
Message-ID	<[email protected]>
Hi Mohamed,

Am 31.10.24 um 14:30 schrieb Mohamed Dawod:
> Thanks Philipp,
> 
> Multiple CPUs worked for virt-arm64 machine
Yippie!

> I tried to launch 2 linux VMs on top of L4 using uvmm and assign 2 CPUs to one 
> of the two VMs and another 2 CPUs to the other one.
> I noticed that the linux booting process becomes slower and as more CPUs are 
> added to qemu with -smp option and passed to the VMs as more as the VM booting 
> becomes more slower!
> Also VMs become working randomly (sometimes they work and sometimes they hang or 
> one of them hangs) >
> Why does this strange behaviour happen when using uvmm and 2 Linux VMs  ?
To make this easier please show me your ned script starting the VMs.

If you use start_vm() please note that the `cpus` parameter takes a bitmap. So 
make sure to start VM1 with `cpus=0x3` and VM2 with `cpus=0xc` to place them on 
separate cores of a four core platform (e.g. QEMU with -smp 4).

Cheers,
Philipp

> 
> Thanks,
> Regards
> 
> On Wed, Oct 30, 2024 at 8:46 PM Philipp Eppelt <[email protected] 
> <mailto:[email protected]>> wrote:
> 
>     Hi Mohamed,
> 
>     Am 29.10.24 um 10:43 schrieb Mohamed Dawod:
>      > Hello,
>      > I'm trying to provide multiple CPUs for a linux VM on top of L4.
>      > I'm using the qemu virt machine and building for aarch64. so I used *-smp*
>      > option to provide more CPUs.
>      >
>      >     $ qemu-system-aarch64 -M virt,virtualization=true -cpu cortex-a57
>     -smp 4 -m
>      >     1024 -kernel ....etc....
>     I'm not sure which gic version qemu uses. Please try setting it explicitly to
>     with gic-version=3 argument:  `-M virt,virtualization=true,gic-version=3`
> 
>      >
>      > Unfortunately, This didn't work. I tried to add more CPU device nodes to
>     the dts
>      > file *virt-arm_virt-64.dts *but it also didn't work. >
>      > I think that it's because of the provided interrupt-controller with
>      > *virt-arm_virt-64.dts* in /l4/pkg/uvmm/conf/dts/ which mentioned that it
>      > supports only one CPU.
>      >
>      >     icsoc {
>      >              compatible = "simple-bus";
>      >              #address-cells = <2>;
>      >              #size-cells = <2>;
>      >              ranges;
>      >
>      >              /* Uvmm will adapt the compatible string depending on the
>     present gic
>      >               * version. It expects reg entries that provide enough space
>     for the
>      >               * Cpu/Dist interface for gicv2 (at least 0x1000, 0x1000) or the
>      >               * Dist/Redist interface for gicv3 (0x10000, 0x20000 *
>     number of cpus).
> 
>     I'm not an expert for ARM64, but judging from the line above I'd say you
>     have to
>     increase the size of the second reg entry. For example for four cores:
>              reg = <0 0x40000 0 0x10000>,
>                    <0 0x50000 0 0x80000>;
> 
>     You should be able to just use the github version of this file, it has a gic
>     node that is configured for 32 cores and comes with four CPU nodes.
>     https://github.com/kernkonzept/uvmm/blob/master/configs/dts/virt-arm_virt-64.dts <https://github.com/kernkonzept/uvmm/blob/master/configs/dts/virt-arm_virt-64.dts>
> 
> 
>      >               * *The entries provided here support any gicv2 setup or a
>     gicv3 setup
>      >               * with one Cpu.*
>      >               */
>      >              gic: interrupt-controller {
>      >                  compatible = "arm,gic-400", "arm,cortex-a15-gic",
>      >     "arm,cortex-a9-gic";
>      >                  #interrupt-cells = <3>;
>      >                  #address-cells = <0>;
>      >                  interrupt-controller;
>      >                  reg = <0 0x40000 0 0x10000>,
>      >                        <0 0x50000 0 0x20000>;
>      >                  };
>      >          };
>      >
>      >
>      > My question now, is there any workaround to support multiple CPUs for virt
>      > machine  on arm64  ?
> 
>     Multiple CPUs should work. For SMP there are a couple of things to consider:
>     - QEMU: -smp parameter
>     - Kernel configuration for SMP and the number of maximum cores
>     - The DTS defines the maximum number of cores for the uvmm will set up. So
>     adding CPU device nodes is the correct path.
>     - The ned script defines the number of cores available at runtime to uvmm. No
>     cpu parameter in the start_vm({}) call means the VM gets access to all cpus.
>     - Linux must of course also support SMP, but that's very likely not the problem
>     here ;-)
> 
>     I hope this sheds some light.
> 
>     Cheers,
>     Philipp
> 
>     -- 
>     [email protected] <mailto:[email protected]> -
>     Tel. 0351-41 883 221
>     http://www.kernkonzept.com <http://www.kernkonzept.com>
> 
>     Kernkonzept GmbH.  Sitz: Dresden.  Amtsgericht Dresden, HRB 31129.
>     Geschäftsführer: Dr.-Ing. Michael Hohmuth
>     _______________________________________________
>     l4-hackers mailing list -- [email protected]
>     <mailto:[email protected]>
>     To unsubscribe send an email to [email protected]
>     <mailto:[email protected]>
> 
> 
> 
> *Driving Innovation! Visit our website www.avelabs.com 
> <http://www.avelabs.com/>*, to read Avelabs Confidentiality Notice, follow this 
> link: http://www.avelabs.com/email/disclaimer.html 
> <http://www.avelabs.com/email/disclaimer.html>

-- 
[email protected] - Tel. 0351-41 883 221
http://www.kernkonzept.com

Kernkonzept GmbH.  Sitz: Dresden.  Amtsgericht Dresden, HRB 31129.
Geschäftsführer: Dr.-Ing. Michael Hohmuth

_______________________________________________
l4-hackers mailing list -- [email protected]
To unsubscribe send an email to [email protected]
OpenPGP_signature.asc (application/pgp-signature, 840 B)