RE: [poky] Error while bitbaking core-image-minimal using Poky 5.0.9 (Scarthgap) on VMware

RAM consumption under bitbake is a topic which you usually need to fine-tune for your setup.
By default bitbake spawns number of build processes equal to number of cores and each of those processes will spawn the same number of compiler threads.
So it's exponential, which is very problematic and oom-kills occur when several memory hungry build processes are run at the same time..

The rule is - the more CPUs, the faster the build.
However there is the RAM caveat - you have to have sufficient RAM for all the spawned processes.
I personally gave to my (Debian) VM 24 cores and 48G RAM with default exponential settings and never observer OOM kill.
My rule of thumb is to give it 2GB RAM for each CPU I assign.

Of course this is not guaranteed to work in all cases.
In worst case, you need cores*cores*x GB of RAM (e.g. for 16 cores you need 256*x GB).
But the reality is better, since likelihood that all running bitbake processes are busy with super-memory-hungry compiler processes is very low.
Since only few components are extremely memory hungry compilation processes, the average memory factor for each bitbake process is lower with higher cpu count.
You also have to take into account that with low number of cores and RAM, also the system eats something.

As example: you build image with 200 components, 8 of them are super-memory-hungry and usually there are some dependencies so they can't run all at the same time.
If you give it two cores, probability that all running processes will be memory hungry is basically 100%, so give it 4GB per core.
It you have 32 cores, you can't get to situation that spawned all processes are memory hungry, so even if some several at the same time, the sum will not be extreme, 2GB should be fine.

Additionally, you can tweak compression settings (e.g. yocto default for xz is set to unbelievable 50% of whole RAM ... imagine that 8 components are using it at the same time)
Also using BB_PRESSURE_MAX_{CPU|IO|MEMORY} configuration will help limit concurrent processes thus allow using less RAM.
Also maybe don't use the default cores*cores formula and tweak BB_NUMBER_THREADS/PARALLEL_MAKE variables.
And there are for sure other RAM relevant configurations/optimizations possible.

---

Regarding the problem with quilt - quilt should be used from recipe sysroot (workdir), not from host.
Per log below bitbake thinks it is installed but it is not found when needed.
I think this may be a consequence of oom-kill.
When oom-kill happens, I'd recommend deleting whole build directory because it is probably corrupted.
Trying to perform a build in corrupted build directory leads to very strange build errors.

---

Peter

> -----Original Message-----
> From: [email protected] <[email protected]> On Behalf Of
> srivamsi_malladi via lists.yoctoproject.org
> Sent: Thursday, May 15, 2025 13:02
> To: [email protected]
> Subject: [poky] Error while bitbaking core-image-minimal using Poky 5.0.9
> (Scarthgap) on VMware
> 
> Hello all,
> 
> TL;DR: I ran into a couple of errors and fixed them with the help of Gemini. I would
> like to know if there are any better solutions to solve these errors.
> 
> I am absolutely new to the Yocto project and Poky, so I thought I will so a 'hello
> world' kind of thing with Poky. For that, I setup a VMware with Ubuntu 22.04 LTS
> and with 27.7GB RAM and 16 cores hardware config. My host computer is a i9-
> 13th gen with 32GB RAM running Windows. I chose the really high (max actually)
> RAM and cores thinking I would have a really fast VM, but due to the problems I
> ran into, I realized more RAM, mores cores != more performance. So, following is
> the problem I ran into.
> 
> When I bitbake core-image-minimal in terminal, I got an error saying do_compile
> failed on gcc 13.0.1 and when I looked at the log, it said error x86_64-poky-linux-
> g++: fatal error: Killed signal terminated program cc1plus.
> After spending some time on Google and some forums, I found that my VM might
> be fighting with my host PC for resources and in this process, the tasks that
> bitbake wants to run (compiling gcc in this case) are not able to get enough PC
> resources due to which cc1plus got terminated which is needed in the process.
> I tried reducing the RAM to 16GB and cores to 8 and then to 4, the error still
> remained. After reducing the RAM to 8GB and cores to 2, gcc could finally
> compile, but I ran into another error.
> 
> 
> When I run bitbake core-image-minimal, I get an error and following are the
> contents of the corresponding log file
> 
> 	DEBUG: Executing python function extend_recipe_sysroot
> 
> 	NOTE: Direct dependencies are
> ['/home/ubuntu2204/work/poky/meta/recipes-devtools/quilt/quilt-
> native_0.67.bb:do_populate_sysroot']
> 
> 	NOTE: Installed into sysroot: []
> 
> 	NOTE: Skipping as already exists in sysroot: ['quilt-native']
> 
> 	DEBUG: Python function extend_recipe_sysroot finished
> 
> 	DEBUG: Executing python function do_patch
> 
> 	DEBUG: Executing python function patch_do_patch
> 
> 	DEBUG: Searching for 0001-Do-not-set-PYTHON_INSTALL_DIR-by-
> running-python.patch in paths:
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c-1.0.4/poky
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c/poky
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/files/poky
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c-1.0.4/
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c/
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-c/files/
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c-1.0.4/x86-64
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c/x86-64
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/files/x86-64
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c-1.0.4/
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c/
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-c/files/
> 
> 	DEBUG: Using /home/ubuntu2204/work/poky/meta/recipes-
> devtools/createrepo-c/createrepo-c/0001-Do-not-set-PYTHON_INSTALL_DIR-by-
> running-python.patch for 0001-Do-not-set-PYTHON_INSTALL_DIR-by-running-
> python.patch
> 
> 	DEBUG: Searching for 0001-include-rpm-rpmstring.h.patch in paths:
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c-1.0.4/poky
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c/poky
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/files/poky
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c-1.0.4/
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c/
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-c/files/
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c-1.0.4/x86-64
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c/x86-64
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/files/x86-64
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c-1.0.4/
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-
> c/createrepo-c/
> 
> 	    /home/ubuntu2204/work/poky/meta/recipes-devtools/createrepo-c/files/
> 
> 	DEBUG: Using /home/ubuntu2204/work/poky/meta/recipes-
> devtools/createrepo-c/createrepo-c/0001-include-rpm-rpmstring.h.patch for 0001-
> include-rpm-rpmstring.h.patch
> 
> 	NOTE: Applying patch '0001-Do-not-set-PYTHON_INSTALL_DIR-by-
> running-python.patch' (../meta/recipes-devtools/createrepo-c/createrepo-c/0001-
> Do-not-set-PYTHON_INSTALL_DIR-by-running-python.patch)
> 
> 	ERROR: Applying patch '0001-Do-not-set-PYTHON_INSTALL_DIR-by-
> running-python.patch' on target directory
> '/home/ubuntu2204/work/poky/build/tmp/work/x86_64-linux/createrepo-c-
> native/1.0.4/git'
> 
> 	CmdError('quilt --quiltrc
> /home/ubuntu2204/work/poky/build/tmp/work/x86_64-linux/createrepo-c-
> native/1.0.4/recipe-sysroot-native/etc/quiltrc push', 0, 'stdout:
> 
> 	stderr: /bin/sh: 1: quilt: not found
> 
> 	')
> 
> 	DEBUG: Python function patch_do_patch finished
> 
> 	DEBUG: Python function do_patch finished
> 
> I asked Gemini about the error and it suggested installing quilt. But it was already
> installed.
> Then I gave it the whole log file and then it explained that quilt-native is available in
> sysroot but not in PATH and so asked me to create a new layer with
> do_patch:prepend()where I should add the bindir (which should be /usr/bin in the
> sysroot) to PATH.
> This fixed the error.
> 
> Now my questions are
> 1. Regarding the first error, I had to decrease the RAM to 8GB and cores to 2
> which cured the error but increased the build time. Is there a better way to handle
> this without compromising on the build time?
> 2. Regarding the second error, did this occur because of some problems in my
> system? Or is this a known error with Poky Scarthgap?
> 
> Thanks & Regards,
> Srivamsi Malladi.
> 
> 
> 
> 
> 
> 
> 
>