Re: [yocto-patches] [auh][PATCH 2/3] modules/steps.py: refactor compile() to use parallel per-machine builds

On 4/25/26 02:51, Alexander Kanavin wrote:
> On Fri, 24 Apr 2026 at 19:04, Richard Purdie via
> lists.yoctoproject.org
> <[email protected]> wrote:
>> Whilst we do need to be mindful of this, I suspect there is some room
>> to parallelise a bit. How much parallelism is this expected to have?
>> Some existing jobs like oe-selftest do run a lot in parallel already so
>> this could be not that much different to that...
> Patch 3/3 sets the parallelism to four:
>
> -        self.opts['machines'] = settings.get('machines', 'qemux86-64
> qemuarm_musl').split()
> +        self.opts['machines'] = settings.get('machines', 'qemux86-64
> qemux86 qemuarm64_musl qemuarm_musl').split()
>
> Alex

Yes. The parallelism is determined by the machines setting.

Let me add more explanation for this patch.

The biggest problem of auh is that it's running too slowly. With serial 
run of builds, you don't dare to specify enough machines because it'll 
just spend too much time. For example, you upgrade the whole oe-core, in 
most of the time, what the build is doing is just building one recipe. 
The first recipe upgrade has already got cross/native tools, glibc, 
etc., available, and subsequent upgrades will only run its own build. 
Such builds consume very little resources. On my server, if auh is the 
only one that is running builds, the PSI data are very close to 0 for 
most of the time.

We don't need to worry about auh using too much resources, even if it's 
running with other autobuilder tests. Because Yocto has very good 
techniques to control this. For example, if we put 
autobuilder-resource-constraints.conf config fragment in local.conf, we 
don't need to worry about auh exhaust the servers' resources.

With this parallel build added, Yocto can configure oe-core's upgrade to 
use machines that oe-core uses as the acceptance criteria. This means a 
successful auh upgrade will very like to be merged without any change. 
If there is some failure, e.g., qemuarm64_musl, the 
developers/maintainers can see it directly without doing more local builds.

Regards,
Qi