Re: [yocto] Limiting Yocto Memory Usage
Randy MacLeod <[email protected]> Tue, 28 Jul 2026 17:20:52 -0400
| Newsgroups | org.yoctoproject.lists.yocto |
|---|---|
| Message-ID | <[email protected]> |
--------------IaP3JtqQmWDFtK2oxpD0P3q0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-0064b401.pphosted.com id 66SL2vNX3948389 On 2026-07-28 16:35, Ferry Toth via lists.yoctoproject.org wrote: > Hi > > Op 28-07-2026 om 09:30 schreef Ferry Toth: >> Hi, >> >> Op 14-07-2026 om 23:33 schreef Randy MacLeod: >>> Hi Luis, >>> >>> On 2026-07-10 17:26, Luis Merayo via lists.yoctoproject.org wrote: >>>> Hi, >>>> >>>> I would like to know if there is a way to limit Bitbake's Memory=20 >>>> usage to a specific percentage of the available RAM. For example,=20 >>>> can the Yocto build be limited to use a maximum of 70% of RAM? >>> Not without using a container to do the build. >>>> >>>> Could these three variables achieve this? >>>> >>>> BB_PRESSURE_MAX_CPU <https://docs.yoctoproject.org/dev/ref-manual/=20 >>>> variables.html#term-BB_PRESSURE_MAX_CPU>, BB_PRESSURE_MAX_IO=20 >>>> <https:// docs.yoctoproject.org/dev/ref-manual/variables.html#term-=20 >>>> BB_PRESSURE_MAX_IO>=C2=A0, BB_PRESSURE_MAX_MEMORY <https://=20 >>>> docs.yoctoproject.org/dev/ref-manual/variables.html#term-=20 >>>> BB_PRESSURE_MAX_MEMORY> >>>> >>>> I see the range for these variables is from 1 to 1000000, so I=20 >>>> would like to understand if there is a way to specify their values=20 >>>> as a function of the available=C2=A0RAM. >>> No, there is only pressure when resources are over-commited so if=20 >>> you have free memory, there's no over-commit. >>> https://docs.kernel.org/accounting/psi.html >>> >>> That's the design intent of the bitbake PRESSURE regulation system. >>> >>> There are plans to work on additional build system regulation. >>> >>> The first idea is to use the jobserver design: >>> https://lore.kernel.org/openembedded-core/?q=3DGNU+AND+jobserver+=20 >>> +AND+f%3Amacleod >>> and that would indirectly limit memory consumption since there would=20 >>> be fewer instances of gcc / rustc / ... running >>> at once. >> >> I do that, since 2 years or so. https://github.com/htot/meta-intel-=20 >> edison/commits/whinlatter/ >> >> In principle you need to catch the call to make and ninja (to force=20 >> them to use the jobserver) and patch bitbake (see setup.sh) to act as=20 >> a jobserver. >> >> Now got it working on wrynose, still need to push that. Maybe later=20 >> today.With wrynose ninja has support for jobserver so doesn't need=20 >> patching, just a bbappend. > > Pushed my wrynose branch now:=20 > https://github.com/htot/meta-intel-edison/tree/wrynose Nice, are you interested in trying to get that or something based on=20 that work merged into oe-core/master ? I have been wanting to do for a while but other responsibilities keep=20 pushing it to the back burner or out of the kitchen completely ! ../Randy > >> With this you can limit the number of compile jobs. On my builds I=20 >> found g++ compiles are the heaviest, about 1GB RAM each. So, with 16=20 >> cores I need 16 GB. >> Separately I found linking nodejs takes 5 x 5GB (5 jobs linking with=20 >> LTO?). So, I limited that to just one (since both nodejs and nodejs-=20 >> native can be built at the same time). >> >> I'm not building rust so there may be other bottlenecks to resolve. >> >>> There's also an even less clear idea about being able to restrict=20 >>> the build by specifying memory usage. >>> That shouldn't be too hard to implement in bitbake in the same point=20 >>> where the pressure checks happen. >>> Are you interested in working on that ? >>> >>> There are of course other levers to pull to reduce CPU/memory=20 >>> consumption: >>> https://docs.yoctoproject.org/dev-manual/limiting-resources.html >>> >>> Have you tried them ? >>> >>> >>>> >>>> Regards, >>>> >>>> Luis >>>> >>>> >>>> >>>> File:RidgeRun.ai banner.png <https://ridgerun.ai/> >>>> This email and any attachments are intended for the sole use of the=20 >>>> named recipient(s) and contain(s) confidential information that may=20 >>>> be proprietary, privileged, or copyrighted under applicable law. If=20 >>>> you are not the intended recipient, do not read, copy, or forward=20 >>>> this email message or any attachments, delete this email message=20 >>>> and any attachments immediately. >>>> >>>> >>>> >>> >>> --=20 >>> # Randy MacLeod >>> # Wind River Linux >>> >> >> > > > -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- > Links: You receive all messages sent to this group. > View/Reply Online (#66668):https://lists.yoctoproject.org/g/yocto/messa= ge/66668 > Mute This Topic:https://lists.yoctoproject.org/mt/120272854/3616765 > Group Owner:[email protected] > Unsubscribe:https://lists.yoctoproject.org/g/yocto/unsub [randy.macleod= @windriver.com] > -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- > --=20 # Randy MacLeod # Wind River Linux --------------IaP3JtqQmWDFtK2oxpD0P3q0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <!DOCTYPE html><html><head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <head> </head> <body> <div class="moz-cite-prefix">On 2026-07-28 16:35, Ferry Toth via lists.yoctoproject.org wrote:<br> </div> <blockquote type="cite" cite="mid:[email protected]">Hi <br> <br> Op 28-07-2026 om 09:30 schreef Ferry Toth: <br> <blockquote type="cite">Hi, <br> <br> Op 14-07-2026 om 23:33 schreef Randy MacLeod: <br> <blockquote type="cite">Hi Luis, <br> <br> On 2026-07-10 17:26, Luis Merayo via lists.yoctoproject.org wrote: <br> <blockquote type="cite">Hi, <br> <br> I would like to know if there is a way to limit Bitbake's Memory usage to a specific percentage of the available RAM. For example, can the Yocto build be limited to use a maximum of 70% of RAM? <br> </blockquote> Not without using a container to do the build. <br> <blockquote type="cite"> <br> Could these three variables achieve this? <br> <br> BB_PRESSURE_MAX_CPU <a class="moz-txt-link-rfc2396E" href="https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-BB_PRESSURE_MAX_CPU"><https://docs.yoctoproject.org/dev/ref-manual/ variables.html#term-BB_PRESSURE_MAX_CPU></a>, BB_PRESSURE_MAX_IO <a class="moz-txt-link-rfc2396E" href="https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-BB_PRESSURE_MAX_IO"><https:// docs.yoctoproject.org/dev/ref-manual/variables.html#term- BB_PRESSURE_MAX_IO></a> , BB_PRESSURE_MAX_MEMORY <a class="moz-txt-link-rfc2396E" href="https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-BB_PRESSURE_MAX_MEMORY"><https:// docs.yoctoproject.org/dev/ref-manual/variables.html#term- BB_PRESSURE_MAX_MEMORY></a> <br> <br> I see the range for these variables is from 1 to 1000000, so I would like to understand if there is a way to specify their values as a function of the available RAM. <br> </blockquote> No, there is only pressure when resources are over-commited so if you have free memory, there's no over-commit. <br> <a class="moz-txt-link-freetext" href="https://docs.kernel.org/accounting/psi.html">https://docs.kernel.org/accounting/psi.html</a> <br> <br> That's the design intent of the bitbake PRESSURE regulation system. <br> <br> There are plans to work on additional build system regulation. <br> <br> The first idea is to use the jobserver design: <br> <a class="moz-txt-link-freetext" href="https://lore.kernel.org/openembedded-core/?q=GNU+AND+jobserver+">https://lore.kernel.org/openembedded-core/?q=GNU+AND+jobserver+</a> +AND+f%3Amacleod <br> and that would indirectly limit memory consumption since there would be fewer instances of gcc / rustc / ... running <br> at once. <br> </blockquote> <br> I do that, since 2 years or so. <a class="moz-txt-link-freetext" href="https://github.com/htot/meta-intel">https://github.com/htot/meta-intel</a>- edison/commits/whinlatter/ <br> <br> In principle you need to catch the call to make and ninja (to force them to use the jobserver) and patch bitbake (see setup.sh) to act as a jobserver. <br> <br> Now got it working on wrynose, still need to push that. Maybe later today.With wrynose ninja has support for jobserver so doesn't need patching, just a bbappend. <br> </blockquote> <br> Pushed my wrynose branch now: <a class="moz-txt-link-freetext" href="https://github.com/htot/meta-intel-edison/tree/wrynose">https://github.com/htot/meta-intel-edison/tree/wrynose</a> <br> </blockquote> <br> Nice, are you interested in trying to get that or something based on that work merged into oe-core/master ?<br> I have been wanting to do for a while but other responsibilities keep pushing it to the back burner or out of the kitchen completely !<br> <br> <p>../Randy</p> <p><br> </p> <blockquote type="cite" cite="mid:[email protected]"><br> <blockquote type="cite">With this you can limit the number of compile jobs. On my builds I found g++ compiles are the heaviest, about 1GB RAM each. So, with 16 cores I need 16 GB. <br> Separately I found linking nodejs takes 5 x 5GB (5 jobs linking with LTO?). So, I limited that to just one (since both nodejs and nodejs- native can be built at the same time). <br> <br> I'm not building rust so there may be other bottlenecks to resolve. <br> <br> <blockquote type="cite">There's also an even less clear idea about being able to restrict the build by specifying memory usage. <br> That shouldn't be too hard to implement in bitbake in the same point where the pressure checks happen. <br> Are you interested in working on that ? <br> <br> There are of course other levers to pull to reduce CPU/memory consumption: <br> <a class="moz-txt-link-freetext" href="https://docs.yoctoproject.org/dev-manual/limiting-resources.html">https://docs.yoctoproject.org/dev-manual/limiting-resources.html</a> <br> <br> Have you tried them ? <br> <br> <br> <blockquote type="cite"> <br> Regards, <br> <br> Luis <br> <br> <br> <br> <a class="moz-txt-link-freetext" href="File:RidgeRun.ai">File:RidgeRun.ai</a> banner.png <a class="moz-txt-link-rfc2396E" href="https://ridgerun.ai/"><https://ridgerun.ai/></a> <br> This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged, or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments, delete this email message and any attachments immediately. <br> <br> <br> <br> </blockquote> <br> -- <br> # Randy MacLeod <br> # Wind River Linux <br> <br> </blockquote> <br> <br> </blockquote> <br> <br> <fieldset class="moz-mime-attachment-header"></fieldset> <pre wrap="" class="moz-quote-pre">-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#66668): <a class="moz-txt-link-freetext" href="https://lists.yoctoproject.org/g/yocto/message/66668">https://lists.yoctoproject.org/g/yocto/message/66668</a> Mute This Topic: <a class="moz-txt-link-freetext" href="https://lists.yoctoproject.org/mt/120272854/3616765">https://lists.yoctoproject.org/mt/120272854/3616765</a> Group Owner: <a class="moz-txt-link-abbreviated" href="mailto:[email protected]">[email protected]</a> Unsubscribe: <a class="moz-txt-link-freetext" href="https://lists.yoctoproject.org/g/yocto/unsub">https://lists.yoctoproject.org/g/yocto/unsub</a> [<a class="moz-txt-link-abbreviated" href="mailto:[email protected]">[email protected]</a>] -=-=-=-=-=-=-=-=-=-=-=- </pre> </blockquote> <p><br> </p> <pre class="moz-signature" cols="72">-- # Randy MacLeod # Wind River Linux</pre> </body> </html> --------------IaP3JtqQmWDFtK2oxpD0P3q0--