Re: [yocto] Default to isolated Python virtual environments (venv)

On Tue, 2026-08-04 at 20:47 +0100, Richard Purdie via lists.yoctoproject.org wrote:
> On Tue, 2026-08-04 at 11:49 -0700, Michael Halstead wrote:
> > I tracked down the persistent issue on the autobuilder setup where
> > worker builds were failing after maintenance specifically on systems
> > like RHEL8-based hosts where we don't run the system python3.
> > 
> > We had `virtualenv_site_packages: true` configured across the cluster
> > to better test distro-provided Python packages and stay aligned with
> > current docs. Setting `virtualenv_site_packages: false` for RHEL8 and
> > OpenSUSE 15.6 based workers resolved the issue.
> 
> Changing the config "virtualenv_site_packages: false" on those older
> systems should be fine.
> 
> > I propose we stop using site packages on the autobuilder in general
> > to avoid similar issues in the future. I already have the changes
> > ready for the next maintenance window.
> > 
> > If we do so our documentation should probably be updated to match so
> > developers avoid these same environment edge cases. We could
> > update ref-manual/system-requirements.html to default to standard,
> > isolated virtual environments and pull host Python dependencies via
> > PyPI rather than assuming distro packages.
> > 
> > Let me know what you think or if there are any edge cases I haven't
> > considered. 
> 
> If we go this route, we may as well drop several of the distros from
> testing and mandate buildtools tarballs :/. I really don't want to go
> down the route of prescribing a venv when we've gone to relative levels
> of trouble to make bitbake easier to run on most systems.
> 
> I am a bit puzzled where we're running into issues as our buildtools-
> tarball does include python3-websockets so on those older systems, we
> should be using buildtools and that should have websockets in there?
> 
> What are we running outside of a buildtools env?

Just to clarify, we need a newer python on those systems for buildbot
itself, which isn't run inside buildtools. That is where the
requirement for the venv comes from.

I'm still puzzled about where/why we can't just use websockets from
buildtools in our build though...

Cheers,

Richard