Re: [bitbake-devel] [PATCH v4 00/11] bitbake-setup PyPI Packaging

Hi Paul,=0A=
=0A=
Thanks for the feedback.  I agree with your recommendations to add more exp=
lanation about what is going on.=0A=
=0A=
I don't think we have had a formal discussion on which tool to use for mana=
ging vendored modules.  I went with vendoring for 3 reasons:=0A=
=0A=
(1) that was the one that Ross had mentioned he used before=0A=
(2) vendoring was last updated in May 2026 whereas python-vendorize hasn't =
had an update on PyPI since October 2022 despite changes on GitHub last mad=
e in April 2024.=0A=
(3) it was promoted as having more advanced features and better suited for =
projects that are more than just a simple CLI=0A=
=0A=
My take on the stability promise is that they want to make clear that vendo=
ring is intended for pip itself and they are wisely marking other uses as u=
nsupported.  The result of this is that vendoring will continue to be maint=
ained as long as pip uses it.=0A=
=0A=
I interpreted the general rule of thumb as recommending against vendoring m=
odules if you want to be in PyPI.  However, there is a notable exception fo=
r packages that do bootstrapping (albeit they mean Python bootstrapping) an=
d those that necessitate changes to the vendored modules.  I feel that bitb=
ake and bitbake-setup could argue a case for both.=0A=
=0A=
I asked Claude about what would be involved with switching to python-vendor=
ize. One important difference is that it does not support automatically mer=
ging patches against the pristine source.  If we went to use it, we would n=
eed to add an additional tool to automatically apply the patches.=0A=
=0A=
To the question you asked in another e-mail: these patches came directly fr=
om the git commit history for bitbake.  They were custom changes to the ven=
dored modules that I reapplied after the update.=0A=
=0A=
The vendoring module also takes care of license collection and generation o=
f the .pyi stubs.    We certainly need the former and the latter may be use=
ful for improving good hygiene going forward.=0A=
=0A=
The import rewriting style of python-vendorize is also simply relative (ie.=
 from . import x) whereas vendoring uses absolute imports (eg. import bb._v=
endor.x).  I think the absolute imports would be preferred as they are more=
 readable and produce fewer errors.=0A=
=0A=
The versions of each upstream package is currently being stored in the vend=
or.txt file.  The versions represent the published releases in PyPI.=0A=
=0A=
Hope this helps clarify things.  Happy to continue the discussion and make =
any changes that are required.=0A=
=0A=
Cheers,=0A=
Rob=0A=
=0A=
=0A=
________________________________________=0A=
From: Paul Barker=0A=
Sent: Monday, June 29, 2026 5:06 AM=0A=
To: Woolley, Rob; [email protected]=0A=
Cc: Richard Purdie=0A=
Subject: Re: [bitbake-devel] [PATCH v4 00/11] bitbake-setup PyPI Packaging=
=0A=
=0A=
On Wed, 2026-06-24 at 10:20 -0700, Rob Woolley via=0A=
lists.openembedded.org wrote:=0A=
> This is an update to the previous series to rebase on master. It is also=
=0A=
> available at: https://github.com/robwoolley/bitbake/tree/b4/add-pypi-v8=
=0A=
>=0A=
> This version of the series integrates the vendoring module to update=0A=
> beautifulsoup4, ply, progressbar, simplediff, and typing_extensions.=0A=
>=0A=
> I also included Richard's latest commit to add version variables to the=
=0A=
> bitbake scripts and dropped the mechanism that I proposed.=0A=
=0A=
Hi Rob,=0A=
=0A=
Patches 1-5 and 11 look good to me.=0A=
=0A=
For patches 6-10 which improve vendoring of upstream packages, I think=0A=
we need to document things clearly. I've also made a couple of review=0A=
comments on some of these patches.=0A=
=0A=
Looking at the Python vendoring module [1]:=0A=
=0A=
    Should I use it?=0A=
=0A=
    This tool has no stability promises -- it has only one intended=0A=
    user: pip. There may be unannounced changes to this codebase at any=0A=
    time, as long as the intended user (i.e. the pip project) is=0A=
    prepared for those changes.=0A=
=0A=
    As a general rule of thumb, if the project is going to be a PyPI=0A=
    package, it should not use this tool.=0A=
=0A=
    Many downstream redistributors have policies against this kind of=0A=
    bundling of dependencies, which means that they'll patch your=0A=
    software to debundle it. This can cause various kinds of issues, due=0A=
    to violations of assumptions being made about where the dependencies=0A=
    are available/which versions are being used. These issues result in=0A=
    difficult-to-debug errors, which are fairly difficult to communicate=0A=
    with end users.=0A=
=0A=
    pip is a very special case with a thorough rationale [2] for=0A=
    vendoring/bundling dependencies with itself.=0A=
=0A=
We need to write down why we're using vendoring despite this warning,=0A=
with a rationale (similar to how pip documents this).=0A=
=0A=
We also need to document who's going to run `vendoring sync` and how=0A=
often. Do we automate this or rely on running it manually?=0A=
=0A=
[1]: https://pypi.org/project/vendoring/=0A=
[2]: https://pip.pypa.io/en/latest/development/vendoring-policy/#rationale=
=0A=
=0A=
Best regards,=0A=
=0A=
--=0A=
Paul Barker=0A=
=0A=