Re: [Buildroot] [PATCH 2/2] support/scripts/size-stats: properly assign pre-compiled .pyc files

Yann,


> -----Original Message-----
> From: [email protected] <[email protected]>
> Sent: Monday, August 10, 2026 8:28 AM
> To: [email protected]
> Cc: [email protected]; Michael Klein <[email protected]>;
> Vincent Fazio <[email protected]>
> Subject: [PATCH 2/2] support/scripts/size-stats: properly assign
> pre-compiled .pyc files
> 
> From: "Yann E. MORIN" <[email protected]>
> 
> Since commit 3fed42456693 (package/python3: use the provided pyc
> compiler), we abide by the PEP 3147 guidelines on where the pyc files
> are located. In practice, this means that, for a .py file /foo/bar.py,
> the corresponding .pyc file will be /foo/__pycache__/bar.pythonX.Y.pyc
> (where X.Y is the current major.minor python version, e.g. 3.14).
> 
> Add support for this layout in the size-stats scrip, which so far only
> supported the legacy, pre-PEP 3147 layout of having the .pyc next to the
> .py.
> 
> When we associate files to a package, we only have the original name,
> and we must construct the associated filenames. For the .pyc, this would
> require that the version of python be passed to the size-stats script,
> but this is not so nice. Instead, just scan the pycache directory to
> find the corresponding files. This is not much nicer either, but at
> least the wart, if bigger, is in a single place that would be easy to
> chop out or rework if need be (e.g. if we need to add support for more
> similarly target-finalize-generated files in the future)...
> 
> Signed-off-by: Yann E. MORIN <[email protected]>
> Cc: Michael Klein <[email protected]>
> Cc: Vincent Fazio <[email protected]>
> ---
>  support/scripts/size-stats | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> diff --git a/support/scripts/size-stats b/support/scripts/size-stats
> index fdcaeaa546..9a0aafbf2e 100755
> --- a/support/scripts/size-stats
> +++ b/support/scripts/size-stats
> @@ -77,7 +77,23 @@ def add_file(filesdict, relpath, abspath, pkg):
>  def get_associated_files(relpath, abspath):
>      # also check for compiled .pyc file
>      if relpath.endswith(".py"):
> +        # Legacy .pyc next to .py
>          yield (relpath + "c", abspath + "c")
> +        # PEP 3147 layout
> +        basename = os.path.basename(abspath).rsplit(".", 1)[0]
> +        reldir = os.path.join(os.path.dirname(relpath), "__pycache__")
> +        absdir = os.path.join(os.path.dirname(abspath), "__pycache__")
> +        try:
> +            for fname in (
> +                fn
> +                for fn in os.listdir(absdir)
> +                if fn.endswith(".pyc")
> +                and fn.rsplit(".", 2)[0] == basename

It's not an immediate concern because we don't have support for it (yet), but there are multiple optimization levels which pyc can be compiled for (1-3) [0][1].

I think (untested) the current rsplit will fail on an optimized file pattern and we can be a bit more future-proof so we don't accidentally run into this same problem again?

Maybe we can leverage `source_from_cache` [2] and compare the file names?

It's probably overkill, but I think this is one of those easily overlooked things where if we did add support for optimization levels that we'd miss fixing the code here to account for it.

Anyway, something to consider.

-Vincent

[0]: https://peps.python.org/pep-0488/
[1]: https://docs.python.org/3.15/using/cmdline.html#cmdoption-O
[2]: https://github.com/python/cpython/blob/20e6c2fc7c174342214d561845419c4030f8638f/Lib/importlib/_bootstrap_external.py#L299
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot