Re: [docs] [PATCH v2 3/4] tools: add gen-doc-links to generate documentation link flags

Hi,

On Mon Jul 20, 2026 at 6:39 PM CEST, Quentin Schulz via lists.yoctoproject.=
org wrote:
> Hi Antonin,
>
> On 7/16/26 2:18 PM, Antonin Godard via lists.yoctoproject.org wrote:
>> The gen-doc-links utility can be used to generate a configuration to be
>> included in any project that want to get access to documentation links
>> for documented variables. For that it opens the objects.inv file
>> generate by our HTML build and creates links for existing variables and
>> tasks.
>>=20
>> Signed-off-by: Antonin Godard <[email protected]>
>> ---
>>   documentation/oecore/doclinks.conf | 1064 ++++++++++++++++++++++++++++=
++++++++
>>   documentation/tools/gen-doc-links  |  112 ++++
>>   2 files changed, 1176 insertions(+)
>>=20
> [...]
>> diff --git a/documentation/tools/gen-doc-links b/documentation/tools/gen=
-doc-links
>> new file mode 100755
>> index 000000000..11b3ef858
>> --- /dev/null
>> +++ b/documentation/tools/gen-doc-links
>> @@ -0,0 +1,112 @@
>> +#!/usr/bin/env python3
>> +#
>> +# SPDX-License-Identifier: MIT
>> +#
>> +# Author: Antonin Godard <[email protected]>
>> +#
>> +# Copyright (C) 2026 Bootlin
>> +#
>> +
>> +import argparse
>> +import sys
>> +
>> +from pathlib import Path
>> +from sphinx.util.inventory import InventoryFile
>> +
>> +
>> +YOCTO_DOCS_URL =3D "https://docs.yoctoproject.org/${LAYERSERIES_COMPAT_=
core}"
>> +BITBAKE_DOCS_URL =3D "https://docs.yoctoproject.org/bitbake/${LAYERSERI=
ES_COMPAT_core}"
>
> Sorry for the mangled link, but this is invalid. We only have bitbake=20
> version number, and not Yocto release name so this won't work.

/o\ you're right=E2=80=A6 good catch. Perhaps this will have to depend on t=
he
releases.json file which contains the mapping.

>> +DOCS_DIR =3D Path(__file__).parent.parent
>> +PREAMBLE =3D """#
>> +# This file is automatically generated with tools/gen-doc-links, do not=
 edit manually.
>> +#
>> +"""
>> +YOCTO_DOCS_SECTION =3D """
>> +# yocto-docs
>> +
>> +"""
>> +BITBAKE_SECTION =3D """
>> +# bitbake
>> +
>> +"""
>> +
>> +
>> +def parse_arguments() -> argparse.Namespace:
>> +    parser =3D argparse.ArgumentParser(description=3D"Generate glossari=
es from Sphinx text output")
>> +
>> +    parser.add_argument("--yocto-docs-inv",
>> +                        type=3DPath,
>> +                        default=3DDOCS_DIR / "_build/html/objects.inv",
>> +                        help=3D"Input yocto-docs inventory file")
>> +
>> +    parser.add_argument("--bitbake-inv",
>> +                        type=3DPath,
>> +                        default=3DDOCS_DIR / "_build/doctrees/__intersp=
hinx_cache__/bitbake_objects.inv",
>> +                        help=3D"Input bitbake inventory file")
>> +
>> +    parser.add_argument("-o", "--output",
>> +                        type=3DPath,
>> +                        default=3DDOCS_DIR / "oecore/doclinks.conf",
>> +                        help=3D"Output doclinks.conf file")
>> +
>> +    return parser.parse_args()
>> +
>> +
>> +def doclink(name: str, uri) -> str:
>> +    return f'{name}[doclink] =3D "{uri}"\n'
>> +
>> +
>> +def main():
>> +    args =3D parse_arguments()
>> +
>> +    yocto_docs_dict =3D {}
>> +
>> +    if not args.yocto_docs_inv.exists():
>> +        print(f"yocto-docs inventory not found at {args.yocto_docs_inv}=
, "
>> +              "use the --yocto-docs-inv option or set YOCTO_DOCS_INV_PA=
TH "
>> +              "in your environment")
>> +        sys.exit(0)
>> +
>> +    if not args.bitbake_inv.exists():
>> +        print(f"bitbake inventory not found at {args.yocto_docs_inv}, "
>> +              "use the --bitbake-inv option or set BITBAKE_INV_PATH "
>> +              "in your environment")
>> +        sys.exit(0)
>
> Don't you want to sys.exit(1) here (and above) instead to show the user=
=20
> this is an error?

I did not want to block creating a commit because the docs wasn't built.
I looked a bit at it and pre-commit does not have a special error code that
could signify a SKIP.

>> +
>> +    yocto_docs_data =3D args.yocto_docs_inv.read_bytes()
>> +    yocto_docs_inv =3D InventoryFile.loads(yocto_docs_data, uri=3DYOCTO=
_DOCS_URL)
>> +
>> +    yocto_docs_uris =3D (
>> +        f"{YOCTO_DOCS_URL}/ref-manual/variables.html#term-",
>> +        f"{YOCTO_DOCS_URL}/ref-manual/tasks.html#term-",
>> +    )
>> +
>> +    for key in sorted(yocto_docs_inv.data):
>> +        inv_entries =3D sorted(yocto_docs_inv.data[key].items())
>> +        for entry, inv_item in inv_entries:
>> +            if inv_item.uri.startswith(yocto_docs_uris):
>> +                yocto_docs_dict[entry] =3D inv_item.uri
>> +
>> +    bitbake_dict =3D {}
>> +    bitbake_data =3D args.bitbake_inv.read_bytes()
>> +    bitbake_inv =3D InventoryFile.loads(bitbake_data, uri=3DBITBAKE_DOC=
S_URL)
>> +
>> +    bitbake_uris =3D (
>> +        f"{BITBAKE_DOCS_URL}/bitbake-user-manual/bitbake-user-manual-re=
f-variables.html#term-",
>> +    )
>> +
>> +    for key in sorted(bitbake_inv.data):
>> +        inv_entries =3D sorted(bitbake_inv.data[key].items())
>> +        for entry, inv_item in inv_entries:
>> +            if inv_item.uri.startswith(bitbake_uris) and entry not in y=
octo_docs_dict:
>> +                bitbake_dict[entry] =3D inv_item.uri
>> +
>
> 1. Does the uri need to be YOCTO_DOCS_URL? We only need to have it in=20
> the output file, which we can add in the write calls below. But we could=
=20
> also simply have the empty ('') string. This should make it faster to=20
> compare (fewer letters to compare against).

True, that's what I did first but then changed it. I'll revert that.

> 2. Why do we need to check that it startswith()? I'm assuming you want=20
> to avoid the ref-manual/terms.html entries? Please document this with a=
=20
> comment.

Will do.

> 3. Can't you avoid the first for-loop and directly iterate over=20
> bitbake_inv.data['std:term'].items() since that's the only thing we're=20
> interested in?

Good point, I'll simplify that.

>> +    with open(args.output, "w") as links_conf:
>> +        links_conf.write(PREAMBLE)
>> +        links_conf.write(YOCTO_DOCS_SECTION)
>> +        [links_conf.write(doclink(name, uri)) for name, uri in yocto_do=
cs_dict.items()]
>> +        links_conf.write(BITBAKE_SECTION)
>> +        [links_conf.write(doclink(name, uri)) for name, uri in bitbake_=
dict.items()]
>
> I'm also a bit concerned about the use of $(LAYERSERIES_COMPAT_core} as=
=20
> this may induce some mismatch between what OE-Core is at and the version=
=20
> used to generate this file. We may actually document a variable that=20
> shouldn't be used anymore, or that isn't accessible yet due to a=20
> different major version being used. We do know what we're building the=20
> docs for (c.f. conf.py) so maybe we should hardcode that?

I did not think of simply using the current version of the documentation an=
d
hardcore that in the file. It's actually probably safer, I agree. I'll see =
how I
can change that, thanks!

Antonin