Re: [yocto] Documentation around adding custom fetchers to layer

On 16 Sep 2025, at 21:04, seth.raymond via lists.yoctoproject.org <[email protected]> wrote:
> 
> Hello,
>   I am attempting to write a couple of custom fetcher that pull secrets from our password manager (using its CLI). To start, I wrote Python classes that inherit from bb.fetch2.FetchMethod and registered them in poky/bitbake/lib/bb/fetch2/__init__.py by importing them and appending them to methods. I confirmed that the fetchers themselves are working as expected and the secrets are getting pulled during the recipe build process.
>   Of course, I don't (necessarily) want to contribute these fetchers upstream to poky. I would like to add them to one of our company's layers. Our directory structure is laid out like:
...
> PYTHONPATH .= ":${LAYERDIR}/lib"
>   When building recipes that use the fetchers, I get parsing errors saying that no fetchers could be found to support the URL:
> 
> bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception NoMethodError: Could not find a fetcher which supports the URL: 'optemplate://aes_key.tpl;output=aes.key'
> The variable dependency chain for the failure is: fetcher_hashes_dummyfunc[vardepvalue]
> 
> I'm not seeing any documentation on how to add a custom fetcher to a layer, and the fetchers in poky don't provide enough information on how to register these fetchers. Is this functionality supported, or will we have to fork poky and keep it up-to-date to add a new fetcher?

This is a bitbake issue, so I’ve copied in the bitbake list.

There’s no standard way to add out-of-bitbake fetchers, as you say you need to add the fetcher instance to bb.fetch2.methods.

I guess you might be able to write a do_fetch prefunc that creates an instance of your fetcher and adds it to the list, but I would not be surprised if there were more places where that would be needed.

python use_my_fetcher() {
  bb.fetch2.methods.append(MyFetcher())
}
do_fetch[prefuncs] += “use_my_fetcher”

Ross