Re: [PATCH v3 0/4] Introduce fdt_overlay_merge() to allow merge of overlay blobs

On 5/19/2025 2:10 AM, Wasim Nazir wrote:
> Hello,
> 
> This is follow-up attempt for fdtoverlaymerge tool.

Please add detail of the first attempt, since we are submitting it after a long time. 

> 
> Currently all the device-tree (DT) code for a given soc is maintained in a
> common kernel repository. For example, this common DT code will have code for
> audio, video, fingerprint, bluetooth etc. Further this, DT code is typically
> split into a base (soc-common) code and board specific code, with the soc code
> being compiled as soc.dtb and board specific code being compiled as respective
> overlay blobs (board1.dtbo, board2.dtbo etc). soc.dtb represents hardware configuration
> of a given SOC while boardX.dtbo represents configuration of a board/platform
> designed using that soc.soc.dtb and boardX.dtbo files are flashed separately on
> target (besides improving the overall size of DT blobs flashed on target, Android
> Treble also requires separation of soc and board DT bits). Bootloader will pick
> one of the board overlay blobs and merge it with soc.dtb, before booting kernel
> which is presented a unified DT blob (soc + board overlay).

Vatsa wrote it as per the Qualcomm source structure in the Android at that time. 
You can see that he is referring "common" kernel repository etc; You need
to rewrite it or explicitly mention it per Qualcomm directory structure
in Android or QCLinux plus the tech-packages / external driver modules we have. 

You need to translate the problem which others outside Qualcomm can understand
and relate in-order to understand these patches. 


> 
> For ease of code maintenance and better control over release management, we are
> exploring allowing some of the tech teams (audio/fingerprint sensor etc) to

"tech teams" is a internal usage within Qualcomm and it will be hard to understand
some times outside QC. Please rewrite. 

> maintain their kernel code (including their DT code) outside a common kernel
> repository. In our experience, this simplifies number of branches maintained in
> core kernel repo. New/experimental features in fingerprint sensor driver for

"kernel repo" is also bit QC specific. I will stop reviewing this patch
since you should consider rewriting this summary. 

> example that needs to be on a separate branch will not result in unnecessary
> branching in core kenrel repo, affecting all other drivers.
> 
> In addition to compiling DT code outside core kernel tree, we also want to merge
> the blobs back to respective blobs found in kernel build tree at buildtime
> (soc.dtb or boardX.dtbo), as otherwise relying on bootloader to do all the
> overlay impacts boot-time.
> 
> This brings up the need to merge two overlay blobs (fingerprint-overlay.dtbo +
> boardX.dtbo), which currently doesn't seem to be supported and which this patch
> series aims to support.
> 
> fdt_overlay_apply() API currently allows for an overlay DT blob to be merged
> with a base blob. It assumes that all external symbols specified in overlay
> blob's __fixups__ section are found in base blob's __symbols__ section and
> aborts on the first instance where a symbol could not be found in base blob.
> This is mostly fine as the primary use of overlay is on a target for its
> bootloader to merge various overlay blobs based on h/w configuration detected.
> But when the number of overlays increased then bootloader takes lot of time to
> apply the overlays on base DT.
> 
> So we need new API/tool to merge all the overlays into single overlay file
> at host (build machine) side, so that on target side bootloader needs to only
> apply merged-overlay-dt to its base-dt. This saves lot of time due to reduced
> number file reading/loading & minimizing repeatative overlay apply.
> In our test setup we see an improvement of ~60% while applying merged-overlay
> at bootloader and the merged-overlay is product of 7 overlays.
> 
> To serve this overlay-merge feature we have introduce fdtoverlaymerge tool
> which takes input as overlays and gives output to merged-overlay.
> The tool uses fdt_overlay_merge() API introduced in libfdt to do the actual work.
> 
> Additional notes:
>   If snprintf (in libc) may not available in some environments, then we will need
>   to write our own snprintf() in libfdt.
> 
> ---
> Changelog:
> 
> v3:
> - Update copy_node & add copy_fragment_to_base to incorporate two cases i.e
>   - Case1: When target is available and we merge fragments
>   - Case2: When target is not available and we add new fragments
> - Change the logic to update fixups & local_fixups in case of overlay merge.
> - Few patches are squashed, reduced to 4 patches.
> - v2-link: https://lore.kernel.org/all/[email protected]/
> 
> 
> Srivatsa Vaddagiri (4):
>   libfdt: overlay_merge: Introduce fdt_overlay_merge()
>   libfdt: overlay_merge: Rename & copy overlay fragments and their
>     properties
>   libfdt: overlay_merge: Update phandles, symbols, fixups & local_fixups
>   fdtoverlaymerge: A tool that merges overlays
> 
>  .gitignore           |   1 +
>  Makefile             |   4 +
>  Makefile.utils       |   6 +
>  fdtoverlaymerge.c    | 223 +++++++++++
>  libfdt/fdt_overlay.c | 901 ++++++++++++++++++++++++++++++++++++++++++-
>  libfdt/fdt_rw.c      |  14 +-
>  libfdt/libfdt.h      |  18 +
>  libfdt/version.lds   |   1 +
>  meson.build          |   2 +-
>  9 files changed, 1146 insertions(+), 24 deletions(-)
>  create mode 100644 fdtoverlaymerge.c
> 
> 
> base-commit: f4c53f4ebf7809a07666bf728c823005e1f1a612
> --
> 2.49.0
>