Re: [PATCH v3 0/4] Introduce fdt_overlay_merge() to allow merge of overlay blobs

On Mon, May 19, 2025 at 10:15:55AM -0700, Trilok Soni wrote:
> On 5/19/2025 2:10 AM, Wasim Nazir wrote:
> > Hello,
> > 
> > This is follow-up attempt for fdtoverlaymerge tool.
> 
> Please add detail of the first attempt, since we are submitting it after a long time. 
> 
> > 
> > Currently all the device-tree (DT) code for a given soc is maintained in a
> > common kernel repository. For example, this common DT code will have code for
> > audio, video, fingerprint, bluetooth etc. Further this, DT code is typically
> > split into a base (soc-common) code and board specific code, with the soc code
> > being compiled as soc.dtb and board specific code being compiled as respective
> > overlay blobs (board1.dtbo, board2.dtbo etc). soc.dtb represents hardware configuration
> > of a given SOC while boardX.dtbo represents configuration of a board/platform
> > designed using that soc.soc.dtb and boardX.dtbo files are flashed separately on
> > target (besides improving the overall size of DT blobs flashed on target, Android
> > Treble also requires separation of soc and board DT bits). Bootloader will pick
> > one of the board overlay blobs and merge it with soc.dtb, before booting kernel
> > which is presented a unified DT blob (soc + board overlay).
> 
> Vatsa wrote it as per the Qualcomm source structure in the Android at that time. 
> You can see that he is referring "common" kernel repository etc; You need
> to rewrite it or explicitly mention it per Qualcomm directory structure
> in Android or QCLinux plus the tech-packages / external driver modules we have. 
> 
> You need to translate the problem which others outside Qualcomm can understand
> and relate in-order to understand these patches. 
> 

Sure, will modify the source-structure add provide more details.

> 
> > 
> > For ease of code maintenance and better control over release management, we are
> > exploring allowing some of the tech teams (audio/fingerprint sensor etc) to
> 
> "tech teams" is a internal usage within Qualcomm and it will be hard to understand
> some times outside QC. Please rewrite. 

Sure.

> 
> > maintain their kernel code (including their DT code) outside a common kernel
> > repository. In our experience, this simplifies number of branches maintained in
> > core kernel repo. New/experimental features in fingerprint sensor driver for
> 
> "kernel repo" is also bit QC specific. I will stop reviewing this patch
> since you should consider rewriting this summary. 

Will send updated summary.

> 
> > example that needs to be on a separate branch will not result in unnecessary
> > branching in core kenrel repo, affecting all other drivers.
> > 
> > In addition to compiling DT code outside core kernel tree, we also want to merge
> > the blobs back to respective blobs found in kernel build tree at buildtime
> > (soc.dtb or boardX.dtbo), as otherwise relying on bootloader to do all the
> > overlay impacts boot-time.
> > 
> > This brings up the need to merge two overlay blobs (fingerprint-overlay.dtbo +
> > boardX.dtbo), which currently doesn't seem to be supported and which this patch
> > series aims to support.
> > 
> > fdt_overlay_apply() API currently allows for an overlay DT blob to be merged
> > with a base blob. It assumes that all external symbols specified in overlay
> > blob's __fixups__ section are found in base blob's __symbols__ section and
> > aborts on the first instance where a symbol could not be found in base blob.
> > This is mostly fine as the primary use of overlay is on a target for its
> > bootloader to merge various overlay blobs based on h/w configuration detected.
> > But when the number of overlays increased then bootloader takes lot of time to
> > apply the overlays on base DT.
> > 
> > So we need new API/tool to merge all the overlays into single overlay file
> > at host (build machine) side, so that on target side bootloader needs to only
> > apply merged-overlay-dt to its base-dt. This saves lot of time due to reduced
> > number file reading/loading & minimizing repeatative overlay apply.
> > In our test setup we see an improvement of ~60% while applying merged-overlay
> > at bootloader and the merged-overlay is product of 7 overlays.
> > 
> > To serve this overlay-merge feature we have introduce fdtoverlaymerge tool
> > which takes input as overlays and gives output to merged-overlay.
> > The tool uses fdt_overlay_merge() API introduced in libfdt to do the actual work.
> > 
> > Additional notes:
> >   If snprintf (in libc) may not available in some environments, then we will need
> >   to write our own snprintf() in libfdt.
> > 
> > ---
> > Changelog:
> > 
> > v3:
> > - Update copy_node & add copy_fragment_to_base to incorporate two cases i.e
> >   - Case1: When target is available and we merge fragments
> >   - Case2: When target is not available and we add new fragments
> > - Change the logic to update fixups & local_fixups in case of overlay merge.
> > - Few patches are squashed, reduced to 4 patches.
> > - v2-link: https://lore.kernel.org/all/[email protected]/
> > 
> > 
> > Srivatsa Vaddagiri (4):
> >   libfdt: overlay_merge: Introduce fdt_overlay_merge()
> >   libfdt: overlay_merge: Rename & copy overlay fragments and their
> >     properties
> >   libfdt: overlay_merge: Update phandles, symbols, fixups & local_fixups
> >   fdtoverlaymerge: A tool that merges overlays
> > 
> >  .gitignore           |   1 +
> >  Makefile             |   4 +
> >  Makefile.utils       |   6 +
> >  fdtoverlaymerge.c    | 223 +++++++++++
> >  libfdt/fdt_overlay.c | 901 ++++++++++++++++++++++++++++++++++++++++++-
> >  libfdt/fdt_rw.c      |  14 +-
> >  libfdt/libfdt.h      |  18 +
> >  libfdt/version.lds   |   1 +
> >  meson.build          |   2 +-
> >  9 files changed, 1146 insertions(+), 24 deletions(-)
> >  create mode 100644 fdtoverlaymerge.c
> > 
> > 
> > base-commit: f4c53f4ebf7809a07666bf728c823005e1f1a612
> > --
> > 2.49.0
> > 
> 

Regards,
Wasim