Re: [PATCH] packfile: fix perf regression with many packs

Ben Knoble <[email protected]>
Newsgroups org.kernel.vger.git
Message-ID <[email protected]>
> Le 12 août 2026 à 15:15, Johannes Schindelin via GitGitGadget <[email protected]> a écrit :
> 
> From: Johannes Schindelin <[email protected]>
> 
> Since 589127caa730 (packfile: move list of packs into the packfile
> store, 2025-10-30), there is a performance regression when many
> packfiles need to be loaded: `packfile_store_add_pack()` now calls
> `packfile_list_remove_internal()` to detect whether the packfile was
> _already_ in the list, if if so, move it to the end of the list. This
> function linearly scans the existing list before every insertion. Newly
> loading N packs therefore has complexity O(N²).
> 
> In one reported use case (https://github.com/microsoft/git/issues/970),
> N equals 37,815 and caused a slow-down of a simple `git rev-parse
> --short HEAD` (which is regularly executed as part of `GIT_PS1`) from
> 0.4s to 4.5s. In another, heavily exercised CI scenario, clone times
> increased from under 2 minutes to over half an hour.
> 
> Let's fix this by establishing a fast path for known-new packfiles.
> 
> The keen reader will note that there is currently only a single,
> "known-new" caller of the `packfile_list_append()` function, and wonder
> why not simply remove this check whether the packfile already exists in
> the list? Originally, when above-mentioned commit introduced that logic,
> there was a second caller in `prepare_midx()`, which would have required
> that check, but that caller was removed in 6aff1f25a046 (packfile:
> always add packfiles to MRU when adding a pack, 2025-10-30). Still, the
> function is declared in a header file, and to avoid any problems with
> in-flight or downstream callers, it is safer to extend the signature to
> be explicit whether or not to skip that check.
> 
> Signed-off-by: Johannes Schindelin <[email protected]>
> ---
>    packfile: fix perf regression with many packs
> 
>    This issue was spotted by a Microsoft Git user with the massive amount
>    of packfiles typical of an average, long-running monorepo checkout.

As a different kind of intermediate solution, would turning on maintenance for that user’s checkout help? (Not sure that would help CI clone times unless the server repacks, of course.)
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.