Re: [PATCH] packfile: fix perf regression with many packs
Ben Knoble <[email protected]>
| Newsgroups | org.kernel.vger.git |
|---|---|
| Message-ID | <[email protected]> |
> Le 12 août 2026 à 15:15, Johannes Schindelin via GitGitGadget <[email protected]> a écrit : > > From: Johannes Schindelin <[email protected]> > > Since 589127caa730 (packfile: move list of packs into the packfile > store, 2025-10-30), there is a performance regression when many > packfiles need to be loaded: `packfile_store_add_pack()` now calls > `packfile_list_remove_internal()` to detect whether the packfile was > _already_ in the list, if if so, move it to the end of the list. This > function linearly scans the existing list before every insertion. Newly > loading N packs therefore has complexity O(N²). > > In one reported use case (https://github.com/microsoft/git/issues/970), > N equals 37,815 and caused a slow-down of a simple `git rev-parse > --short HEAD` (which is regularly executed as part of `GIT_PS1`) from > 0.4s to 4.5s. In another, heavily exercised CI scenario, clone times > increased from under 2 minutes to over half an hour. > > Let's fix this by establishing a fast path for known-new packfiles. > > The keen reader will note that there is currently only a single, > "known-new" caller of the `packfile_list_append()` function, and wonder > why not simply remove this check whether the packfile already exists in > the list? Originally, when above-mentioned commit introduced that logic, > there was a second caller in `prepare_midx()`, which would have required > that check, but that caller was removed in 6aff1f25a046 (packfile: > always add packfiles to MRU when adding a pack, 2025-10-30). Still, the > function is declared in a header file, and to avoid any problems with > in-flight or downstream callers, it is safer to extend the signature to > be explicit whether or not to skip that check. > > Signed-off-by: Johannes Schindelin <[email protected]> > --- > packfile: fix perf regression with many packs > > This issue was spotted by a Microsoft Git user with the massive amount > of packfiles typical of an average, long-running monorepo checkout. As a different kind of intermediate solution, would turning on maintenance for that user’s checkout help? (Not sure that would help CI clone times unless the server repacks, of course.)