Re: [PATCH v2 0/4] odb: eagerly load alternates

Karthik Nayak <[email protected]> Thu, 13 Aug 2026 05:28:18 -0700
Newsgroups	org.kernel.vger.git
Message-ID	<CAOLa=ZTLq2xjkC12B=4wPJA9UJ1PWS4_iUAyqgcJ7GYnbM2YrQ@mail.gmail.com>
Patrick Steinhardt <[email protected]> writes:

> Hi,
>
> when initializing the object database we only eagerly initialize the
> primary object database source. If the primary source has alternates,
> those alternates are only initialized the first time we really access
> the object database.
>
> When introduced in ace1534d6f (Introduce SHA1_FILE_DIRECTORIES to
> support multiple object databases., 2005-05-07), alternates were
> originally only loaded when a given object wasn't found in the primary
> object database. This was also reinforced by later optimization, for
> example in 693d2bc625 (Attempt to delay prepare_alt_odb during get_sha1,
> 2007-05-26), where we tried to avoid loading alternates in even more
> cases. But as Git has evolved, we eventually started to eagerly parse
> alternates all over the codebase, including on every single object
> lookup, and consequently deferring this operation does not really buy us
> much anymore.
>
> The result of this is that we have calls to `odb_prepare_alternates()`
> cluttered all over the code base. This is somewhat awkward, and as
> almost every Git command ends up reading objects at it doesn't even buy
> us anything.
>
> This patch series thus gets rid of the lazy-loading. Besides simplifying
> the codebase a bit, it also prepares us for moving alternates into the
> "files" backend as discussed in [1].
>
> The series is built on top of 010afd3166 (The 12th batch, 2026-08-07)
> with ps/odb-make-creation-pluggable at e927cfeb21 (odb: make creation of
> on-disk structures pluggable, 2026-08-07) merged into it.
>
> Changes in v2:
>   - Add a missing word to a commit message.
>   - Explain why we don't have to handle GIT_ALTERNATE_OBJECT_DIRECTORIES
>     when re-preparing the object database.
>   - Link to v1: https://patch.msgid.link/20260810-pks-odb-eagerly-prepare-alternates-v1-0-f0fa4a4004e1@pks.im
>

V2 looks good, I have one nit, but it's not work re-rolling :)

> Thanks!
>
> Patrick
>
> [1]: <[email protected]>
>
> ---
> Patrick Steinhardt (4):
>       odb: decouple source path comparisons from `the_repository`
>       odb: eagerly initialize alternates
>       odb: drop `loaded_alternates` field
>       odb: drop `alternates_db` field
>
>  builtin/fsck.c         |   3 --
>  builtin/pack-objects.c |   3 --
>  commit-graph.c         |   4 --
>  loose.c                |   1 -
>  object-name.c          |   1 -
>  odb.c                  | 109 ++++++++++++++++++++++++-------------------------
>  odb.h                  |  22 +++++-----
>  odb/source.h           |   7 ++++
>  odb/streaming.c        |   1 -
>  pack-bitmap.c          |   2 -
>  packfile.c             |   1 -
>  packfile.h             |   2 -
>  12 files changed, 70 insertions(+), 86 deletions(-)
>
> Range-diff versus v1:
>
> 1:  25802adffa = 1:  721907c60d odb: decouple source path comparisons from `the_repository`
> 2:  1e73b730d8 ! 2:  3b2c23566c odb: eagerly initialize alternates
>     @@ Commit message
>          many calls to `odb_prepare_alternates()` cluttered around the code base
>          whenever we are about to iterate through the sources.
>
>     -    This lazy loading doesn't really add much value: the moment where read
>     -    any object we _have_ to load the alternates anyway. So given that most
>     -    of our commands would access the object database this optimization is
>     -    not really buying us much in the first place. Quite on the contrary, it
>     -    makes the code harder to understand and is a potential source of bugs in
>     -    case any callsite forgot to prepare alternates before we iterate through
>     -    the sources.
>     +    This lazy loading doesn't really add much value: the moment where we
>     +    read any object we _have_ to load the alternates anyway. So given that
>     +    most of our commands would access the object database this optimization
>     +    is not really buying us much in the first place. Quite on the contrary,
>     +    it makes the code harder to understand and is a potential source of bugs
>     +    in case any callsite forgot to prepare alternates before we iterate
>     +    through the sources.
>
>          Historically though there was a reason why we deferred lazy-loading: it
>          may happen that the repository has "core.ignoreCase" configured, and we
> 3:  2ca1aa2a37 = 3:  df5d7df91d odb: drop `loaded_alternates` field
> 4:  1e97c93bdf ! 4:  50a37ef385 odb: drop `alternates_db` field
>     @@ odb.c: void odb_free(struct object_database *o)
>       	pthread_mutex_destroy(&o->replace_mutex);
>
>      @@ odb.c: void odb_prepare(struct object_database *o, enum odb_prepare_flags flags)
>     - 	 * the lifetime of the process.
>     + 	 * Reprepare alt odbs, in case the alternates file was modified
>     + 	 * during the course of this process. This only _adds_ odbs to
>     + 	 * the linked list, so existing odbs will continue to exist for
>     +-	 * the lifetime of the process.
>     ++	 * the lifetime of the process. Consequently, we don't have to
>     ++	 * reprocess GIT_ALTERNATE_OBJECT_DIRECTORIES here.
>       	 */
>       	if (flags & ODB_PREPARE_FLUSH_CACHES) {
>      -		odb_prepare_alternates(o);
>
> ---
> base-commit: f6ad67a7977439ad8351d42e6ccfd11f714db765
> change-id: 20260804-pks-odb-eagerly-prepare-alternates-3efb0a38e0dd
signature.asc (application/pgp-signature, 690 B)