Re: [PATCH] odb/files: be less aggressive with geometric repacking

On Thu, Aug 20, 2026 at 11:40:46PM -0700, Elijah Newren wrote:
> On Tue, Aug 11, 2026 at 2:17 AM Patrick Steinhardt <[email protected]> wrote:
[snip]
> > Being this aggressive is also causing problems as reported by our users.
> > When running lots of concurrent writers, those writes will constantly
> > end up spawning maintenance jobs that end up repacking objects. As we
> > also prune objects, a concurrently running process that tries to write
> > an object may see that the sharding directories get removed under their
> > feet. While we try re-creating such leading directories, we only do so a
> > single time, and it may happen that the directory vanishes again before
> > we had the chance to create the loose object. This is not a new problem,
> > but it is exacerbated by us running maintenance this aggressively.
> 
> Unrelated to this patch...but should git avoid pruning the loose
> object sharding directories?

I was wondering about that, too. There are two contradicting arguments
to make here:

  - Pruning the sharding directories allows us to quickly determine that
    an empty shard cannot have an object.

  - Not pruning the sharding directories may avoid a lot of write churn.

The question is how large the impact of these two individual arguments
is.

By gut feeling, I think that the first argument is somewhat weak. Not
having empty directories means that looking up a loose object by its
path will be slightly faster because we have to walk one less directory
in the hierarchy. But this really only matters in the case where we look
for a nonexistent object, which does not happen all that often because
we prefer searching packfiles first.

Furthermore, iterating through all objects in the object database will
be faster, as we don't have to open each of the directories only to find
them empty. But again, that's not really something that we do all that
frequently.

On the other hand, we _do_ have to recreate the loose object shards
quite frequently as that's how we write data into a repository. And as
we've seen, pruning those shards can easily cause races.

So in the end I think it could be a useful thing to explore. The only
thing I wonder is whether there's a good reason for why we prune those
that I miss.

Patrick