Re: Performance regression in connectivity check during receive-pack (git 2.54)

On 26/07/24 05:46AM, Patrick Steinhardt wrote:
> This would solve a bunch of smaller issues that we're currently
> grappling with where some of the concepts in Git really want to operate
> across all of the alternates:
> 
>   - The OBJECT_INFO_SECOND_READ flag can be dropped as it becomes an
>     implementation detail.
> 
>   - We can fix the performance regression because we can now easily
>     reorder access to read via packfiles first across all sub-sources.

Letting the backend control the ordering would be a nice property.

>   - Commit graphs and bitmap really are a singleton, so loading them via
>     multiple sources is awkward.

I completely agree. Having the ODB source be more self-contained with
the alternates better fits the shape of commits graphs and alternates
IMO.

>   - The object storage extension that I've written got quite a bit ugly
>     as it wasn't quite clear where exactly to draw the line. Especially
>     hadnling the environment variables mentioned above into the "files"
>     backend removes one point of friction I encountered.
> 
>   - Object database maintenance needs to be aware of the other non-local
>     sources.
> 
> Also, doing that change isn't as bad as it may sound at first. We'd
> still retain the whole `struct odb_source` list because we want to have
> them for submodule sources. Furthermore, alternates aren't required for
> isolation either as we currently use them via the temporary object
> directory. An alternative implementation may use a completely separate
> mechanism to achieve write isolation, which is also why we have made the
> environment variables pluggable that the `struct odb_transaction` ends
> up passing to the child process.

Once all temporary object directory users are updated to use ODB
transactions, we could stop reording source list when starting/ending a
transaction. Instead the transaction could be tracked separately
internally and during ODB read/writes the transaction could be directly
used as needed. This is something I plan to tackle in a future series
soon.

> I think overall this could simplify some of the design, and it makes a
> bunch of issues that I have been struggling with go away. The devil may
> be in the details of course, but I think transitioning towards this
> should be doable.

I am certainly a fan of this direction. We do lose some flexibility in
terms of supporting alternates more generically, but I'm not sure
supporting alternates of different source types in the same repo would
be something we want anyways in practice due to the additional
complexity.

-Justin