Re: [PATCH] index-pack: speed up promisor link recording

--5oeHElVf8Oqz8MHf
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2026-08-02 at 21:33:15, Arijit Banerjee via GitGitGadget wrote:
> From: Arijit Banerjee <[email protected]>
>=20
> When indexing a promisor pack, index-pack parses every reconstructed
> non-blob object into the shared object model to record its outgoing links.
> Since parse_object_buffer() runs under read_mutex, worker threads seriali=
ze
> while allocating persistent tree, commit, and tag structures that are only
> needed to enumerate those links.
>=20
> Read the links directly from the reconstructed object buffers instead. Ke=
ep
> the strict and fsck paths unchanged, use worker-local typed oidmaps during
> normal promisor indexing, and merge them after the workers exit. Transfer
> entries during the merge so that it does not temporarily duplicate the
> complete link set.
>=20
> The typed entries preserve checks previously performed as a side effect of
> object parsing. Reject malformed commit and tag headers, conflicting
> expected types, and targets whose actual type disagrees when the target is
> present in the pack. Preserve commit-graft handling and the existing poli=
cy
> of recording only subtree entries from trees.
>=20
> With three runs per version on Debian 12, median end-to-end wall-clock ti=
me
> for a --filter=3Dblob:none clone of linux.git decreased from 156 seconds =
to
> 133 seconds (15%). Trace2 attributed the change to the initial index-pack
> --promisor phase, whose median duration decreased from 121 seconds to 98
> seconds (19%). System CPU time decreased by 46%.
>=20
> Two paired spot checks against GitHub showed end-to-end reductions of 18%
> and 26%. These measurements include network and server variability and are
> therefore corroborating rather than controlled results. A third pair was =
not
> interpretable because the baseline request encountered a transport stall.
>=20
> A full-clone control showed no material change, taking approximately 256
> seconds with either version. This is expected because full clones do not
> exercise promisor-link recording.
>=20
> t5302-pack-index.sh passed with both SHA-1 and SHA-256, while
> t0410-partial-clone.sh and t5616-partial-clone.sh also passed. New covera=
ge
> checks malformed commit headers, conflicting link types, and mismatched t=
ag
> target types.
>=20
> Signed-off-by: Arijit Banerjee <[email protected]>
> ---
>     index-pack: speed up promisor link recording
>=20
>     AI assistance: OpenAI Codex was used to identify the bottleneck and
>     assist with the implementation, testing, and benchmark analysis. I
>     reviewed the resulting change and take responsibility for this
>     submission.

I don't think SubmittingPatches really allows more than trivial changes
written by AI:

    The Developer's Certificate of Origin requires contributors to certify
    that they know the origin of their contributions to the project and
    that they have the right to submit it under the project's license.
    It's not yet clear that this can be legally satisfied when submitting
    significant amount of content that has been generated by AI tools.

    [...]

    To avoid these issues, we will reject anything that looks AI
    generated, that sounds overly formal or bloated, that looks like AI
    slop, that looks good on the surface but makes no sense, or that
    senders don=E2=80=99t understand or cannot explain.

This doesn't look like it's a trivial change, so I don't believe this
patch can be accepted.
--=20
brian m. carlson (they/them)
Toronto, Ontario, CA

--5oeHElVf8Oqz8MHf
Content-Type: application/pgp-signature; name=signature.asc

-----BEGIN PGP SIGNATURE-----

wr0EABYKAG8Fgmpvu/8JEHwMSWKIh6KBRxQAAAAAAB4AIHNhbHRAbm90YXRpb25z
LnNlcXVvaWEtcGdwLm9yZ5rw2cErxSZcPebEqPNsBOEgSPGjuWegHh41ZBdlqjC3
FiEECCzmip28ZfuD0cORfAxJYoiHooEAAAixAP4/n+sYKiOTzVx5uQX132qS+ivY
I/mEWRbgHfB5iA2pbgD/VPOCx8MHCZ4Denc35gJabiCcggq+/e1NWy1BW9tVpgU=
=9o/6
-----END PGP SIGNATURE-----

--5oeHElVf8Oqz8MHf--