Re: [PATCH v4 0/6] odb: make packfile generation pluggable

Karthik Nayak <[email protected]> Fri, 21 Aug 2026 05:38:11 -0700
Newsgroups	org.kernel.vger.git
Message-ID	<CAOLa=ZQMjb1SzYTVVuMF0ajmre_5_q=L6bmSQwYY233f-RiVXA@mail.gmail.com>
Patrick Steinhardt <[email protected]> writes:

> Hi,
>
> this patch series makes packfile generation pluggable.
>
> Note that this series only makes those parts pluggable that are required
> for the transport layer. The other parts that relate to packfile
> generation as required by our repository maintenance is kept as-is, as
> there is a bunch of options there that are way too specific to the
> "files" backend to be portable. This should ultimately not be much of a
> problem though, as maintenance itself is already pluggable in the first
> place.
>
> It's a bit of a shame though for git-pack-objects(1), which still isn't
> usable with alternate backends. I tried several times to find good
> solutions for making it fully pluggable, but due to the backend-specific
> options it's an utter mess. I want to eventually address this though:
> same as with git-refs(1), I want to introduce git-objects(1) to care
> about all things ODB. And as part of that command we can also introduce
> a command that generates packfiles in a generic fashion, without all the
> cruft that git-pack-objects(1) has. This is part of a future patch
> series though.
>
> Changes in v4:
>   - Improve an error message.
>   - Sneak in a small stylistic fix while at it.
>   - Link to v3: https://patch.msgid.link/[email protected]
>
> Changes in v3:
>   - Fix a use-after-scope bug on abnormal exit when child processes are
>     cleaned up via `mark_child_for_cleanup()`, as noticed by Elijah.
>   - Link to v2: https://patch.msgid.link/[email protected]
>
> Changes in v2:
>   - Mostly remove the dependencies on `the_repository` in "bundle.c".
>   - Link to v1: https://patch.msgid.link/[email protected]
>
> The series is built on top of 2c78326f81 (The 11th batch, 2026-08-05).
>
> Thanks!
>
> Patrick
>
> ---
> Patrick Steinhardt (6):
>       odb: introduce interface to generate packfiles
>       upload-pack: generate packfiles via the object database
>       send-pack: generate packfiles via the object database
>       builtin/bundle: refactor option handling for progress meter
>       bundle: get (mostly) rid of `the_repository`
>       bundle: generate packfiles via the object database
>
>  builtin/bundle.c      |  34 +++++------
>  bundle.c              |  97 ++++++++++++++++++--------------
>  bundle.h              |   3 +-
>  odb.c                 |  21 +++++++
>  odb.h                 | 152 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  odb/source-files.c    | 149 +++++++++++++++++++++++++++++++++++++++++++++++++
>  odb/source.h          |  33 +++++++++++
>  send-pack.c           | 101 +++++++++++----------------------
>  t/t5516-fetch-push.sh |  12 ++--
>  upload-pack.c         | 125 +++++++++++++++--------------------------
>  10 files changed, 508 insertions(+), 219 deletions(-)
>
> Range-diff versus v3:
>
> 1:  4a56334af1 = 1:  33039a0ab8 odb: introduce interface to generate packfiles
> 2:  1ff0eaf6b7 ! 2:  7093fcee83 upload-pack: generate packfiles via the object database
>     @@ upload-pack.c: static void create_pack_file(struct upload_pack_data *pack_data,
>      -	 */
>      +		oid_array_append(&opts.haves,
>      +				 &pack_data->extra_edge_obj.objects[i].item->oid);
>     -+
>     +
>      +	opts.thin = pack_data->use_thin_pack;
>      +	if (!pack_data->no_progress)
>      +		opts.progress = ODB_GENERATE_PACK_PROGRESS_STANDARD;
>     @@ upload-pack.c: static void create_pack_file(struct upload_pack_data *pack_data,
>      +	opts.progress_fd = -1;
>      +
>      +	if (odb_generate_pack(the_repository->objects, &generator, &opts))
>     -+		die("git upload-pack: unable to fork git-pack-objects");
>     ++		die("git upload-pack: unable to generate pack");
>      +	odb_generate_pack_options_release(&opts);
>     -
>     ++
>      +	/*
>      +	 * We read from generator->err to capture stderr output for the
>      +	 * progress bar, and generator->out to capture the pack data.
> 3:  22a19a9a70 = 3:  0a2ca04c01 send-pack: generate packfiles via the object database
> 4:  5d2275c90b = 4:  2d339ee7b7 builtin/bundle: refactor option handling for progress meter
> 5:  0f00e6d234 = 5:  3cf0210247 bundle: get (mostly) rid of `the_repository`
> 6:  ae6af210ff ! 6:  d3345e4407 bundle: generate packfiles via the object database
>     @@ Commit message
>
>       ## builtin/bundle.c ##
>      @@ builtin/bundle.c: static int parse_options_cmd_bundle(int argc,
>     + }
>
>       static int cmd_bundle_create(int argc, const char **argv, const char *prefix,
>     - 			     struct repository *repo UNUSED) {
>     +-			     struct repository *repo UNUSED) {
>      -	struct strvec pack_opts = STRVEC_INIT;
>     ++			     struct repository *repo UNUSED)
>     ++{
>       	int progress = isatty(STDERR_FILENO);
>       	int version = -1;
>       	struct option options[] = {
>
> ---
> base-commit: 2c78326f810173a4f3aefd8021f1e07575412481
> change-id: 20260807-b4-pks-odb-generate-pack-f30fbcdef3fc

Everything looks good now. Thanks!
signature.asc (application/pgp-signature, 690 B)