[PATCH GSoC v21 00/13] cat-file: add remote-object-info to batch-command

This patch series is a continuation of Eric Ju's
([email protected]) and Calvin Wan's ([email protected]) patch
series [1] and [2] respectively.

Sometimes it is beneficial to retrieve information about an object
without having to download it completely. The server logic for
retrieving size has already been implemented and merged in a2ba162cda
(object-info: support for retrieving object info, 2021-04-20) [3].
This patch series implements the client option for it.

Eric's series adds the remote-object-info command to cat-file
--batch-command. This command allows the client to make an object-info
command request to a server that supports protocol v2.

If the server uses protocol v2 but does not support the object-info
capability, cat-file --batch-command will die.
If a user attempts to use remote-object-info with protocol v1, cat-file
--batch-command will die.

Currently, only the size (%(objectsize)) is supported end to end in this
implementation. The type (%(objecttype)) is known by the client's
allow-list and request path but is not supported on the server side
nor the response parsing. A follow up series will add full end-to-end
support for %(objecttype).

The default format for remote-object-info is set to "%(objectname)
%(objectsize)". Once %(objecttype) is supported, the default format will
be unified accordingly.

If the batch command format includes unsupported fields such as
%(objecttype), %(objectsize:disk), or %(deltabase), the command will
return empty strings for each unsupported field.

This series completes Eric's work mainly with the refactor of the
validation of the placeholders with an allow-list that filters what the
client asks with what the server is capable of providing, following Jeff
King's idea [4].

GitHub CI: https://github.com/pabloosabaterr/git/actions/runs/30086202054

[1]: https://lore.kernel.org/git/[email protected]/
[2]: https://lore.kernel.org/git/[email protected]/#t
[3]: https://git.kernel.org/pub/scm/git/git.git/commit/?id=a2ba162cda2acc171c3e36acbbc854792b093cb7
[4]: https://lore.kernel.org/git/[email protected]/

Changes in v21:
- Dropped unnecessary change in git-cat-file.adoc
- Fixed EXPAND_DATA_INIT macro formating
- typo and blank line before WARNING:

---
Calvin Wan (3):
      fetch-pack: move fetch initialization
      serve: advertise object-info feature
      transport: add client support for object-info

Eric Ju (3):
      cat-file: declare loop counter inside for()
      t1006: extract helper functions into new 'lib-cat-file.sh'
      cat-file: add remote-object-info to batch-command

Pablo Sabater (7):
      transport-helper: fix memory leak of helper on disconnect
      fetch-pack: drop the static advertise_sid variable
      fetch-pack: use unsigned int for hash_algo variable
      fetch-pack: move write_fetch_command_and_capabilities() to connect.c
      connect: make write_fetch_command_and_capabilities() more generic
      protocol-caps: check object existence regardless of the attributes requested
      cat-file: make remote-object-info allow-list adapt to the server

 Documentation/git-cat-file.adoc        |  23 +-
 Documentation/gitprotocol-v2.adoc      |  21 +-
 Makefile                               |   1 +
 builtin/cat-file.c                     | 224 +++++++++-
 connect.c                              |  34 ++
 connect.h                              |   8 +
 fetch-object-info.c                    | 156 +++++++
 fetch-object-info.h                    |  25 ++
 fetch-pack.c                           |  58 +--
 meson.build                            |   1 +
 object-file.c                          |  10 +
 odb.h                                  |   9 +
 protocol-caps.c                        |  45 +-
 serve.c                                |   5 +-
 t/lib-cat-file.sh                      |  16 +
 t/meson.build                          |   1 +
 t/t1006-cat-file.sh                    |  15 +-
 t/t1017-cat-file-remote-object-info.sh | 747 +++++++++++++++++++++++++++++++++
 t/t5701-git-serve.sh                   |  63 +++
 transport-helper.c                     |  12 +-
 transport-internal.h                   |   8 +
 transport.c                            |  44 ++
 transport.h                            |   9 +
 23 files changed, 1444 insertions(+), 91 deletions(-)

Range-diff versus v20:

 1:  ddc6386af3 =  1:  6c868e0e29 transport-helper: fix memory leak of helper on disconnect
 2:  24e0c07dde =  2:  40579c0095 cat-file: declare loop counter inside for()
 3:  08882a9ddd =  3:  b70d1f91c4 t1006: extract helper functions into new 'lib-cat-file.sh'
 4:  0925c208c1 =  4:  5c5050d541 fetch-pack: drop the static advertise_sid variable
 5:  f130092dd9 =  5:  29c321ad3e fetch-pack: use unsigned int for hash_algo variable
 6:  37326255e9 =  6:  bdcd407a22 fetch-pack: move write_fetch_command_and_capabilities() to connect.c
 7:  069b9d4def =  7:  970d59e2cf connect: make write_fetch_command_and_capabilities() more generic
 8:  4225d3087c =  8:  86c64d10d7 fetch-pack: move fetch initialization
 9:  af3b2bb293 =  9:  7007808819 protocol-caps: check object existence regardless of the attributes requested
10:  99df8374df = 10:  2f5acaa540 serve: advertise object-info feature
11:  380bca2a43 = 11:  abb9306254 transport: add client support for object-info
12:  eb48e94f6f ! 12:  36fe34a596 cat-file: add remote-object-info to batch-command
    @@ Documentation/git-cat-file.adoc: one per line, and print information based on th
      `contents` command followed by an object prints contents in the same way
     -`--batch` would.
     +`--batch` would. The `remote-object-info` command followed by a remote and
    -+objects IDs prints object info from the remote without downloading the objects.
    ++object IDs prints object info from the remote without downloading the objects.

      You can specify the information shown for each object by using a custom
      `<format>`. The `<format>` is copied literally to stdout for each
    -@@ Documentation/git-cat-file.adoc: newline. The available atoms are:
    - `deltabase`::
    - 	If the object is stored as a delta on-disk, this expands to the
    - 	full hex representation of the delta base object name.
    --	Otherwise, expands to the null OID (all zeroes). See `CAVEATS`
    -+	Otherwise, expands to the null OID (all zeroes). See `CAVEATS` section
    - 	below.
    -
    - `rest`::
     @@ Documentation/git-cat-file.adoc: newline. The available atoms are:
      	after that first run of whitespace (i.e., the "rest" of the
      	line) are output in place of the `%(rest)` atom.
    @@ Documentation/git-cat-file.adoc: newline. The available atoms are:
      If no format is specified, the default format is `%(objectname)
     -%(objecttype) %(objectsize)`.
     +%(objecttype) %(objectsize)`, except for `remote-object-info` commands which
    -+use `%(objectname) %(objectsize)` because "%(objecttype)" is not supported yet.
    ++use `%(objectname) %(objectsize)` because `%(objecttype)` is not supported yet.
    ++
     +WARNING: When "%(objecttype)" is supported, the default format WILL be unified,
     +so DO NOT RELY on the current default format to stay the same!!!

    @@ builtin/cat-file.c: static const struct parse_cmd {
      static void batch_objects_command(struct batch_options *opt,

      ## object-file.c ##
    -@@ object-file.c: struct odb_transaction *odb_transaction_files_begin(struct odb_source *source)
    +@@ object-file.c: int odb_transaction_files_begin(struct odb_source *source,

    - 	return &transaction->base;
    + 	return 0;
      }
     +
     +void free_object_info_contents(struct object_info *object_info)
13:  75b1c61c67 ! 13:  71e19e8d27 cat-file: make remote-object-info allow-list adapt to the server
    @@ builtin/cat-file.c: struct expand_data {
     +	struct string_list remote_allowed_atoms;
      };

    -+#define EXPAND_DATA_INIT  { .mode = S_IFINVALID, \
    -+			    .type = OBJ_BAD, \
    -+			    .remote_allowed_atoms = STRING_LIST_INIT_NODUP }
    ++#define EXPAND_DATA_INIT  {  \
    ++	.mode = S_IFINVALID, \
    ++	.type = OBJ_BAD,     \
    ++	.remote_allowed_atoms = STRING_LIST_INIT_NODUP, \
    ++}
     +
      static int is_atom(const char *atom, const char *s, int slen)
      {

---
base-commit: 9a0c4701dcd5725c4184599322b52933ff5005ca
change-id: 20260608-ps-eric-work-rebase-b73ae84ba671