[PATCH v2 0/8] odb: unify read and write streams

Hi,

we have two different kind of object database streams in our code base:
`odb_write_stream` and `odb_read_stream`. While those are used for
different use cases, the provided functionality is ultimately the exact
same.

This patch series thus refactors these streams so that we have a single
`odb_stream`, only. This allows us to reuse the streams for different
kinds of purposes and makes them more generally useful overall. For
example, it's trivially possible now to create an object stream for any
given object and then write that stream into a different source.

The series is built on top of 5b2471720c (The 10th batch, 2026-08-03).

Changes in v2:
  - Use the correct object type when hashing in-memory objects.
  - Remove a stale comment.
  - Adapt a commit message to mention that renames will follow in
    subsequent commits.
  - Add another commit to rename `struct input_zstream_data`.
  - Link to v1: https://patch.msgid.link/[email protected]

Thanks!

Patrick

---
Patrick Steinhardt (8):
      odb/streaming: track write stream size in the structure
      odb/streaming: drop `is_finished` field
      odb/streaming: support streaming arbitrary object types
      odb/streaming: rename `struct odb_read_stream`
      odb/streaming: consolidate read and write streams
      odb/streaming: rename `struct read_object_fd_data`
      odb/streaming: rename `struct input_zstream_data`
      odb/streaming: unify function names to create new streams

 archive-tar.c                 |   8 ++--
 archive-zip.c                 |  12 ++---
 builtin/index-pack.c          |   8 ++--
 builtin/pack-objects.c        |  18 ++++----
 builtin/unpack-objects.c      |  44 ++++++++++--------
 object-file.c                 |  76 +++++++++++++++---------------
 object-file.h                 |   2 +-
 object.c                      |   6 +--
 odb.c                         |   4 +-
 odb.h                         |   4 +-
 odb/source-files.c            |   7 ++-
 odb/source-inmemory.c         |  35 ++++++++------
 odb/source-loose.c            |  33 ++++++++------
 odb/source-packed.c           |   5 +-
 odb/source.h                  |  13 +++---
 odb/streaming.c               | 104 ++++++++++++++++++++----------------------
 odb/streaming.h               |  69 ++++++++++------------------
 odb/transaction.c             |   6 +--
 odb/transaction.h             |   8 ++--
 pack-check.c                  |   4 +-
 packfile.c                    |   8 ++--
 packfile.h                    |   4 +-
 t/unit-tests/u-odb-inmemory.c |  37 ++++++++-------
 23 files changed, 251 insertions(+), 264 deletions(-)

Range-diff versus v1:

1:  0085df877f = 1:  1966710c12 odb/streaming: track write stream size in the structure
2:  5fbbfd9010 = 2:  87c7981a6c odb/streaming: drop `is_finished` field
3:  52e5b87761 ! 3:  9aede44fba odb/streaming: support streaming arbitrary object types
    @@ object-file.c: int index_fd(struct index_state *istate, struct object_id *oid,
     
      ## odb/source-inmemory.c ##
     @@ odb/source-inmemory.c: static int odb_source_inmemory_write_object_stream(struct odb_source *source,
    - 	hash_object_file(source->odb->repo->hash_algo, data, total_read, OBJ_BLOB, oid);
    + 		goto out;
    + 	}
    + 
    +-	hash_object_file(source->odb->repo->hash_algo, data, total_read, OBJ_BLOB, oid);
    ++	hash_object_file(source->odb->repo->hash_algo, data, total_read,
    ++			 stream->type, oid);
      
      	ret = odb_source_inmemory_write_object(source, data, stream->size,
     -					       OBJ_BLOB, oid, NULL, NULL, 0);
    @@ odb/streaming.h: int odb_stream_blob_to_fd(struct object_database *odb,
      
      #endif /* STREAMING_H */
     
    + ## odb/transaction.h ##
    +@@ odb/transaction.h: struct odb_transaction {
    + 
    + 	/*
    + 	 * This callback is expected to write the given object stream into
    +-	 * the ODB transaction. Note that for now, only blobs support streaming.
    ++	 * the ODB transaction.
    + 	 *
    + 	 * The resulting object ID shall be written into the out pointer. The
    + 	 * callback is expected to return 0 on success, a negative error code
    +
      ## t/unit-tests/u-odb-inmemory.c ##
     @@ t/unit-tests/u-odb-inmemory.c: void test_odb_inmemory__write_object_stream(void)
      	struct odb_source_inmemory *source = odb_source_inmemory_new(odb);
4:  f178d441f0 = 4:  ca84a2b645 odb/streaming: rename `struct odb_read_stream`
5:  0d72d27078 ! 5:  838394bffc odb/streaming: consolidate read and write streams
    @@ Commit message
         new `struct odb_stream` base. Other than that though, the changes are
         rather straight forward.
     
    +    Some of the structures and functions are now somewhat misnamed. These
    +    will be fixed in subsequent commits.
    +
         Signed-off-by: Patrick Steinhardt <[email protected]>
     
      ## builtin/unpack-objects.c ##
6:  ced59bdc85 = 6:  850b7e081d odb/streaming: rename `struct read_object_fd_data`
-:  ---------- > 7:  c3fe9f8b0c odb/streaming: rename `struct input_zstream_data`
7:  f76f4350ef = 8:  4df81651ba odb/streaming: unify function names to create new streams

---
base-commit: 5b2471720c93ee30e5764a19f3d3b3ae9ec9712a
change-id: 20260724-pks-odb-stream-unification-334dc2a75888