Re: [RFC PATCH 0/7] repack: add --drop-filtered to reclaim space in partial clones

On 16/07/26 18:58, Siddharth Shrimali wrote:
> This is an RFC series seeking feedback on the design and approach.
> Several pieces are still missing (noted below) and the commit
> organization needs cleanup.
> 
> Partial clones let you work with large repositories without downloading
> every blob up front and the missing blobs are lazily fetched from the promisor
> remote on demand. Over time, though, these lazily-fetched blobs
> accumulate locally and there is currently no safe, built-in way to
> reclaim that disk space instead of re-cloning.
> 
> This series adds a "git repack --drop-filtered --filter=<spec>" command
> that removes large, locally-held promisor blobs that are recoverable
> from the promisor remote. The dropped blobs become absent locally but
> remain lazily re-fetchable, making the partial-clone still reversible.
> 
> How it works:
>    * Enumerate promisor objects directly (ODB_FOR_EACH_OBJECT_PROMISOR_ONLY)
>      and select the blobs exceeding the filter threshold. Because every
>      enumerated object is a promisor object, it is guaranteed recoverable and
>      locally-created objects are never candidates.


This looks like the right approach to me. Going through the promisor 
repack path instead of write_filtered_pack() matches how repack already 
splits promisor objects out.


> 
>    * Rebuild the promisor pack without the selected blobs, reusing the
>      existing repack machinery, so the drop is crash-safe.
> 
>    * Record each dropped object in a drop log
>      ($GIT_DIR/objects/info/promisor-dropped) so a later change can
>      explain a failed lazy fetch (when it was dropped, which filter
>      matched, which remotes) instead of a bare "could not fetch" error.
> 
>    * --dry-run lists the candidates and changes nothing.
> 
> Planned follow-ups:
>    * Safety guards: refuse to run while a merge/rebase/cherry-pick is in
>      progress, and refuse to drop blobs referenced by the current index.


I think these matter before we present this as a real space-reclaim
tool. Without the index guard especially, users may drop blobs and then
immediately fetch them back on the next command that needs the worktree.

The drop log and remote-object-info can wait. I would not block the
next RFC round on them.

On the UI, I am fine with a separate --dry-run for now (same as
Christian). We can revisit a --drop-filtered=<mode> form later if we
grow more drop-specific options.

Thanks.
Siddharth


> 
>    * Authoritative remote verification: the drop log currently lists all
>      configured promisor remotes rather than the exact remote each object
>      is recoverable from, because there is no client-side way to query a
>      remote for object availability yet. A "remote-object-info" command
>      is being added to the "git cat-file --batch" protocol for this. Once
>      available, the exact remote can be recorded.
> 
> Known issues to address in v2:
>    * There is churn between "enumerate promisor blobs" and "actually drop
>      filtered promisor blobs". The former introduces
>      enumerate_promisor_blobs() with an interim signature that the latter
>      rewrites. These will be reorganized so the function is introduced
>      in its final form.
> 
>    * The tests are in a standalone commit. They will instead be
>      distributed into the commits that introduce the behavior they test.
> 
> Siddharth Shrimali (7):
>    builtin/repack.c: add --drop-filtered and --dry-run options
>    list-objects-filter: add list_objects_filter__filter_oidset()
>    repack-promisor: allow excluding objects from the rebuilt promisor
>      pack
>    builtin/repack: enumerate promisor blobs for --drop-filtered
>    t7706: test --drop-filtered enumeration and validation
>    builtin/repack: actually drop filtered promisor blobs
>    repack-promisor: record dropped objects in a drop log
> 
>   builtin/repack.c                |  76 ++++++++++++++++-
>   list-objects-filter.c           |  45 ++++++++++
>   list-objects-filter.h           |  16 ++++
>   repack-filtered.c               |  81 ++++++++++++++++++
>   repack-promisor.c               | 106 ++++++++++++++++++++++-
>   repack.h                        |  12 ++-
>   t/meson.build                   |   1 +
>   t/t7706-repack-drop-filtered.sh | 145 ++++++++++++++++++++++++++++++++
>   8 files changed, 478 insertions(+), 4 deletions(-)
>   create mode 100755 t/t7706-repack-drop-filtered.sh
>