bug#79444: baseenc SIGSEGV on macOS

Bruno Haible <[email protected]> writes:

> Here's the relevant output on macOS 13:
>
> empty1d...
> Running command: 'basenc --base64 -d 'empty1d.1' > empty1d.O 2> empty1d.E'
> sh: line 1: 79138 Segmentation fault: 11  basenc --base64 -d 'empty1d.1' > empty1d.O 2> empty1d.E
> basenc.pl: test empty1d failed: exit status mismatch:  expected 0, got 139
> creating file 'empty2d.1' with contents ''
> creating file 'empty2d.2' with contents ''
> creating file 'empty2d.3' with contents ''

Thanks.

Looks like it was caused by this commit:

commit 4141ae3e4970dbc206b56366b0581f04fa87dfc1
Author:     Pádraig Brady <[email protected]>
AuthorDate: Mon Sep 8 17:40:00 2025 +0100
Commit:     Pádraig Brady <[email protected]>
CommitDate: Wed Sep 10 23:24:46 2025 +0100

    maint: basenc: refactor all encodings to use finalize
    
    Finalize was required for base58, but it's a more general mechanism
    which simplifies the logic for all encodings
    
    * src/basenc.c (do_decode): Always call base_decode_ctx_finalize(),
    rather than the awkward double loop at end of buffer.
    * tests/basenc/basenc.pl: Add basenc finalization tests.1

Focusing on this change:

-          ok = base_decode_ctx (&ctx, inbuf, sum, outbuf, &n);
+          if (sum)
+            ok = base_decode_ctx (&ctx, inbuf, sum, outbuf, &n);
+          else
+            ok = base_decode_ctx_finalize (&ctx, &outbuf, &n);

For an empty file 'sum' would be zero and base_decode_ctx is never
called. For --base32 here is what that function pointer calls:

static bool
base32_decode_ctx_wrapper (struct base_decode_context *ctx,
                           char const *restrict in, idx_t inlen,
                           char *restrict out, idx_t *outlen)
{
  bool b = base32_decode_ctx (&ctx->ctx.base32, in, inlen, out, outlen);
  ctx->i = ctx->ctx.base32.i;
  return b;
}

If that is never called, using ctx->i is undefined behavior. Therefore,
the padding can cause a crash:

static bool
base32_ctx_has_padding (struct base_decode_context *ctx)
{
  return ctx->i && ctx->ctx.base32.buf[ctx->i - 1] == '=';
}

On the other systems we probably get lucky and ctx->i is set to zero.

That is my hypothesis at least, will need to write a patch to test it.

Collin