Re: [PATCH 1/2] iconvdata: SHIFT_JISX0213 lacks pending character reset (CVE-2026-77117)

On 8/21/26 6:12 AM, Florian Weimer wrote:
> This fixes bug 34556.

LLM-assisted review found what you already noted in the ticket that
euc-jisx0213.c has the same flaw, but I'll ignore this for now since
that flaw will get it's own bug and a new reserved CVE ID.

Reviewed-by: Carlos O'Donell <[email protected]>

> ---
>   iconvdata/shift_jisx0213.c | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/iconvdata/shift_jisx0213.c b/iconvdata/shift_jisx0213.c
> index 61c9c3ce6d..e9179f605e 100644
> --- a/iconvdata/shift_jisx0213.c
> +++ b/iconvdata/shift_jisx0213.c
> @@ -226,6 +226,9 @@
>   	    STANDARD_FROM_LOOP_ERR_HANDLER (1);				      \
>   	  }								      \
>         }									      \
> +    else 								      \
> +      /* There was a pending character.  Clear it.  */			      \
> +      *statep = 0;							      \

OK. We load the pending into ch, and then clear it, and then output it.

This is required because:

(a) EMIT_SHIFT_TO_INIT is only triggered when flushing the stream, and
     it would emit the pending character, but it is not required to flush
     between calls.

(b) BODY must be able to handle a restart according to the POSIX wording:
     "Subsequent calls with inbuf as other than a null pointer or a pointer
      to a null pointer cause the conversion to take place from the current
      state of the conversion descriptor."

The CVE fix looks good to me.

There is another bug present here that results in data loss and should
probably get it's own bug filed...

What happens if the combining character is the last character?

In such a case loop.c skips calling BODY (because there is no input to
consume) in that case and returns a success but drops the combining
character even if there is room, and this leads to a data loss scenario?

LLM-assisted test case:

   FAIL: iconvdata/tst-shift-jisx0213-e2big-loss
   info: with flush:    produced 2 code point(s): U+304B U+309A
   info: without flush: produced 1 code point(s): U+304B
   tst-shift-jisx0213-e2big-loss.c:165: error: blob comparison failed
     left  (no_flush):  4B 30 00 00                        ← only U+304B
     right (expected):  4B 30 00 00 9A 30 00 00            ← U+304B U+309A

Shall we consider this another distinct bug since data is lost?

~~~
/* Test SHIFT_JISX0213 combining-character loss across an E2BIG boundary.
    Copyright (C) 2026 Free Software Foundation, Inc.
    This file is part of the GNU C Library.

    The GNU C Library is free software; you can redistribute it and/or
    modify it under the terms of the GNU Lesser General Public
    License as published by the Free Software Foundation; either
    version 2.1 of the License, or (at your option) any later version.

    The GNU C Library is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
    Lesser General Public License for more details.

    You should have received a copy of the GNU Lesser General Public
    License along with the GNU C Library; if not, see
    <https://www.gnu.org/licenses/>.  */

/* SHIFT_JISX0213 bytes 0x82 0xF5 map to a combining sequence of two UCS-4
    code points: U+304B (HIRAGANA LETTER KA) + U+309A (COMBINING SEMI-VOICED
    SOUND MARK).  When the output buffer has room for only the first code
    point, the converter emits U+304B, consumes the whole input character, and
    queues U+309A in the converter state, returning E2BIG.

    This exercises the case where the combining character is the *last* input
    unit and the application continues per the POSIX E2BIG protocol - drain the
    output already produced, supply a larger output buffer, and call iconv()
    again with the same descriptor - WITHOUT issuing a terminating
    iconv (cd, NULL, ...) flush.  POSIX does not require such a flush to
    continue or complete a conversion after E2BIG.

    Because all input was consumed by the first call, the continuation call
    never re-enters the conversion loop body, so the queued U+309A is never
    emitted: iconv () reports success with all input consumed, but the second
    code point is silently dropped.  It surfaces only if the application
    happens to issue a flush.

    A correct conversion of the complete input must yield both code points
    regardless of where the output-buffer boundary falls, so the check at the
    end of do_test asserts that both are produced without a flush.  Against a
    buggy library this comparison FAILS, demonstrating the data loss; the
    accompanying with-flush pass confirms the byte was merely parked in the
    converter state rather than corrupted.  */

#include <errno.h>
#include <iconv.h>
#include <stdio.h>
#include <string.h>
#include <wchar.h>

#include <support/check.h>
#include <support/support.h>

/* The single combining character, and nothing after it.  */
static const char sjis_combining[] = { '\x82', '\xf5' };

/* Its complete expansion in WCHAR_T (UTF-32, host endianness).  */
static const wchar_t expected[] = { 0x304b, 0x309a };

/* Convert the input described above using an output window of WINDOW bytes,
    following
    the POSIX E2BIG-continuation pattern.  If FLUSH is nonzero, a terminating
    iconv (cd, NULL, ...) flush is issued after all input is consumed.  The
    produced bytes are appended to RESULT (of RESULT_SIZE bytes); the total
    number of bytes produced is returned.  */
static size_t
convert (size_t window, char *result, size_t result_size, int flush)
{
   iconv_t cd = iconv_open ("WCHAR_T", "SHIFT_JISX0213");
   TEST_VERIFY_EXIT (cd != (iconv_t) -1);

   char window_buf[8];
   TEST_VERIFY_EXIT (window <= sizeof window_buf);

   char *inptr = (char *) sjis_combining;
   size_t inleft = sizeof sjis_combining;
   size_t result_len = 0;
   int guard = 0;

   /* Ordinary application loop: feed input, and on E2BIG drain the produced
      output and retry with a fresh output window.  Stop once all input has
      been consumed.  No flush happens here.  */
   while (inleft > 0)
     {
       char *outptr = window_buf;
       size_t outleft = window;

       errno = 0;
       size_t r = iconv (cd, &inptr, &inleft, &outptr, &outleft);
       size_t produced = outptr - window_buf;

       TEST_VERIFY_EXIT (result_len + produced <= result_size);
       memcpy (result + result_len, window_buf, produced);
       result_len += produced;

       if (r == (size_t) -1)
     {
       if (errno == E2BIG)
         {
           /* Detect a non-progress / infinite-loop failure mode.  */
           TEST_VERIFY_EXIT (++guard < 16);
           continue;
         }
       FAIL_EXIT1 ("window %zu: iconv: %m", window);
     }

       /* Success: the entire input has been consumed.  */
       break;
     }

   TEST_COMPARE (inleft, 0);

   if (flush)
     {
       char *outptr = window_buf;
       size_t outleft = window;

       errno = 0;
       size_t r = iconv (cd, NULL, NULL, &outptr, &outleft);
       TEST_VERIFY_EXIT (r != (size_t) -1);
       size_t produced = outptr - window_buf;

       TEST_VERIFY_EXIT (result_len + produced <= result_size);
       memcpy (result + result_len, window_buf, produced);
       result_len += produced;
     }

   TEST_VERIFY_EXIT (iconv_close (cd) == 0);
   return result_len;
}

static void
report (const char *what, const char *result, size_t len)
{
   size_t ncp = len / sizeof (wchar_t);
   printf ("info: %s: produced %zu code point(s):", what, ncp);
   for (size_t i = 0; i < ncp; i++)
     printf (" U+%04X", (unsigned int) ((const wchar_t *) result)[i]);
   putchar ('\n');
}

static int
do_test (void)
{
   /* An output window of exactly one UCS-4 code point (4 bytes) forces the
      boundary to fall between U+304B and U+309A.  */
   enum { WINDOW = sizeof (wchar_t) };

   /* With a terminating flush the queued code point is recovered: this shows
      the byte is parked in converter state, not corrupted.  */
   char with_flush[64];
   size_t with_flush_len = convert (WINDOW, with_flush, sizeof with_flush, 1);
   report ("with flush", with_flush, with_flush_len);
   TEST_COMPARE_BLOB (with_flush, with_flush_len, expected, sizeof expected);

   /* Without a flush - the POSIX continuation pattern - the second code point
      of the combining sequence is dropped.  */
   char no_flush[64];
   size_t no_flush_len = convert (WINDOW, no_flush, sizeof no_flush, 0);
   report ("without flush", no_flush, no_flush_len);

   /* The conversion of the complete input must produce both code points even
      though the application issued no flush.  This FAILS on a library that
      drops the queued U+309A, demonstrating the data loss.  */
   TEST_COMPARE_BLOB (no_flush, no_flush_len, expected, sizeof expected);

   return 0;
}

#include <support/test-driver.c>
~~~

>   									      \
>       put32 (outptr, ch);							      \
>       outptr += 4;							      \
> 
> base-commit: 6467136459cc86a4426e35ac8d8f9de4673b9f1f


-- 
Cheers,
Carlos.