Re: new test failures on Cygwin and OpenBSD

Hi Paul,

In order to understand the scope of the problem I asked:

> > in which locales the problem occurs:
> >    - in the "C" locale,
> >    - in single-byte locales other than "C",
> >    - or both?
> I've tested only OpenBSD 7.9. Apparently it supports only the C and the en_US.UTF-8 locales

I ran your experiment on glibc, Cygwin, OpenBSD (both with and without
the modules 'mbrtoc32-regular' and 'uchar-h-c23').

glibc:
C                  OK
fr_FR              OK

Cygwin:
C                  0xA0..0xFF: c32isprint = 1, isprint = 0
fr_FR.ISO8859-1    OK

OpenBSD:
C                  0xA0..0xFF: c32isprint = 1, isprint = 0

So, the problem exists only for the "C" locale.

It is related to the trouble we have with mbrtowc in the C locale:
https://sourceware.org/bugzilla/show_bug.cgi?id=19932
https://sourceware.org/bugzilla/show_bug.cgi?id=29511
https://lists.gnu.org/archive/html/bug-gnulib/2023-03/msg00145.html

> I see several possible fixes:
>
> (1) Skip the sed tests on problematic platforms.
>
> (2) Change Gnulib c32isprint so that in a single-byte locale, it succeeds if and only if given one of the at most 255 char32_t values corresponding to the unsigned char value that isprint succeeds on.
>
> (3) Replace mbrtoc32 on problematic platforms where in single-byte locales it can yield values that cause isprint to be inconsistent with c32isprint.
>
> (4) Change 'sed' so that in single-byte locales it uses only traditional <ctype.h> primitives like isprint, and never uses <uchar.h> functions.

(1) is not good. 'sed' is supposed to work alike on all platforms; the unit
tests are the means to ensuring that.

(2) is not good, IMO. char32_t values, starting with ISO C 23, are supposed
to be Unicode code points and therefore behave in a locale-independent way.
We need to overcome the old mess [1][2].

(4) This would be an ugly workaround. Similar work would then be needed in
other packages as well.

So, for me, (3) is the right approach: In the "C" locale, map 0x80..0xFF not
to U+0080..U+00FF but to U+DF80..U+DFFF, like musl libc does it. This will
ensure that c32isprint returns 0 on these inputs, like isprint does.

If you agree with this approach, I can implement it.

Bruno

[1] https://www.gnu.org/software/libunistring/manual/html_node/The-wchar_005ft-mess.html
[2] https://www.gnu.org/software/libunistring/manual/html_node/The-char32_005ft-problem.html