RFC: mbrtowc() and wcrtomb() implementation problems

Keith Marshall <[email protected]> Wed, 12 Feb 2020 22:03:17 +0000

Newsgroups	gmane.comp.gnu.mingw.user
Organization	MinGW.org Project
Message-ID	<[email protected]>

Hello, MinGW Users,

I've been following up on https://osdn.net/projects/mingw/ticket/39687,
and I've identified a possible resolution for the reported problem; I've
also identified several other problems with the original implementation,
some of which I can easily fix, but some are more problematic.

FTR, Danny Smith contributed the original implementation, back in 2005;
it does indeed exhibit the reported fault, (which is also reflected in
the complementary mbsrtowcs() implementation).  Other problems, which
I've identified in _all_ related functions are:

1) There is no true MBCS support; only conversions between wchar_t and
DBCS codesets (corresponding to MB_LEN_MAX = 2) are supported.  Today,
Windows supports MBCS codesets with MB_LEN_MAX = 5.

2) Representation of wchar_t as UTF-16LE entities makes it impossible to
have effective handling, in mbrtowc() and wcrtomb(), for code points
which lie off the BMP.  Such code points are represented by surrogate
pairs, in UTF-16, and there is no standards conformant mechanism for
passing such surrogate pairs through the single wchar_t argument to
either of these functions.

I can address the first of these issues; the second is more problematic.
 I can imagine subverting use of the mbstate_t reference argument, to
store intermediate state for partially completed surrogate conversions,
but what would be an appropriate function return value, in such cases?
Alternatively, should valid surrogate pair representations just be
reported as invalid, for the purpose of such conversions?

What do you, the users of MinGW, think?

-- 
Regards,
Keith.

Public key available from keys.gnupg.net
Key fingerprint: C19E C018 1547 DE50 E1D4 8F53 C0AD 36C6 347E 5A3F

_______________________________________________
MinGW-Users mailing list
[email protected]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.osdn.me/mailman/listinfo/mingw-users
Also: mailto:[email protected]?subject=unsubscribe

signature.asc (application/pgp-signature, 833 B)