Re: isspace() breaks encoding in rfc2047.c on some systems

On Tue, 16 Mar 2010, Urs Janßen wrote:

> On Tue, Mar 16, 2010 at 08:40:33PM +0100, Dennis Preiser wrote:
>> I encountered the following issue:
>>
>> On my system (MacOSX 10.4, powerpc-apple-darwin8.11.0) isspace()
>> returns TRUE for '0xa0' (I use an UTF-8 locale). 0xa0 can be the second
>> byte of a two byte UTF-8 character, for instance:
>>
>> ??	0xc5 0xa0
>>
>> This causes the isbetween() macro in rfc2047.c to return TRUE and the
>> various encoding functions fail. For quoted-printable and the character
>> above in the subject it looks like this:
>>
>> Subject: =?UTF-8?Q?=C5?=
>>
>> followed by 0xA0. (The internal inews complains about unencoded 8-bit
>> chars, which is correct.)
>>
>> Here some code to check the behaviour of isspace() (returns TRUE with an
>> UTF-8 locale on my system):
>>
>> #include <stdio.h>
>> #include <ctype.h>
>> #include <locale.h>
>>
>> int
>> main(
>> 	void)
>> {
>> 	unsigned char ch = 0xa0;
>>
>> 	setlocale(LC_ALL, "");
>> 	printf("isspace(0xa0) = %s\n", isspace(ch) ? "TRUE" : "FALSE");
>> 	return 0;
>> }
>>
>> According to google (quick check with 'isspace(0xa0)'), there seems to
>> be some systems whith such a behaviour. MacOSX, (older?) BSD and linux
>> with an older glibc version.
>>
>> I think we have to decide whether it is worse to be fixed and if so, how.
>
> I'd use a configure check for it (e.g. your code above, but with UTF-8 as
> locale in setlocale) to check for broken isspace(). at least when not
> crosscompiling this should give us exact results. Thomas?

I've been using the wctype.h macros where they're available.

There's some differences though - I recall that Linux and Solaris differ
regarding whether the 128-255 are "printable" in UTF-8 mode.

-- 
Thomas E. Dickey
http://invisible-island.net
ftp://invisible-island.net