Re: portability issues with unicodeio
Bruno Haible <[email protected]>
| Newsgroups | gmane.comp.lib.gnulib.bugs,gmane.comp.parsers.bison.bugs |
|---|---|
| Message-ID | <5129694.dEMNA6ixTA@omega> |
Hi Akim, Kiyoshi-san, > Unfortunately on Kiyoshi's environment (SunOS hidden 5.11 11.3 i86pc i386 i86pc, > GCC 9.3.0) we get '?' instead of '.' in the C locale. We get a genuine ASCII > '?', it's not some fallback from the terminal which fails to display the > character. And we properly get the bullet with en_US.UTF-8. I am adding a unit test for the 'unicodeio' module (see below). It contains a test for the C locale, and this test works fine on Solaris 11.3. Therefore I have to ask Kiyoshi two questions: 1) What is the output of the program 'locale'? $ locale 2) What are the link dependencies of the coreutils printf program? $ ldd $coreutilsbin/printf Bruno 2020-07-08 Bruno Haible <[email protected]> unicodeio: Add tests. * tests/test-unicodeio.c: New file. * tests/test-unicodeio1.sh: New file. * tests/test-unicodeio2.sh: New file. * tests/test-unicodeio3.sh: New file. * modules/unicodeio-tests: New file. 2020-07-08 Bruno Haible <[email protected]> unicodeio: Document link requirements. * modules/unicodeio (Link): New section.
0001-unicodeio-Document-link-requirements.patch
(text/x-patch, 1000 B)
From 77157f813248fb7dfd4d854ab85d107a1fe026b6 Mon Sep 17 00:00:00 2001 From: Bruno Haible <[email protected]> Date: Wed, 8 Jul 2020 20:00:00 +0200 Subject: [PATCH 1/2] unicodeio: Document link requirements. * modules/unicodeio (Link): New section. --- ChangeLog | 5 +++++ modules/unicodeio | 3 +++ 2 files changed, 8 insertions(+) diff --git a/ChangeLog b/ChangeLog index 9cac500..aa36674 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2020-07-08 Bruno Haible <[email protected]> + + unicodeio: Document link requirements. + * modules/unicodeio (Link): New section. + 2020-07-07 Bruno Haible <[email protected]> doc: Remove support for some very old platforms. diff --git a/modules/unicodeio b/modules/unicodeio index 8164ac4..51721db 100644 --- a/modules/unicodeio +++ b/modules/unicodeio @@ -24,6 +24,9 @@ lib_SOURCES += unicodeio.h unicodeio.c Include: "unicodeio.h" +Link: +$(LTLIBICONV) when linking with libtool, $(LIBICONV) otherwise + License: GPL -- 2.7.4
0002-unicodeio-Add-tests.patch
(text/x-patch, 5.6 KB)
From 7f89a25089f411dce4b22b55000de75c4bffe524 Mon Sep 17 00:00:00 2001 From: Bruno Haible <[email protected]> Date: Wed, 8 Jul 2020 20:01:36 +0200 Subject: [PATCH 2/2] unicodeio: Add tests. * tests/test-unicodeio.c: New file. * tests/test-unicodeio1.sh: New file. * tests/test-unicodeio2.sh: New file. * tests/test-unicodeio3.sh: New file. * modules/unicodeio-tests: New file. --- ChangeLog | 9 ++++++ modules/unicodeio-tests | 21 +++++++++++++ tests/test-unicodeio.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/test-unicodeio1.sh | 5 +++ tests/test-unicodeio2.sh | 15 +++++++++ tests/test-unicodeio3.sh | 15 +++++++++ 6 files changed, 146 insertions(+) create mode 100644 modules/unicodeio-tests create mode 100644 tests/test-unicodeio.c create mode 100755 tests/test-unicodeio1.sh create mode 100755 tests/test-unicodeio2.sh create mode 100755 tests/test-unicodeio3.sh diff --git a/ChangeLog b/ChangeLog index aa36674..c2182f0 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,14 @@ 2020-07-08 Bruno Haible <[email protected]> + unicodeio: Add tests. + * tests/test-unicodeio.c: New file. + * tests/test-unicodeio1.sh: New file. + * tests/test-unicodeio2.sh: New file. + * tests/test-unicodeio3.sh: New file. + * modules/unicodeio-tests: New file. + +2020-07-08 Bruno Haible <[email protected]> + unicodeio: Document link requirements. * modules/unicodeio (Link): New section. diff --git a/modules/unicodeio-tests b/modules/unicodeio-tests new file mode 100644 index 0000000..d771ff8 --- /dev/null +++ b/modules/unicodeio-tests @@ -0,0 +1,21 @@ +Files: +tests/test-unicodeio1.sh +tests/test-unicodeio2.sh +tests/test-unicodeio3.sh +tests/test-unicodeio.c +tests/macros.h +m4/locale-fr.m4 +m4/locale-zh.m4 +m4/codeset.m4 + +Depends-on: +setlocale + +configure.ac: +gt_LOCALE_FR_UTF8 +gt_LOCALE_ZH_CN + +Makefile.am: +TESTS += test-unicodeio1.sh test-unicodeio2.sh test-unicodeio3.sh +check_PROGRAMS += test-unicodeio +test_unicodeio_LDADD = $(LDADD) $(LIB_SETLOCALE) $(LIBICONV) diff --git a/tests/test-unicodeio.c b/tests/test-unicodeio.c new file mode 100644 index 0000000..be91a91 --- /dev/null +++ b/tests/test-unicodeio.c @@ -0,0 +1,81 @@ +/* Tests for Unicode character output. + + Copyright (C) 2020 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see <https://www.gnu.org/licenses/>. */ + +/* Written by Bruno Haible, 2020. */ + +#include <config.h> + +/* Specification. */ +#include "unicodeio.h" + +#include <locale.h> +#include <string.h> + +#include "macros.h" + +#define TEST_CODE 0x2022 +#define TEST_CODE_AS_UTF8 "\xe2\x80\xa2" +#define TEST_CODE_AS_GB18030 "\x81\x36\xa6\x31" + +static char result[64]; + +static long +success_callback (const char *buf, size_t buflen, void *callback_arg) +{ + memcpy (result, buf, buflen); + result[buflen] = '\0'; + return 42; +} + +static long +failure_callback (unsigned int code, const char *msg, void *callback_arg) +{ + ASSERT (code == TEST_CODE); + strcpy (result, "."); + return 55; +} + +int +main (int argc, char *argv[]) +{ + /* configure should already have checked that the locale is supported. */ + if (setlocale (LC_ALL, "") == NULL) + return 1; + + switch (unicode_to_mb (TEST_CODE, success_callback, failure_callback, NULL)) + { + case 42: + if (argc > 1) + switch (argv[1][0]) + { + case '2': + ASSERT (strcmp (result, TEST_CODE_AS_UTF8) == 0); + break; + case '3': + ASSERT (strcmp (result, TEST_CODE_AS_GB18030) == 0); + break; + } + break; + case 55: + ASSERT (strcmp (result, ".") == 0); + break; + default: + ASSERT (0); + } + + return 0; +} diff --git a/tests/test-unicodeio1.sh b/tests/test-unicodeio1.sh new file mode 100755 index 0000000..5f5f1bd --- /dev/null +++ b/tests/test-unicodeio1.sh @@ -0,0 +1,5 @@ +#!/bin/sh + +# Test in the C locale. +LC_ALL=C \ +${CHECKER} ./test-unicodeio${EXEEXT} 1 diff --git a/tests/test-unicodeio2.sh b/tests/test-unicodeio2.sh new file mode 100755 index 0000000..f0ec8fb --- /dev/null +++ b/tests/test-unicodeio2.sh @@ -0,0 +1,15 @@ +#!/bin/sh + +# Test whether a specific UTF-8 locale is installed. +: ${LOCALE_FR_UTF8=fr_FR.UTF-8} +if test $LOCALE_FR_UTF8 = none; then + if test -f /usr/bin/localedef; then + echo "Skipping test: no french Unicode locale is installed" + else + echo "Skipping test: no french Unicode locale is supported" + fi + exit 77 +fi + +LC_ALL=$LOCALE_FR_UTF8 \ +${CHECKER} ./test-unicodeio${EXEEXT} 2 diff --git a/tests/test-unicodeio3.sh b/tests/test-unicodeio3.sh new file mode 100755 index 0000000..1544c96 --- /dev/null +++ b/tests/test-unicodeio3.sh @@ -0,0 +1,15 @@ +#!/bin/sh + +# Test whether a specific GB18030 locale is installed. +: ${LOCALE_ZH_CN=zh_CN.GB18030} +if test $LOCALE_ZH_CN = none; then + if test -f /usr/bin/localedef; then + echo "Skipping test: no transitional chinese locale is installed" + else + echo "Skipping test: no transitional chinese locale is supported" + fi + exit 77 +fi + +LC_ALL=$LOCALE_ZH_CN \ +${CHECKER} ./test-unicodeio${EXEEXT} 3 -- 2.7.4