Re: Invalid inputs FAILED (input.at:102) on Solaris

On December 8, 2020 7:55 am Akim Demaille wrote:
> Hi Martin,
> 
>> Le 7 déc. 2020 à 17:47, Martin Rehak <[email protected]> a écrit :
>> 
>> Hi Akim,
>> 
>>> Once compiled, src/mbcat should imitate what bison does when it quotes
>>> the input.  So could you feed it with tests/testsuite.dir/004/input.y
>>> and see what difference there is between the input and the output?
>>> 
>>> ./src/mbcat tests/testsuite.dir/004/input.y >/tmp/out.y
>>> diff tests/testsuite.dir/004/input.y /tmp/out.y
>>> 
>>> And if there are diffs, try to make them visible (cat -v for instance).
>>> Or attach /tmp/out.y.
>> 
>> So I am puzzled even more.
>> 
>> $ ./src/mbcat tests/testsuite.dir/004/input.y >/tmp/out.y
>> $ diff tests/testsuite.dir/004/input.y /tmp/out.y
>> $ echo $?
>> 0
> 
> This is probably related to the current locale.  Could you also try
> with LC_ALL=C set in your environment?  It will probably not change
> anything because mbcat does not call setlocale.  I have uploaded a
> new tarball with it.  Please, try the same procedure with:
> 
> https://urldefense.com/v3/__https://www.lrde.epita.fr/*akim/private/bison/bison-3.7.4.154-1bbfe.tar.gz__;fg!!GqivPVa7Brio!IhS-xLEI8xJEjIItvUHlZ0x86B0VsWGzdFfdnkyjje57NelDgoye4bIxXEywYSeVPA$ 
> https://urldefense.com/v3/__https://www.lrde.epita.fr/*akim/private/bison/bison-3.7.4.154-1bbfe.tar.lz__;fg!!GqivPVa7Brio!IhS-xLEI8xJEjIItvUHlZ0x86B0VsWGzdFfdnkyjje57NelDgoye4bIxXEzjHy5RuA$ 
> https://urldefense.com/v3/__https://www.lrde.epita.fr/*akim/private/bison/bison-3.7.4.154-1bbfe.tar.xz__;fg!!GqivPVa7Brio!IhS-xLEI8xJEjIItvUHlZ0x86B0VsWGzdFfdnkyjje57NelDgoye4bIxXEyBuUGVnw$ 
> 
> with both your default locale, and with LC_ALL forced to C.
> 
> 
> 
> Thanks a lot for keeping up with this!

Hi Akim,

we must be missing something:

$ for l in en_US.UTF-8 C ""; do LC_ALL=$l src/mbcat tests/testsuite.dir/004/input.y >/tmp/out.y; diff tests/testsuite.dir/004/input.y /tmp/out.y; echo $?; done
0
0
0

After applying the source with explicit setlocale(), the output is of 
course same.

As far as I understand the core of the test is to run bison on input.y.

But there is this in input.at:

# Autotest's diffing, when there are NUL bytes, just reports "binary
# files differ".  So don't leave NUL bytes.
AT_PERL_CHECK([[-p -e 's{([\0\377])}{sprintf "\\x%02x", ord($1)}ge' stderr]], [],
[[input.y:1.1-2: error: invalid characters: '\0\001\002\377?'
    1 | \x00^A^B\xff?
      | ^~
input.y:2.2: error: invalid null character
    2 | "\x00"
      |  ^
input.y:4.1: error: invalid character: '?'
    4 | ?
      | ^
input.y:5.14: error: invalid character: '}'
    5 | default: 'a' }
      |              ^
input.y:6.1: error: invalid character: '%'
    6 | %&
      | ^
input.y:6.2: error: invalid character: '&'
    6 | %&
      |  ^
input.y:7.1-17: error: invalid directive: '%a-does-not-exist'
    7 | %a-does-not-exist
      | ^~~~~~~~~~~~~~~~~
input.y:8.1: error: invalid character: '%'
    8 | %-
      | ^
input.y:8.2: error: invalid character: '-'
    8 | %-
      |  ^
input.y:9.1-10.0: error: missing '%}' at end of file
    9 | %{
      | ^~
]])

How to run the input.at so I am able to observe what is it doing?

And all this seem to work very well regardless of what locale we 
set.

$ pwd
/builds/mrehak/workspace/bison/components/bison/build/amd64/tests/testsuite.dir/004
$ LC_ALL=C
$ export LC_ALL
$ xxd i
00000000: 0001 02ff 3f0a                           ....?.
$ ../../../src/bison i 2>o
$ /usr/perl5/5.22/bin/perl -p -e 's{([\0\377])}{sprintf "\\x%02x", ord($1)}ge' o >op
$ xxd op
00000000: 693a 312e 312d 323a 2065 7272 6f72 3a20  i:1.1-2: error: 
00000010: 696e 7661 6c69 6420 6368 6172 6163 7465  invalid characte
00000020: 7273 3a20 275c 305c 3030 315c 3030 325c  rs: '\0\001\002\
00000030: 3337 373f 270a 2020 2020 3120 7c20 5c78  377?'.    1 | \x
00000040: 3030 0102 5c78 6666 3f0a 2020 2020 2020  00..\xff?.      
00000050: 7c20 5e7e 0a                             | ^~.

Could you look into attached trace of run script?

Thank you
-- 
m.