Re: testsuite under wine

On 1/5/23 7:33 PM, Jacob Bachmeyer via Fortran wrote:
> NightStrike wrote:
>> On Fri, Dec 23, 2022 at 11:00 PM Jacob Bachmeyer <[email protected]> 
>> wrote:
>>> NightStrike wrote:
>>>> On Wed, Dec 21, 2022 at 11:37 PM Jacob Bachmeyer 
>>>> <[email protected]> wrote:
>>>>> [...]
>>>> So at least we know for sure that this particular instance of extra
>>>> characters is coming from Wine.  Maybe Wine can be smart enough to
>>>> only translate \n into \r\n instead of translating \r\n into \r\r\n.
>>>> Jacek / Eric, comments here?  I'm happy to try another patch, the
>>>> first one was great.
>>>>
>>> I doubt that Wine is doing that translation.  MinGW libc produces output
>>> conformant to Windows conventions, so printf("\n") on a text handle
>>> emits "\r\n", which Wine passes along.  POSIX convention is that "\n" is
>>> translated to "\r\n" in the kernel terminal driver upon output, so the
>>> kernel translates the "\n" in the "\r\n" into /another/ "\r\n", yielding
>>> "\r\r\n" at the pty master end.  This is why DejaGnu testsuites must be
>>> prepared to discard excess carriage returns.  The first CR came from
>>> MinGW libc; the second CR came from the kernel terminal driver; the LF
>>> was ultimately passed through.
>>
>> Jacek and I have been digging into this on IRC, and he's been very
>> helpful in trying to get further, but we're still stuck.  We tried to
>> be more introspective, inserting strace both as "strace script wine"
>> and as "script strace wine".  We tried running just "wine a.exe"
>> without any extra glue, and logging the raw SSH packets from putty.
>> After many iterations on these and other tests, Jacek finally had the
>> idea to try removing Windows entirely from the equation, and we ran
>> with a purely unix program / compiler combination:
>>
>> #include <unistd.h>
>>
>> int main()
>> {
>>         write(1, "test\r\n", 6);
>>         return 0;
>> }
>>
>> (and also as "test\n", 5)
>>
>> In both versions, the following was observed:
>>
>> case 1) ./a.out | xxd
>> case 2) script -c ./a.out out; xxd out
>> case 3) enable putting logging, ./a.out
>>
>> In case 1, xxd showed no extra \r's.  In cases 2 and 3, there was an
>> extra \r (either 0d 0d 0a for test\r\n, or 0d 0a for test\n).
>>
>> So, is it possible after all of this back and forth regarding mingw,
>> wine, and others, that it's down to the write() system call that's
>> inserting extra \r's?  Is this expected?
> 
> "This is why DejaGnu testsuites must be prepared to discard excess 
> carriage returns."
> 
> The write(2) system call inserts nothing and simply hands off the buffer 
> to the relevant part of the kernel I/O subsystem.  (The kernel in POSIX 
> is *not* a monolithic black box.)  When stdout for your test program is 
> a pty slave, that relevant part is the kernel terminal driver.  The 
> kernel terminal driver is converting "\n" to "\r\n" upon output to the 
> associated port, since hardware terminals typically *do* require CRLF. 
> The associated port in this case is virtual and part of the kernel pty 
> subsystem, which presents octets written to that port to its associated 
> pty master device.  The user-visible pty slave device acts just like a 
> serial terminal, including all translations normally done for handling 
> serial terminals.
> 
> A pty is conceptually a null-modem cable connected between two 
> infinitely-fast serial ports on the same machine, although the slave 
> will still report an actual baud rate if queried.  (Run "stty" with no 
> arguments under script(1), an ssh session, or an X11 terminal emulator 
> to see what a pty slave looks like on your machine.)
> 
> In your case 1, the pty subsystem is not used and output is collected 
> over a pipe.  Using "./a.out > out; xxd out" would produce the same 
> results.  In cases 2 and 3, there is a pty involved, either set up by 
> script(1) or by sshd (assuming you meant "enable putty logging" in case 
> 3) that performs the standard terminal translations.  In all cases, 
> strace(1) will show the exact string written to the pty slave device, 
> which will not include any extra CRs because *those* *are* *inserted* 
> *by* *the* *kernel* *terminal* *driver* as the data is transferred to 
> the pty master device's read queue.
> 
> This insertion of carriage returns is expected and standardized behavior 
> in POSIX and is the reason Unix could use bare LF as end-of-line even 
> though hardware terminals always needed CRLF.  CP/M (and therefore 
> MS-DOS which began its existence as a cheap CP/M knockoff) did not have 
> this translation layer and instead dumped the complexity of a two-octet 
> end-of-line sequence on user programs, leading to much confusion even 
> today.  This is not a Wine issue, although the terminal escape sequences 
> in your original issue *were* from Wine.  Note that the number of excess 
> carriage returns that a DejaGnu testsuite must be prepared to discard is 
> unspecified because running tests on remote targets may result in *any* 
> *number* of CRs preceding each LF by the time the results reach the test 
> driver machine in more complex testing lab environments.
> 
> 
> -- Jacob

Agree