bug#75440: dejagnu does not like java.io.Console of OpenJDK >= 22

--refeik-4b82c5a3-1c44-4e21-8eb9-9215abd13c61
Content-Type: text/html; charset=UTF-8

<div style="font-family: 'verdana'; font-size: 12px; color: #000;">
<div>Dear Jacob,</div>
<div style="font-family: verdana; font-size: 12px; color: rgb(0, 0, 0);">&nbsp;</div>
<div style="font-family: verdana; font-size: 12px; color: rgb(0, 0, 0);">sorry for my late reply and thanks for your rapid and detailed answer. Also I am sorry for the faulty default.exp, it was a quick &amp; dirty version to trigger the bug and only a short snipped of my real project. It should only show the problem with the first echo. However, I have enhanced it somehow now, see the attachement.</div>
<div style="font-family: verdana; font-size: 12px; color: rgb(0, 0, 0);">&nbsp;</div>
<div>&gt; Tcl interpolates backslash escapes in double-quoted strings</div>
<div>I changed now \r\n to \\r\\n in all regular expressions.</div>
<div style="font-family: verdana; font-size: 12px; color: rgb(0, 0, 0);">&nbsp;</div>
<div>Please forgive me, I am not an expert in expect script but I try to do my best. I would be happy if you could give me some (direct) advices.</div>
<div>I will open a corresponding issue at https://bugreport.java.com and report back a link. Hopefully, they will look into/fix it and not only dismiss that using dejagnu is an edge case.</div>
<div style="font-family: verdana; font-size: 12px; color: rgb(0, 0, 0);">&nbsp;</div>
<div>Many thanks again,<br>Chris</div>
<br><br><br></div>
<div id="sub-body-container" style="margin: 10px 5px 5px 10px; padding: 10px 0px 10px 10px; border-left: 2px solid rgb(195, 217, 229);">
<div style="margin: 0px 0px 10px;">
<div><strong>Gesendet: </strong>Donnerstag, 9. Januar 2025 um 02:55</div>
<div><strong>Von: </strong>"Jacob Bachmeyer" &lt;[email protected]&gt;</div>
<div><strong>An: </strong>"Chris Nospam" &lt;[email protected]&gt;, [email protected]</div>
<div><strong>Betreff: </strong>Re: bug#75440: dejagnu does not like java.io.Console of OpenJDK &gt;= 22</div>
</div>
<div class="moz-cite-prefix">On 1/8/25 10:44, Chris Nospam via Bug-dejagnu via wrote:</div>
<blockquote>
<pre class="moz-quote-pre">Dear maintainers,

starting with OpenJDK Java 22 java programs using java.io.Console cannot be testet with dejagnu any more. With JDK 21 it works like a charm.</pre>
</blockquote>
<p>This is a JVM bug or misfeature.&nbsp; We have recently had similar problems with MinGW GCC tested under Wine. In that case, it was a consequence of Wine's new support for Windows quasi-ptys.</p>
<blockquote>
<pre class="moz-quote-pre">I belive it has to do with excape sequences that are not filtered out correctly or similar. I am using dejagnu 1.6.3-1, debain package under Ubuntu 24.04.</pre>
</blockquote>
<p>DejaGnu does not and cannot filter escape sequences.&nbsp; Expect matches the patterns it is given.&nbsp; You can write patterns in your testsuite that accommodate or even verify escape sequences from the program under test.</p>
<blockquote>
<pre class="moz-quote-pre">For an easy reproduction of the bug I have attached a simple example simple.zip. Just unpack it and execute exectest.sh. Be sure to update JAVA_HOME in this script to a jdk 22 or 23 installation.

In testsuite/simple.log some escape sequences are shown were no one are expected:
^[[?1h^[=^[[?2004hhello
^[[?1l^[&gt;^[[?1000l^[[?2004lhello</pre>
</blockquote>
<p>The JVM IO facilities are inserting escape sequences without your program calling for them. This is a JVM bug or misfeature that Expect (on which DejaGnu is built) exposes.</p>
<blockquote>
<pre class="moz-quote-pre">The attached simple echo example works after simply commenting the lines 13-14 (using Console) and uncommenting lines 16-17 (using BufferedReader) in testsuite/simple.sh.
Alternatively it also works by passing -Djdk.console=java.base to the jvm (1st line in exectest.sh), to trigger the old implementation in JDK.

#!/usr/bin/env -S java --source 23
// #!/usr/local/java/jdk-23.0.1/bin/java --source 23 -Djdk.console=java.base

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

public class ConsoleTest {

    public static void main(String[] args) throws IOException {
        System.out.print("prompt&gt; ");

        java.io.Console c = System.console();
        String input = c.readLine();

        // BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
        // String input = in.readLine();

        System.out.println(input);
        System.out.println("prompt&gt; ");
    }

}
</pre>
</blockquote>
<p>First, think about what your program actually does compared to what your testsuite code expects it to do.&nbsp; Examine the message produced at the end of the test run in simple.log.&nbsp; In fact, that message is produced *after* echo.exp has completed.&nbsp; Where does that message come from?</p>
<p>(You will benefit far more from the lesson if you find it for yourself than if I tell you.&nbsp; There is only one place in your testsuite that can emit that message.)</p>
<p>Second, the inability to send text to the program under test is not a failure:&nbsp; it is an error and probably an UNRESOLVED test.&nbsp; A block that is executed as a result of catching a Tcl error probably should be invoking "error" or possibly "unresolved", not "fail" or even "pass" (unless you are testing an error condition).</p>
<p>Third, Tcl interpolates backslash escapes in double-quoted strings.&nbsp; You will need to double (some of) the backslashes and/or review re_syntax(n) for Tcl regexp syntax.</p>
<p>Fourth, think about simple_test could actually fail.&nbsp; That depends on what you are trying to do, but any "pass" should have a corresponding "fail" somewhere.</p>
<p>(Most of the above refers to Expect code in your testsuite/config/default.exp file.)</p>
<blockquote>
<pre class="moz-quote-pre">It has something to do with the change of the Java default Console provider to JLine in Java 22.
<a class="moz-txt-link-freetext" href="https://bugs.openjdk.org/browse/JDK-8309141is" target="_blank" rel="noopener noreferrer">https://bugs.openjdk.org/browse/JDK-8309141is</a> is not special with DejaGnu; we have recently had similar issues with
 Wine, encountered while testing mingw32 builds on *nix hosts
However, as Java Console works everywhere else (windows/linux console/bash/EclipseIDE internal "console"/IntelliJ internal "console") as far I can see, it seems to be something special with dejagnu.</pre>
</blockquote>
<p>No, the problem is with the JVM and/or your programs and scripts.&nbsp; It appears to work everywhere else because those escape sequences either perform terminal control functions or are ignored.&nbsp; You can write regexps that will optionally match and discard the escape sequences.&nbsp; Expect exposes this issue because it is strict about matching exactly what you tell it to match.</p>
<p>&nbsp;</p>
<p>-- Jacob</p>
<p>&nbsp;</p>
</div>
--refeik-4b82c5a3-1c44-4e21-8eb9-9215abd13c61
Content-Type: application/x-zip-compressed
Content-Disposition: attachment; filename=simple.zip
Content-Transfer-Encoding: base64
X-UI-Content-Hash: c6f53b317bbddf4e2373eaf1a8cf9e546a49b612ff031e93f74dea63f82e15e

UEsDBBQAAAAIAGyCJ1qQ0RLGiAAAAJ8AAAALABwAZXhlY3Rlc3Quc2hVVAkAA/tFfWek+H9ndXgL
AAEE8VYAAATxVgAAU1bULy0u0k/KzNMvzuDiSq0oyC8qUfByDHOM9/D3dbUFy+bkJyfm6GclliXq
Z6Vk6xoZ6xnoGcLUBjiGeNiqVMO11IIMs1KpBonXcnElpyiUpBaXFJdmlqRyFeUq6KYpaOnl5KcD
yeLSXK6i0jyQtIKubkl+fo5CcWZuQU6qQmpyRr4e0HwuAFBLAwQKAAAAAACDiylaAAAAAAAAAAAA
AAAACgAcAHRlc3RzdWl0ZS9VVAkAAxX5f2cW+X9ndXgLAAEE8VYAAATxVgAAUEsDBBQAAAAIADuH
KVqDcqh8eQEAADcDAAATABwAdGVzdHN1aXRlL3NpbXBsZS5zaFVUCQADAfJ/Zwbyf2d1eAsAAQTx
VgAABPFWAABtUctOwzAQvPsrlnBxVMXhcSMqBygHJCQkyg04OMmmdUns4EcLQv13nMRFJHQviXZ2
ZnbHpyepMzrNhUxRbiFZwoZvOSSJUU4XCBeXJE3hdJiqVcHrtBtIN+V7cnHJzth5z52SIFn4CVYo
aVSN8w5mOTdIiGhapW3vwoRiN66qUGP5hLxEnU3h+8e7zwJbK5T8j8nW2aXVyJsDm7Qur0UBRc2N
gdvB/RmNhW9CwFfAjeXWf7ZKlNBwIamXEXL18gZcr0wMdq3VzsAfdy8AoXLlRbmEDycszKHitcHs
F92tRY1ATzo0/sPqavllLDZMOcta72dp1GrVtPYaojgjo9HDkeEGENJbBX5IlcbZiOLfaZzmQJK4
m/Rp1/qXHg3qQsbTZYZ0vJyneEUhmaeVD0Ie22F6ZC1p9Cpf5QJzt7qCCGZBaAZ9P5pIiApoP8Dw
w/lsadRFGcXTMLsKT2C1w7HIHtC/yhHGke16szgL62OxVmMpMv7bE7InP1BLAwQUAAAACACDiyla
DX1zO4kCAACzBQAAFAAcAHRlc3RzdWl0ZS9zaW1wbGUubG9nVVQJAAMW+X9nGPl/Z3V4CwABBPFW
AAAE8VYAAK1UTWsbMRA9R5BfkMsgfIgP+xGTOInBKaVQaCk9tOkpBSNrZ71q1tJWH45DyH/vrHY3
jfMFheqy7NO890YzI12i82CDhuUtyMoqB0bDZRXgs9AA53B0OpuczfITmOSTE/ZVeLVBkEaXahUs
/VE0cbZn08X0OGlkUisdtslKB8b29ubzOTi1bmoETz4OCGDsu6ywCISZEjbCqqjiZgxoBa22jH0L
Wiu9Ai/sCn0H/nAtkgVnM1cJi1mBvwT5ZEvhcGmELVzWBqa4bUA4iBAU6KRVTcyzVORZGtvLpq9L
dufbkVuhRqskKO3RlkLiq3JEMNZn5Cu8rLJY1OwDndDUmHXVyGI1gvI4WBVYilD7wc0bUydCF0mn
nbgGpSqf2afMNeJGQ9rrpq7aZ40168ZfwFDEf0yoV4pAhrIyMac0TVmFdW322cHVu6PqYE6fSZ4f
Vx3awfXBRfvJ87zut+ueNCTFPr7/9GUGEYVDUkbpsQAeAT4G1vpK6uj/SVuaFm0dlIYcHFK5C/f0
CL9J4fUTdEnOoCu1KoCAKWjjwTSo49TeVO0s4BZl8FRyxjsO3MXddiHNervuoBHOAR/hVvmFXBcc
7h+CEovAU/6I1i6HulgEhxb4T813tkqh6kdajwpKnMSUiQm+CX7MIdKoiR3/kNoh6Q62fl3hFq0G
B7q9SDd9HKOU3phr0iqpd3CjfKU046O7djbvu/hnYrJGoUOz6MZ2URlz3YsevSX6Ag1GMliL2vdg
78XpwqGQFaf7Xdz2+U6P39DuCbArB6Peq1Zu6NMG7dLQ5PE+hN61F5NwtLeLL7RY49Pgh1gfPa5c
Uys/BP1tRXxG+AuPED3K7TS3czzU8PR8Ot5nfwBQSwMECgAAAAAAoWUpWgAAAAAAAAAAAAAAABcA
HAB0ZXN0c3VpdGUvc2ltcGxlLnRlc3RzL1VUCQADvbZ/Z762f2d1eAsAAQTxVgAABPFWAABQSwME
FAAAAAgAoWUpWskxBUcXAAAAHAAAAB8AHAB0ZXN0c3VpdGUvc2ltcGxlLnRlc3RzL2VjaG8uZXhw
VVQJAAO9tn9nwbZ/Z3V4CwABBPFWAAAE8VYAACvOzC3ISY0vSS0uUajOSM3Jya+F0VwAUEsDBBQA
AAAIAIOLKVpJxXWy0AAAABoBAAAUABwAdGVzdHN1aXRlL3NpbXBsZS5zdW1VVAkAAxb5f2cg+X9n
dXgLAAEE8VYAAATxVgAANY5NSwMxEIbPnV/x0pMeNqmLrXVhDyIIinjQ3mVNp5vAmiz5KOu/d0jp
3OZh3veZA6eMWDx+/mBsdAnB42AL3gYPPOLuoWv33WaLdtNu6WPI7swwwZ/cWKJsci2ZZb/73t03
s2km58vSjL4QrVZ93yO533liZPEkCCD6MpaPRVg44TxEV1tSR5Ap3i1En8V750fkIY6cL/DKNC9z
iFknI3pjdX1aP0tDmFhfbLraistXoCrQbGxQkodSil6eXt87WJ6mgBuBbDIfsa5gfQv6B1BLAwQK
AAAAAACAiylaAAAAAAAAAAAAAAAAEQAcAHRlc3RzdWl0ZS9jb25maWcvVVQJAAMP+X9nD/l/Z3V4
CwABBPFWAAAE8VYAAFBLAwQUAAAACABuZylafxmiSY4CAAAKCgAAHAAcAHRlc3RzdWl0ZS9jb25m
aWcvZGVmYXVsdC5leHBVVAkAAx+6f2dWun9ndXgLAAEE8VYAAATxVgAAtVZNj5swEL3zK6ZeVCXZ
he4eetmVuuqlUqX+A0wRASdBAsPaRolE+e81NuYrJKWRwgVl7Hnvzfh5CCcCuAhpHLI4EElG8lLA
y1eL6zgTZdGFdbRgeVYIQPr9DZCKhpQfCQOkf5FTIoIoiwF9lIlAlgoaGHvKN4ANCGM5g2fLkr8j
4ElWpCRQQqCqobJAPvs034bpmezJ2lD6cGkmpLlH+UV4pEESn+/SCi21MK3qjFLBgPtF1+Hyg84j
p4JEoq2meZCtwRFUUHdRA11BFzKso269TJZpHJS8OQxMf3z/+etVd0Me1IVtK0N0TMQBaN6iP7U6
SQzYCMRojSl66wX1akm+G1R0H6ELtE2l1bMnNTZOPbZb494ztxlLX3dEsgN7VHLfEUZEyWiryWz2
olBEB6iaIqULDImso/YHuaNW9e3ZhUkKKMrLNJbHJtQ2EOQk25NwEAfSKNyzMANWUprQ/Tu6Iqd1
5UAFM2QPI8c+KACHEUDuptvtbtq99QWTNwZpngqKkPMBDxq4SKOiMyfN1t/3oMOCVWcMmePkO0ce
cFGK9YwxJscuCJdyZTqTzeFlKowDBuuBoioYkm+9CTUzbwYpiEZYEOVZRuhiTOzZbQb25/Gvwl8c
bHpO39HEjtOWczcPg+nMNTMzsuflFpwwTaHyfNf+vVpvHt//eH4NbZehwvhzDR9lLt3y7yTVbZ3S
jIF5jyv72s3d8bC8PpgC9h/t9uO4sTUX9kAvNmv94B/ZursikrSrd27sKkbsvWLf3RjSK7j9fZHA
q71s+JHldN/6YjxWdZvkWF2k4NNCBUvustGm7PcE21LcJIychl/Wm1TcQHvxs72M8b8+xr2StzFb
N8xN+XPTb/j3yvoLUEsDBAoAAAAAAJlNKVoAAAAAAAAAAAAAAAAOABwAdGVzdHN1aXRlL2xpYi9V
VAkAA4KMf2eCjH9ndXgLAAEE8VYAAATxVgAAUEsDBAoAAAAAAEd8J1oAAAAAAAAAAAAAAAAYABwA
dGVzdHN1aXRlL2xpYi9zaW1wbGUuZXhwVVQJAANmO31nhox/Z3V4CwABBPFWAAAE8VYAAFBLAQIe
AxQAAAAIAGyCJ1qQ0RLGiAAAAJ8AAAALABgAAAAAAAEAAAD9gQAAAABleGVjdGVzdC5zaFVUBQAD
+0V9Z3V4CwABBPFWAAAE8VYAAFBLAQIeAwoAAAAAAIOLKVoAAAAAAAAAAAAAAAAKABgAAAAAAAAA
EAD9Qc0AAAB0ZXN0c3VpdGUvVVQFAAMV+X9ndXgLAAEE8VYAAATxVgAAUEsBAh4DFAAAAAgAO4cp
WoNyqHx5AQAANwMAABMAGAAAAAAAAQAAAMmBEQEAAHRlc3RzdWl0ZS9zaW1wbGUuc2hVVAUAAwHy
f2d1eAsAAQTxVgAABPFWAABQSwECHgMUAAAACACDiylaDX1zO4kCAACzBQAAFAAYAAAAAAABAAAA
tIHXAgAAdGVzdHN1aXRlL3NpbXBsZS5sb2dVVAUAAxb5f2d1eAsAAQTxVgAABPFWAABQSwECHgMK
AAAAAAChZSlaAAAAAAAAAAAAAAAAFwAYAAAAAAAAABAA/UGuBQAAdGVzdHN1aXRlL3NpbXBsZS50
ZXN0cy9VVAUAA722f2d1eAsAAQTxVgAABPFWAABQSwECHgMUAAAACAChZSlayTEFRxcAAAAcAAAA
HwAYAAAAAAABAAAAtIH/BQAAdGVzdHN1aXRlL3NpbXBsZS50ZXN0cy9lY2hvLmV4cFVUBQADvbZ/
Z3V4CwABBPFWAAAE8VYAAFBLAQIeAxQAAAAIAIOLKVpJxXWy0AAAABoBAAAUABgAAAAAAAEAAAC0
gW8GAAB0ZXN0c3VpdGUvc2ltcGxlLnN1bVVUBQADFvl/Z3V4CwABBPFWAAAE8VYAAFBLAQIeAwoA
AAAAAICLKVoAAAAAAAAAAAAAAAARABgAAAAAAAAAEAD9QY0HAAB0ZXN0c3VpdGUvY29uZmlnL1VU
BQADD/l/Z3V4CwABBPFWAAAE8VYAAFBLAQIeAxQAAAAIAG5nKVp/GaJJjgIAAAoKAAAcABgAAAAA
AAEAAAC0gdgHAAB0ZXN0c3VpdGUvY29uZmlnL2RlZmF1bHQuZXhwVVQFAAMfun9ndXgLAAEE8VYA
AATxVgAAUEsBAh4DCgAAAAAAmU0pWgAAAAAAAAAAAAAAAA4AGAAAAAAAAAAQAP1BvAoAAHRlc3Rz
dWl0ZS9saWIvVVQFAAOCjH9ndXgLAAEE8VYAAATxVgAAUEsBAh4DCgAAAAAAR3wnWgAAAAAAAAAA
AAAAABgAGAAAAAAAAAAAALSBBAsAAHRlc3RzdWl0ZS9saWIvc2ltcGxlLmV4cFVUBQADZjt9Z3V4
CwABBPFWAAAE8VYAAFBLBQYAAAAACwALANsDAABWCwAAAAA=
--refeik-4b82c5a3-1c44-4e21-8eb9-9215abd13c61
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline

X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KQnVnLWRlamFn
bnUgbWFpbGluZyBsaXN0CkJ1Zy1kZWphZ251QGdudS5vcmcKaHR0cHM6Ly9saXN0cy5nbnUub3Jn
L21haWxtYW4vbGlzdGluZm8vYnVnLWRlamFnbnUK

--refeik-4b82c5a3-1c44-4e21-8eb9-9215abd13c61--