| Newsgroups |
gmane.comp.lang.smalltalk.squeak.general |
| Message-ID |
<[email protected]> |
Marcel Taeumel uploaded a new version of Multilingual to project The Trunk:
http://source.squeak.org/trunk/Multilingual-mt.291.mcz
==================== Summary ====================
Name: Multilingual-mt.291
Author: mt
Time: 29 May 2026, 3:51:02.274187 pm
UUID: 2bb8b716-544d-a641-9b0d-64709ae6e24f
Ancestors: Multilingual-mt.290
On Windows, the latest OSVM (2026.04) expects Squeak to write UTF16 LE onto stderr and stdout. We can accommodate for this within the image but should fix this regression later.
See: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/771
=============== Diff against Multilingual-mt.290 ===============
Item was changed:
StandardFileStream subclass: #MultiByteFileStream
instanceVariableNames: 'converter lineEndConvention wantsLineEndConversion'
+ classVariableNames: 'Cr CrLf Lf LineEndDefault LineEndStrings LookAheadCount UseUTF16ForStderr UseUTF16ForStdout'
- classVariableNames: 'Cr CrLf Lf LineEndDefault LineEndStrings LookAheadCount'
poolDictionaries: ''
category: 'Multilingual-TextConversion'!
!MultiByteFileStream commentStamp: '<historical>' prior: 0!
The central class to access the external file. The interface of this object is similar to good old StandardFileStream, but internally it asks the converter, which is a sub-instance of TextConverter, and do the text conversion.
It also combined the good old CrLfFileStream. CrLfFileStream class>>new now returns an instance of MultiByteFileStream.
There are several pitfalls:
* You always have to be careful about the binary/text distinction. In #text mode, it usually interpret the bytes.
* A few file pointer operations treat the file as uninterpreted byte no matter what. This means that if you use 'fileStream skip: -1', 'fileStream position: x', etc. in #text mode, the file position can be in the middle of multi byte character. If you want to implement some function similar to #peek for example, call the saveStateOf: and restoreStateOf: methods to be able to get back to the original state.
* #lineEndConvention: and #wantsLineEndConversion: (and #binary) can cause some puzzling situation because the inst var lineEndConvention and wantsLineEndConversion are mutated. If you have any suggestions to clean up the protocol, please let me know.!
Item was added:
+ ----- Method: MultiByteFileStream class>>newForStdio: (in category 'stdio') -----
+ newForStdio: moniker
+ "Use crlf as line end convention on windows, lf on all other platforms. Also make sure that the converter is initialized."
+
+ | lineEndConvention |
+ lineEndConvention := self lineEndDefault.
+ lineEndConvention == #crlf ifFalse: [
+ lineEndConvention := #lf ].
+ ^self new
+ lineEndConvention: lineEndConvention;
+ initializeConverterForStdio: moniker;
+ yourself!
Item was added:
+ ----- Method: MultiByteFileStream class>>useUTF16ForStderr (in category 'stdio') -----
+ useUTF16ForStderr
+ <preference: 'Encode the contents of stderr file with UCS-2/UTF-16 (little endian).'
+ category: 'Files'
+ description: 'If true, then the contents of stderr will use UTF-16 text conversion instead of the system language default. This is useful for Windows platforms, where the console expect UTF-16 for Unicode output.'
+ type: #Boolean>
+ ^ UseUTF16ForStderr ifNil: [ Smalltalk platformName = 'Win32' ]!
Item was added:
+ ----- Method: MultiByteFileStream class>>useUTF16ForStderr: (in category 'stdio') -----
+ useUTF16ForStderr: booleanOrNil
+
+ UseUTF16ForStderr := booleanOrNil.
+ self voidStdioFiles.!
Item was added:
+ ----- Method: MultiByteFileStream class>>useUTF16ForStdout (in category 'stdio') -----
+ useUTF16ForStdout
+ <preference: 'Encode the contents of stdout file with UCS-2/UTF-16 (little endian).'
+ category: 'Files'
+ description: 'If true, then the contents of stdout will use UTF-16 text conversion instead of the system language default. This is useful for Windows platforms, where the console expect UTF-16 for Unicode output.'
+ type: #Boolean>
+ ^ UseUTF16ForStdout ifNil: [ Smalltalk platformName = 'Win32' ]!
Item was added:
+ ----- Method: MultiByteFileStream class>>useUTF16ForStdout: (in category 'stdio') -----
+ useUTF16ForStdout: booleanOrNil
+
+ UseUTF16ForStdout := booleanOrNil.
+ self voidStdioFiles.!
Item was added:
+ ----- Method: MultiByteFileStream>>initializeConverterForStdio: (in category 'initialize-release') -----
+ initializeConverterForStdio: moniker
+ "Specialized version of #initializeConverter for stdio use, which allows for overriding the system default according to preferences. For example, Windows may expect UCS-2/UTF-16 for console output."
+
+ self converter: (moniker caseOf: {
+ [#stdin] -> [Locale currentPlatform newSystemConverter].
+ [#stdout] -> [
+ self class useUTF16ForStdout
+ ifTrue: [UTF16TextConverter new useLittleEndian: true; yourself]
+ ifFalse: [Locale currentPlatform newSystemConverter]].
+ [#stderr] -> [
+ self class useUTF16ForStderr
+ ifTrue: [UTF16TextConverter new useLittleEndian: true; yourself]
+ ifFalse: [Locale currentPlatform newSystemConverter]] }).!
Squeak-dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]