Re: Odp: Pd: Missing legacy Arabic encoding

--000000000000b9f284065129bcae
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Windows can use other ways to map 16-bit codes in its *legacy* Console
buffer (using old CHAR_INFO structure), it can perfectly internally use
compatibility characters, or PUAs of the BMP, and still present an API that
exposes connforming sequences. You're talking about an old implementation
that was built even  long before the Arabic script was extended (and newer
scripts using contextual joining behaviors, that have never been part of
the BMP, shcih as Adlam, and other scripts like Mongolian that also may
need such sequences with ZWJ/ZWNJ controls, or with other formatting
characters like those specific to Mongolian like FVS1...FVS4 and MVS, or
those common to many Bhramic scripts, that the *legacy* Console did not
support.
The *legacy* console was not built to support more than one plane
(including many CJK cgaracters). The newer console can!

Le mer. 6 mai 2026 =C3=A0 18:37, [email protected] <[email protected]=
> a
=C3=A9crit :

> Have you read the L2/26-077 proposal? Using ZWJ or ZWNJ would not work fo=
r
> the compatibility purposes at all as already explained in the proposal.
> This is because ZWJ or ZWNJ would take the space of one character tile in
> the CHAR_INFO structure. Suppose that you're trying to map 0xD0 from FP16=
4
> to a sequence of U+FE7C U+200D U+064B (=EF=B9=BC=E2=80=8D=D9=8B). The leg=
acy application fills
> the 80=C3=9725 screen with all 0xD0 tiles. You subsequently try to captur=
e the
> tiles with a Win32 program by using ReadConsoleOutputA into an 80=C3=9725=
 buffer
> of 2000 tiles. This succeeds and captures 0xD0 into all the tiles. You th=
en
> try to capture the tiles using ReadConsoleOutputW into an 80=C3=9725 buff=
er.
> Each sequence U+FE7C U+200D U+064B would take a sequence of three CHAR_IN=
FO
> structures to store, meaning 6000 such structures for the whole screen. B=
ut
> the 80=C3=9725 buffer has only room for 2000 instances of the structure (=
one per
> character tile). Since CHAR_INFO stores 16-bit character code, by that sa=
me
> logic the compatibility characters would have to be in BMP for it to work=
.
> In Windows 95 Vietnamese and Windows 95 Thai, there are instances where o=
ne
> character tile takes multiple CHAR_INFO structures, causing visual width =
to
> be smaller than logical width, and when that happens, the remaining space
> at the end of the line is left blank, allowing for CP1258/CP874 combining
> characters in those systems to map 1:1 to their Unicode equivalents.
> Windows 3.1/95/98/ME Arabic don't work that way and don't use combining
> characters or ZWJ sequences, so visual width is always equivalent to
> logical width, each character tile maps 1:1 to a CHAR_INFO structure and
> all characters may fill the entire line, which would be impossible if som=
e
> of those characters were mapped to composition sequences or non-BMP
> characters. Since there is currently no sufficient evidence of user
> community that would need to use those mappings, there are no plans for
> those characters to be added to Unicode, and therefore the only solution
> for the ReadConsoleOutputW to work properly in this case is to use agreed
> upon private use mappings for those compatibility characters.
>
>
> *Dnia 06 maja 2026 18:06* Philippe Verdy via Unicode
> <[email protected]> < [email protected] > napisa=C5=82(a):
>
> You actually don't need any new compatibility characters for Arabic
> contextual forms, or for other contextual forms in other joining scripts
> (like Adlam, or even Mongolian whichbis a LTR script).
>
> You just have to prepend or append a ZWJ or ZWNJ formatting control to th=
e
> unified letter if you want to override its default contextual presentatio=
n
> form.
>
>
> Le mar. 5 mai 2026, 00:48, Asmus Freytag via Unicode <
> [email protected]> a =C3=A9crit :
>
> The issue at hand is the distinction between a theoretical gap and
> real-life problem.
>
>
> You have demonstrated that there are specifications that, if chained in
> the right way, can lead to ambiguities or gaps in interchange.
>
>
> What we don't have is an actual use case with real-life consequences for =
a
> set of existing users, not hypothetical ones.
>
>
> When it comes to encoding decisions based on existing documents, there is
> a strong presumption that once sufficiently many documents exist that
> contain a character, that this character will be needed in digitizing the=
se
> documents, whether immediately, or eventually (e.g. in the case of future
> scholarly studies). Also, the texts themselves exist, barring accidents, =
in
> permanence. Therefore, it is justified to consider irrevocably allocating=
 a
> character that will map to this source in perpetuity, even though each
> encoded character carries a small cost for implementers.
>
> However, when it comes to legacy characters, there's an additional cost
> that is imposed, and that is based on the fact that characters that are
> encoded solely for compatibility will usually violate one or more of the
> other encoding principles, something that incrementally complicates the
> standard. Even for people who never intend to use that character.
>
> Therefore, the SEW is on solid ground when it demands not only a
> hypothetical scenario, but evidence of actual impact on actual users. Not
> only whether some application could invoke an API, but whether such
> applications exist and are used today to access documents encoded using t=
he
> legacy characters in a way that is compromised irreparably by not having =
an
> encoding for them.
>
>
> A./
>
>
>
> On 5/4/2026 10:24 AM, [email protected] via Unicode wrote:
>
> In UTC 187 Minutes, "Asmus Freytag noted that the fact that lists of
> things existed in the past does not make these things plain text. Ned
> Holbrook pointed out that the purported issue occurs in a closed system,
> not in public interchange.". However, the arguments in the proposal do
> not merely hinge on the encodings being lists of characters, but
> specifically points out methods to interchange text, including an example
> of copying terminal output and pasting to Notepad, where the copying
> invokes the mapping of the current terminal codepage to UCS-2 (as is
> CHAR_INFO compatible) and the pasting writes it into plain text. Win32 is
> also not a closed system, as Win32 can capture the tiles of the output of
> Windows 3.1 Arabic DOS/Win16 programs and Windows 95/98/ME Arabic
> DOS/Win16/Win32 programs, but Win32 can also interact with public text
> interchange systems by reading and writing to files and network. I'm not
> saying that Unicode absolutely must include those characters, but those
> kinds of misleading claims are causing users to misunderstand what the
> proposal is about, and I don't want Unicode to be relying on uninformed
> decisions to evaluate proposals.
>
>
> *Dnia 18 kwietnia 2026 13:36* [email protected] via Unicode
> <[email protected]> < [email protected] >
> <[email protected]> napisa=C5=82(a):
>
> The SEW subsequently explained that the actual reason is due to
> insufficient evidence of user community that would need to use the
> resulting mapping. Despite Win32 being a highly popular platform with
> plenty of backwards compatibility and native UCS-2 terminal support, the
> specific use cases of installing codepages into Windows NT and using
> terminal tiles from Windows 3.1/95/98/ME are not sufficiently documented,
> making it difficult for any user communities to form around it. So it see=
ms
> like the idea of standardizing legacy Arabic terminal BMP mappings is a
> dead end for now.
>
>
> *Dnia 17 kwietnia 2026 22:59* [email protected] via Unicode
> <[email protected]> < [email protected] >
> <[email protected]> napisa=C5=82(a):
>
> The Recommendations in L2/26-100 claim that Microsoft's documentation of
> legacy Arabic encodings is available at
> https://learn.microsoft.com/en-us/typography/legacy/legacy_arabic_fonts.
> However, that article only demonstrates two encodings of TrueType fonts,
> which are used in Windows 3.1 but are completely different from the eight
> terminal encodings. Unlike the TrueType encodings which represent interna=
l
> shaping mappings and are not used for text interchange, the terminal
> encodings have been demonstrated to be directly used in text interchange
> through int 10h and ReadConsoleOutputA/WriteConsoleOutputA as already
> demonstrated in L2/26-077. The Recommendations also claim that the propos=
al
> does not demonstrate any need for interchange or encoding, but the propos=
al
> actually demonstrated such a need due to the logical extension of the Win=
32
> terminal API to the functions ReadConsoleOutputW/WriteConsoleOutputW, whi=
ch
> are in Windows NT and may be used on the output of previously ran program=
s
> (including those that used the legacy Arabic terminal encodings), which
> given the CHAR_INFO structure, therefore implies a need for all the tiles
> to map to BMP for interchange. I'm not objecting to the SEW's conclusion =
of
> "Users are expected to use PUA.", which can indeed be used to provide a
> mapping even if not standardized, but the reasoning given was flawed.
>
>
> *Dnia 09 stycznia 2026 17:25* [email protected] < [email protected]
> > <[email protected]> napisa=C5=82(a):
>
> The following Win32 C code will output 256 characters in system console
> codepage into the character grid, capture those character tiles in UCS-2 =
if
> possible, and then output the current console codepage number.
>
>
> #include <windows.h>
> #include <stdio.h>
> int main(){
> HANDLE hConsole=3DGetStdHandle(STD_OUTPUT_HANDLE);
> CHAR_INFO screen[256];
> COORD size=3D{16,16,};
> COORD pos=3D{0,0,};
> SMALL_RECT rect=3D{0,0,15,15,};
> for(int i=3D0;i<256;i++){
> screen[i].Attributes=3D0xF0;
> screen[i].Char.AsciiChar=3Di;
> }
> WriteConsoleOutputA(hConsole,screen,size,pos,&rect);
> CHAR_INFO screenu[256];
> if(ReadConsoleOutputW(hConsole,screenu,size,pos,&rect)){
> for(int i=3D0;i<256;i++) printf("%04X ",screenu[i].Char.UnicodeChar);
> }
> else{
> printf("error %08X\n",GetLastError());
> }
> printf("codepage %u",GetConsoleOutputCP());
> }
>
> In most cases, whenever a legacy Win32 codepage is used, the application
> can run on Windows NT to capture the UCS-2 mapping of those character cel=
ls
> to the BMP (although for CJK codepages a more complex setup would be
> necessary due to thousands of fullwidth characters with 2-byte sequences)=
.
>
>
> However, in Arabic versions of Windows 9x (95/98/ME) the resulting
> character set has many presentation forms that are not in Unicode. This i=
s
> the result when running on Windows ME: https://i.imgur.com/QFm3SkI.png in
> 10=C3=9720 font, https://i.imgur.com/KUbLQ0A.png in 10=C3=9718 font (same=
 result
> also appears in Windows 95/98). 5=C3=9712, 7=C3=9712, 8=C3=9712, 10=C3=97=
18, 10=C3=9720, and 12=C3=9716
> bitmap fonts have been attested with that character set (VGAOEM.FON,
> 8514OEM.FON, DOSAPP.FON). The 10=C3=9720 font has slightly different mapp=
ing
> than the other sizes: 0x93 is =C3=B6 instead of =C3=B4, and 0x97 is missi=
ng (causing
> the following characters on the same line to be drawn at the wrong
> position). It also claims to be using codepage 720, but many characters
> differ from their CP720 mappings, including the bundled CP_720.NLS mappin=
gs
> (for example, =D9=80 (U+0640 ARABIC TATWEEL) is 0x95 in CP720, but in the
> console 0x95 is =D8=B4 instead, and the tatweel is at 0xFF). On Windows
> 9x, ReadConsoleOutputW is not supported so the UCS-2 mappings of the
> console character tiles cannot be captured (error 0x00000078
> ERROR_CALL_NOT_IMPLEMENTED).
>
>
> When that program runs on Arabic versions of Windows NT, the visual outpu=
t
> is of the CP437 character set if one of the bundled bitmap fonts is used =
(
> https://i.imgur.com/RxjtxMH.png), or the CP720 set if Lucida Console is
> used, with the Arabic letters either having glitchy font substitution (NT
> 4.0, NT 5.0/2000) or the .notdef glyph (NT 5.1/XP and up). In fact, it
> seems that the only Arabic bitmap fonts that occur in Windows NT are CP12=
56
> fonts, which are not used in terminals. So this appears to be one of thos=
e
> permanent Windows compatibility regressions that occured when Windows 9x
> ended, where the terminals can no longer render legacy Arabic text. Even =
if
> the user managed to use registry hacks to set the font to Courier New or
> Simplified Arabic Fixed, it would still use the CP720 mapping which is no=
t
> compatible with the Windows 9x set.
>
>
> It appears that in the Windows 9x Arabic terminal character set, 244
> characters (=E2=80=87=EF=BA=80=EF=BA=81=EF=BA=82=EF=BA=83=EF=BA=84=EF=BA=
=85=EF=BA=87=EF=BA=88=EF=BA=8A=EF=BA=8B=EF=BA=8D=EF=BA=8E=EF=BA=8F=EF=BA=91=
=EF=BA=93=E2=96=BA=E2=97=84=E2=86=95=EF=BA=95=C2=B6=C2=A7=EF=BA=97=EF=BA=99=
=E2=86=91=E2=86=93=E2=86=92=E2=86=90=EF=BA=9B=EF=B9=B0=E2=96=B2=E2=96=BC
> !"#$%&'()*+,-./0123456789:;<=3D>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefg=
hijklmnopqrstuvwxyz{|}~=EF=BA=9D=EF=BA=9F=EF=BA=A1=C3=A9=C3=A2=EF=BA=A3=C3=
=A0=EF=BA=A5=C3=A7=C3=AA=C3=AB=C3=A8=C3=AF=C3=AE=EF=BA=A7=EF=BA=A9=EF=BA=AB=
=EF=BA=AD=EF=BA=AF=C3=B4=EF=BA=B3=C3=BB=C3=B9=EF=BA=B7=EF=BA=BB=C2=A3=EF=BA=
=BF=EF=BB=81=EF=BB=85=EF=BB=89=EF=BB=8A=EF=BB=8B=EF=BB=8C=EF=BB=8D=EF=BB=8E=
=EF=BB=8F=EF=BB=90=EF=BB=91=EF=BB=93=EF=BB=95=EF=BB=97=EF=BB=99=EF=BB=9B=C2=
=AB=C2=BB=EF=B9=B1=E2=96=92=EF=B9=B2=E2=94=82=E2=94=A4=EF=B9=B4=EF=B9=B6=EF=
=B9=B7=EF=B9=B8=D9=A0=D9=A1=D9=A2=D9=A3=EF=B9=B9=EF=B9=BA=E2=94=90=E2=94=94=
=E2=94=B4=E2=94=AC=E2=94=9C=E2=94=80=E2=94=BC=EF=B9=BB=EF=B9=BE=D9=A4=D9=A5=
=D9=A6=D9=A7=D9=A8=D9=A9=D8=8C=EF=B9=BF=EF=B1=9E=EF=B1=9F=EF=B1=A0=EF=B3=B2=
=EF=B1=A1=EF=B3=B3=EF=B1=A2=E2=94=98=E2=94=8C=D8=9B=D8=9F=C2=A4=EF=BB=9D=EF=
=BB=9F=EF=BB=A1=EF=BB=A3=EF=BB=A5=EF=BB=A7=C2=B5=EF=BB=A9=EF=BB=AB=EF=BB=AC=
=EF=BB=AD=EF=BB=AF=EF=BB=B0=EF=BB=B1=EF=BB=B2=EF=BB=B3=EF=B3=B4=EF=B9=BC=EF=
=B9=BD=EF=BA=B1=EF=BA=B5=EF=BA=B9=EF=BA=BD=EF=B9=B3=C2=B0=C2=B7=E2=96=A0=D9=
=80)
> are already in Unicode, but 12 characters are not in Unicode:
>
> =E2=80=A2 6 of them are pieces of lam-alef ligatures (0xDD, 0xDE, 0xF9, 0=
xFB,
> 0xFC, 0xFD)
>
> =E2=80=A2 2 of them are shadda with fathatan ligatures without or with ta=
tweel
> (0xD0, 0xD1)
>
> =E2=80=94 in some legacy Microsoft fonts, shadda with fathatan is mapped =
to
> private use U+E818
>
> =E2=80=A2 4 of them are disunifications of seen/sheen/sad/dad occuring ei=
ther with
> or without tail
>
> =E2=80=94 =EF=B9=B3 (U+FE73 ARABIC TAIL FRAGMENT) was originally encoded =
in Unicode 3.2
> for CP864 compatibility; in that codepage, the forms of seen/sheen/sad/da=
d
> attach to the tail fragment
>
> =E2=80=94 forms with included tail: 0x92, 0x95, 0x98, 0x8A
>
> =E2=80=94 forms without tail (attaching to tail fragment like in CP864): =
0xF3,
> 0xF4, 0xF5, 0xF6
>
>
> If someone tried to make a Win32 console implementation and tried to
> implement both Windows 9x Arabic terminal character set compatibility and
> wide string API (ReadConsoleOutputW) compatibility simultaneously, then
> they would run into the issue that there is currently no standardized
> mapping to handle that scenario. What should Windows 9x Arabic console
> compatible implementations do in that case?
>
>
>
>
>
>
>
>

--000000000000b9f284065129bcae
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Windows can use other ways to map 16-bit codes in its *leg=
acy* Console buffer (using old CHAR_INFO structure), it can perfectly inter=
nally use compatibility characters, or PUAs of the BMP, and still present a=
n API that exposes connforming sequences. You&#39;re talking about an old i=
mplementation that was built even=C2=A0 long before the Arabic script was e=
xtended (and newer scripts using contextual joining behaviors, that have ne=
ver been part of the BMP, shcih as Adlam, and other scripts like Mongolian =
that also may need such sequences with ZWJ/ZWNJ controls, or with other for=
matting characters like those specific to Mongolian like FVS1...FVS4 and MV=
S, or those common to many Bhramic scripts, that the *legacy* Console did n=
ot support.<br>The *legacy* console was not built to support more than one =
plane (including many CJK cgaracters). The newer console can!</div><br><div=
 class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmai=
l_attr">Le=C2=A0mer. 6 mai 2026 =C3=A0=C2=A018:37, <a href=3D"mailto:piotru=
[email protected]">[email protected]</a> &lt;<a href=3D"mailto:piotrunio-20=
[email protected]">[email protected]</a>&gt; a =C3=A9crit=C2=A0:<br></div><blockq=
uote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1p=
x solid rgb(204,204,204);padding-left:1ex"><p>Have you read the L2/26-077 p=
roposal? Using ZWJ or ZWNJ would not work for the compatibility purposes at=
 all as already explained in the proposal. This is because ZWJ or ZWNJ woul=
d take the space of one character tile in the CHAR_INFO structure. Suppose =
that you&#39;re trying to map 0xD0 from FP164 to a sequence of U+FE7C U+200=
D U+064B (=EF=B9=BC=E2=80=8D=D9=8B). The legacy application fills the 80=C3=
=9725 screen with all 0xD0 tiles. You subsequently try to capture the tiles=
 with a Win32 program by using ReadConsoleOutputA into an 80=C3=9725 buffer=
 of 2000 tiles. This succeeds and captures 0xD0 into all the tiles. You the=
n try to capture the tiles using ReadConsoleOutputW into an 80=C3=9725 buff=
er. Each sequence U+FE7C U+200D U+064B would take a sequence of three CHAR_=
INFO structures to store, meaning 6000 such structures for the whole screen=
. But the 80=C3=9725 buffer has only room for 2000 instances of the structu=
re (one per character tile). Since CHAR_INFO stores 16-bit character code, =
by that same logic the compatibility characters would have to be in BMP for=
 it to work. In Windows 95 Vietnamese and Windows 95 Thai, there are instan=
ces where one character tile takes multiple CHAR_INFO structures, causing v=
isual width to be smaller than logical width, and when that happens, the re=
maining space at the end of the line is left blank, allowing for CP1258/CP8=
74 combining characters in those systems to map 1:1 to their Unicode equiva=
lents. Windows 3.1/95/98/ME Arabic don&#39;t work that way and don&#39;t us=
e combining characters or ZWJ sequences, so visual width is always equivale=
nt to logical width, each character tile maps 1:1 to a CHAR_INFO structure =
and all characters may fill the entire line, which would be impossible if s=
ome of those characters were mapped to composition sequences or non-BMP cha=
racters. Since there is currently no sufficient evidence of user community =
that would need to use those mappings, there are no plans for those charact=
ers to be added to Unicode, and therefore the only solution for the ReadCon=
soleOutputW to work properly in this case is to use agreed upon private use=
 mappings for those compatibility characters.</p><p><br></p><div><blockquot=
e style=3D"padding-top:12px"><p style=3D"padding-bottom:12px"><strong>Dnia =
06 maja 2026 18:06</strong> <a href=3D"mailto:[email protected]" rel=
=3D"noopener noreferrer nofollow" target=3D"_blank"><span style=3D"margin-l=
eft:4px">Philippe Verdy via Unicode</span></a><span style=3D"margin-left:4p=
x"> &lt; <a href=3D"mailto:[email protected]" target=3D"_blank">unic=
[email protected]</a> &gt;</span> napisa=C5=82(a):</p><div id=3D"m_-2650=
882641749569987gwpb67625a5"><div id=3D"m_-2650882641749569987gwpb67625a5h">=
<div><div dir=3D"auto"><p>You actually don&#39;t need any new compatibility=
 characters for Arabic contextual forms, or for other contextual forms in o=
ther joining scripts (like Adlam, or even Mongolian whichbis a LTR script).=
</p><div dir=3D"auto"><br></div><div dir=3D"auto">You just have to prepend =
or append a ZWJ or ZWNJ formatting control to the unified letter if you wan=
t to override its default contextual presentation form.</div><div dir=3D"au=
to"><br></div></div><p><br></p><div><div dir=3D"ltr">Le mar. 5 mai 2026, 00=
:48, Asmus Freytag via Unicode &lt;<a href=3D"mailto:[email protected]=
rg" rel=3D"noopener noreferrer" target=3D"_blank">[email protected]<=
/a>&gt; a =C3=A9crit=C2=A0:<br></div><blockquote style=3D"margin:0px 0px 0p=
x 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p>Th=
e issue at hand is the distinction between a theoretical gap and real-life =
problem.</p><p><br></p><p>You have demonstrated that there are specificatio=
ns that, if chained in the right way, can lead to ambiguities or gaps in in=
terchange.</p><p><br></p><p>What we don&#39;t have is an actual use case wi=
th real-life consequences for a set of existing users, not hypothetical one=
s.</p><p><br></p><p>When it comes to encoding decisions based on existing d=
ocuments, there is a strong presumption that once sufficiently many documen=
ts exist that contain a character, that this character will be needed in di=
gitizing these documents, whether immediately, or eventually (e.g. in the c=
ase of future scholarly studies). Also, the texts themselves exist, barring=
 accidents, in permanence. Therefore, it is justified to consider irrevocab=
ly allocating a character that will map to this source in perpetuity, even =
though each encoded character carries a small cost for implementers.<br><br=
>However, when it comes to legacy characters, there&#39;s an additional cos=
t that is imposed, and that is based on the fact that characters that are e=
ncoded solely for compatibility will usually violate one or more of the oth=
er encoding principles, something that incrementally complicates the standa=
rd. Even for people who never intend to use that character.<br><br>Therefor=
e, the SEW is on solid ground when it demands not only a hypothetical scena=
rio, but evidence of actual impact on actual users. Not only whether some a=
pplication could invoke an API, but whether such applications exist and are=
 used today to access documents encoded using the legacy characters in a wa=
y that is compromised irreparably by not having an encoding for them.</p><p=
><br></p><p>A./</p><p><br></p><p><br></p><p>On 5/4/2026 10:24 AM, <a href=
=3D"mailto:[email protected]" rel=3D"noreferrer" target=3D"_blank">piotr=
[email protected]</a> via Unicode wrote:<br></p><blockquote type=3D"cite"><p>=
In UTC 187 Minutes, &quot;<span style=3D"color:rgb(0,0,0);font-family:&quot=
;DMCA Sans Serif 10.0 dev1&quot;;font-size:medium;font-style:normal;font-va=
riant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spac=
ing:normal;text-align:start;text-indent:0px;text-transform:none;white-space=
:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-colo=
r:initial;float:none;display:inline">Asmus Freytag noted that the fact that=
 lists of things existed in the past does not make these things plain text.=
 Ned Holbrook pointed out that the purported issue occurs in a closed syste=
m, not in public interchange.</span>&quot;. However, the arguments in the p=
roposal do not merely hinge on the encodings being lists of characters, but=
 specifically points out methods to interchange text, including an example =
of copying terminal output and pasting to Notepad, where the copying invoke=
s the mapping of the current terminal codepage to UCS-2 (as is CHAR_INFO co=
mpatible) and the pasting writes it into plain text. Win32 is also not a cl=
osed system, as Win32 can capture the tiles of the output of Windows 3.1 Ar=
abic DOS/Win16 programs and Windows 95/98/ME Arabic DOS/Win16/Win32 program=
s, but Win32 can also interact with public text interchange systems by read=
ing and writing to files and network. I&#39;m not saying that Unicode absol=
utely must include those characters, but those kinds of misleading claims a=
re causing users to misunderstand what the proposal is about, and I don&#39=
;t want Unicode to be relying on uninformed decisions to evaluate proposals=
.</p><p><br></p><div><blockquote style=3D"padding-top:12px"><p style=3D"pad=
ding-bottom:12px"><strong>Dnia 18 kwietnia 2026 13:36</strong> <a href=3D"m=
ailto:[email protected]" rel=3D"noopener noreferrer nofollow norefer=
rer" target=3D"_blank"><span style=3D"margin-left:4px">[email protected]=
 via Unicode</span></a><span style=3D"margin-left:4px"> </span><a href=3D"m=
ailto:[email protected]" rel=3D"noreferrer" target=3D"_blank"><span =
style=3D"margin-left:4px">&lt; [email protected] &gt;</span></a> nap=
isa=C5=82(a):</p><div id=3D"m_-2650882641749569987gwpb67625a5_m_53710848870=
79304582gwpbf2d884a"><div id=3D"m_-2650882641749569987gwpb67625a5_m_5371084=
887079304582gwpbf2d884ah"><div><p>The SEW subsequently explained that the a=
ctual reason is due to insufficient evidence of user community that would n=
eed to use the resulting mapping. Despite Win32 being a highly popular plat=
form with plenty of backwards compatibility and native UCS-2 terminal suppo=
rt, the specific use cases of installing codepages into Windows NT and usin=
g terminal tiles from Windows 3.1/95/98/ME are not sufficiently documented,=
 making it difficult for any user communities to form around it. So it seem=
s like the idea of standardizing legacy Arabic terminal BMP mappings is a d=
ead end for now.</p><p><br></p><div><blockquote style=3D"padding-top:12px">=
<p style=3D"padding-bottom:12px"><strong>Dnia 17 kwietnia 2026 22:59</stron=
g> <a href=3D"mailto:[email protected]" rel=3D"noopener noreferrer n=
ofollow noreferrer" target=3D"_blank"><span style=3D"margin-left:4px">piotr=
[email protected] via Unicode</span></a><span style=3D"margin-left:4px"> </sp=
an><a href=3D"mailto:[email protected]" rel=3D"noreferrer" target=3D=
"_blank"><span style=3D"margin-left:4px">&lt; [email protected] &gt;=
</span></a> napisa=C5=82(a):</p><div id=3D"m_-2650882641749569987gwpb67625a=
5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8"><div id=3D"m_-2650882641749=
569987gwpb67625a5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8h"><div><p>Th=
e Recommendations in L2/26-100 claim that Microsoft&#39;s documentation of =
legacy Arabic encodings is available at <a href=3D"https://learn.microsoft.=
com/en-us/typography/legacy/legacy_arabic_fonts" rel=3D"noreferrer" target=
=3D"_blank">https://learn.microsoft.com/en-us/typography/legacy/legacy_arab=
ic_fonts</a>. However, that article only demonstrates two encodings of True=
Type fonts, which are used in Windows 3.1 but are completely different from=
 the eight terminal encodings. Unlike the TrueType encodings which represen=
t internal shaping mappings and are not used for text interchange, the term=
inal encodings have been demonstrated to be directly used in text interchan=
ge through int 10h and ReadConsoleOutputA/WriteConsoleOutputA as already de=
monstrated in L2/26-077. The Recommendations also claim that the proposal d=
oes not demonstrate any need for interchange or encoding, but the proposal =
actually demonstrated such a need due to the logical extension of the Win32=
 terminal API to the functions ReadConsoleOutputW/WriteConsoleOutputW, whic=
h are in Windows NT and may be used on the output of previously ran program=
s (including those that used the legacy Arabic terminal encodings), which g=
iven the CHAR_INFO structure, therefore implies a need for all the tiles to=
 map to BMP for interchange. I&#39;m not objecting to the SEW&#39;s conclus=
ion of &quot;Users are expected to use PUA.&quot;, which can indeed be used=
 to provide a mapping even if not standardized, but the reasoning given was=
 flawed.</p><p><br></p><div><blockquote style=3D"padding-top:12px"><p style=
=3D"padding-bottom:12px"><strong>Dnia 09 stycznia 2026 17:25</strong> <a hr=
ef=3D"mailto:[email protected]" rel=3D"noreferrer" target=3D"_blank"><sp=
an style=3D"margin-left:4px">[email protected]</span></a><span style=3D"=
margin-left:4px"> </span><a href=3D"mailto:[email protected]" rel=3D"nor=
eferrer" target=3D"_blank"><span style=3D"margin-left:4px">&lt; piotrunio-2=
[email protected] &gt;</span></a> napisa=C5=82(a):</p><div id=3D"m_-265088264174956=
9987gwpb67625a5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8_gwpa05276c7"><=
div id=3D"m_-2650882641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884=
a_gwp2281a7f8_gwpa05276c7h"><div><div id=3D"m_-2650882641749569987gwpb67625=
a5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8_gwpa05276c7_gwpa8b5f718"><d=
iv id=3D"m_-2650882641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884a=
_gwp2281a7f8_gwpa05276c7_gwpa8b5f718h"><div><div id=3D"m_-26508826417495699=
87gwpb67625a5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8_gwpa05276c7_gwpa=
8b5f718_gwpa8b5f718"><div id=3D"m_-2650882641749569987gwpb67625a5_m_5371084=
887079304582gwpbf2d884a_gwp2281a7f8_gwpa05276c7_gwpa8b5f718_gwpa8b5f718h"><=
div><div id=3D"m_-2650882641749569987gwpb67625a5_m_5371084887079304582gwpbf=
2d884a_gwp2281a7f8_gwpa05276c7_gwpa8b5f718_gwpa8b5f718_gwpa8b5f718"><div id=
=3D"m_-2650882641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884a_gwp2=
281a7f8_gwpa05276c7_gwpa8b5f718_gwpa8b5f718_gwpa8b5f718h"><div><p>The follo=
wing Win32 C code will output 256 characters in system console codepage int=
o the character grid, capture those character tiles in UCS-2 if possible, a=
nd then output the current console codepage number.<br></p><p><br></p><p>#i=
nclude &lt;windows.h&gt;<br>#include &lt;stdio.h&gt;<br>int main(){<br>HAND=
LE hConsole=3DGetStdHandle(STD_OUTPUT_HANDLE);<br>CHAR_INFO screen[256];<br=
>COORD size=3D{16,16,};<br>COORD pos=3D{0,0,};<br>SMALL_RECT rect=3D{0,0,15=
,15,};<br>for(int i=3D0;i&lt;256;i++){<br>screen[i].Attributes=3D0xF0;<br>s=
creen[i].Char.AsciiChar=3Di;<br>}<br>WriteConsoleOutputA(hConsole,screen,si=
ze,pos,&amp;rect);<br>CHAR_INFO screenu[256];<br>if(ReadConsoleOutputW(hCon=
sole,screenu,size,pos,&amp;rect)){<br>for(int i=3D0;i&lt;256;i++) printf(&q=
uot;%04X &quot;,screenu[i].Char.UnicodeChar);<br>}<br>else{<br>printf(&quot=
;error %08X\n&quot;,GetLastError());<br>}<br>printf(&quot;codepage %u&quot;=
,GetConsoleOutputCP());<br>}<br><br></p><p>In most cases, whenever a legacy=
 Win32 codepage is used, the application can run on Windows NT to capture t=
he UCS-2 mapping of those character cells to the BMP (although for CJK code=
pages a more complex setup would be necessary due to thousands of fullwidth=
 characters with 2-byte sequences).<br></p><p><br></p><p>However, in Arabic=
 versions of Windows 9x (95/98/ME) the resulting character set has many pre=
sentation forms that are not in Unicode. This is the result when running on=
 Windows ME:=C2=A0<a href=3D"https://i.imgur.com/QFm3SkI.png" rel=3D"noopen=
er noreferrer noreferrer" target=3D"_blank">https://i.imgur.com/QFm3SkI.png=
</a>=C2=A0in 10=C3=9720 font, <a href=3D"https://i.imgur.com/KUbLQ0A.png" r=
el=3D"noopener noreferrer noreferrer" target=3D"_blank">https://i.imgur.com=
/KUbLQ0A.png</a>=C2=A0in 10=C3=9718 font (same result also appears in Windo=
ws 95/98). 5=C3=9712, 7=C3=9712, 8=C3=9712, 10=C3=9718, 10=C3=9720, and 12=
=C3=9716 bitmap fonts have been attested with that character set (VGAOEM.FO=
N, 8514OEM.FON, DOSAPP.FON). The 10=C3=9720 font has slightly different map=
ping than the other sizes: 0x93 is =C3=B6 instead of =C3=B4, and 0x97 is mi=
ssing (causing the following characters on the same line to be drawn at the=
 wrong position). It also claims to be using codepage 720, but many charact=
ers differ from their CP720 mappings, including the bundled=C2=A0CP_720.NLS=
 mappings (for example, =D9=80 (U+0640 ARABIC TATWEEL) is 0x95 in CP720, bu=
t in the console 0x95 is =D8=B4 instead, and the tatweel is at 0xFF). On Wi=
ndows 9x,=C2=A0ReadConsoleOutputW is not supported so the UCS-2 mappings of=
 the console character tiles cannot be captured (error 0x00000078 ERROR_CAL=
L_NOT_IMPLEMENTED).<br></p></div></div></div><p><br></p><p>When that progra=
m runs on Arabic versions of Windows NT, the visual output is of the CP437 =
character set if one of the bundled bitmap fonts is used (<a href=3D"https:=
//i.imgur.com/RxjtxMH.png" rel=3D"noopener noreferrer noreferrer" target=3D=
"_blank">https://i.imgur.com/RxjtxMH.png</a>), or the CP720 set if Lucida C=
onsole is used, with the Arabic letters either having glitchy font substitu=
tion (NT 4.0, NT 5.0/2000) or the .notdef glyph (NT 5.1/XP and up). In fact=
, it seems that the only Arabic bitmap fonts that occur in Windows NT are C=
P1256 fonts, which are not used in terminals. So this appears to be one of =
those permanent Windows compatibility regressions that occured when Windows=
 9x ended, where the terminals can no longer render legacy Arabic text. Eve=
n if the user managed to use registry hacks to set the font to Courier New =
or Simplified Arabic Fixed, it would still use the CP720 mapping which is n=
ot compatible with the Windows 9x set.<br></p></div></div><p><br></p></div>=
</div></div></div><div id=3D"m_-2650882641749569987gwpb67625a5_m_5371084887=
079304582gwpbf2d884a_gwp2281a7f8_gwpa05276c7_gwpa8b5f718"><div id=3D"m_-265=
0882641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8_gw=
pa05276c7_gwpa8b5f718h"><div><p>It appears that in the Windows 9x Arabic te=
rminal character set, 244 characters (=E2=80=87=EF=BA=80=EF=BA=81=EF=BA=82=
=EF=BA=83=EF=BA=84=EF=BA=85=EF=BA=87=EF=BA=88=EF=BA=8A=EF=BA=8B=EF=BA=8D=EF=
=BA=8E=EF=BA=8F=EF=BA=91=EF=BA=93=E2=96=BA=E2=97=84=E2=86=95=EF=BA=95=C2=B6=
=C2=A7=EF=BA=97=EF=BA=99=E2=86=91=E2=86=93=E2=86=92=E2=86=90=EF=BA=9B=EF=B9=
=B0=E2=96=B2=E2=96=BC !&quot;#$%&amp;&#39;()*+,-./0123456789:;&lt;=3D&gt;?@=
ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~=EF=BA=9D=EF=
=BA=9F=EF=BA=A1=C3=A9=C3=A2=EF=BA=A3=C3=A0=EF=BA=A5=C3=A7=C3=AA=C3=AB=C3=A8=
=C3=AF=C3=AE=EF=BA=A7=EF=BA=A9=EF=BA=AB=EF=BA=AD=EF=BA=AF=C3=B4=EF=BA=B3=C3=
=BB=C3=B9=EF=BA=B7=EF=BA=BB=C2=A3=EF=BA=BF=EF=BB=81=EF=BB=85=EF=BB=89=EF=BB=
=8A=EF=BB=8B=EF=BB=8C=EF=BB=8D=EF=BB=8E=EF=BB=8F=EF=BB=90=EF=BB=91=EF=BB=93=
=EF=BB=95=EF=BB=97=EF=BB=99=EF=BB=9B=C2=AB=C2=BB=EF=B9=B1=E2=96=92=EF=B9=B2=
=E2=94=82=E2=94=A4=EF=B9=B4=EF=B9=B6=EF=B9=B7=EF=B9=B8=D9=A0=D9=A1=D9=A2=D9=
=A3=EF=B9=B9=EF=B9=BA=E2=94=90=E2=94=94=E2=94=B4=E2=94=AC=E2=94=9C=E2=94=80=
=E2=94=BC=EF=B9=BB=EF=B9=BE=D9=A4=D9=A5=D9=A6=D9=A7=D9=A8=D9=A9=D8=8C=EF=B9=
=BF=EF=B1=9E=EF=B1=9F=EF=B1=A0=EF=B3=B2=EF=B1=A1=EF=B3=B3=EF=B1=A2=E2=94=98=
=E2=94=8C=D8=9B=D8=9F=C2=A4=EF=BB=9D=EF=BB=9F=EF=BB=A1=EF=BB=A3=EF=BB=A5=EF=
=BB=A7=C2=B5=EF=BB=A9=EF=BB=AB=EF=BB=AC=EF=BB=AD=EF=BB=AF=EF=BB=B0=EF=BB=B1=
=EF=BB=B2=EF=BB=B3=EF=B3=B4=EF=B9=BC=EF=B9=BD=EF=BA=B1=EF=BA=B5=EF=BA=B9=EF=
=BA=BD=EF=B9=B3=C2=B0=C2=B7=E2=96=A0=D9=80) are already in Unicode, but 12 =
characters are not in Unicode:<br></p><p>=E2=80=A2 6 of them are pieces of =
lam-alef ligatures (0xDD, 0xDE, 0xF9, 0xFB, 0xFC, 0xFD)<br></p><p>=E2=80=A2=
 2 of them are shadda with fathatan ligatures without or with tatweel (0xD0=
, 0xD1)<br></p><p>=E2=80=94 in some legacy Microsoft fonts, shadda with fat=
hatan is mapped to private use U+E818<br></p><p>=E2=80=A2 4 of them are dis=
unifications of seen/sheen/sad/dad occuring either with or without tail<br>=
</p><p>=E2=80=94=C2=A0=EF=B9=B3 (U+FE73 ARABIC TAIL FRAGMENT) was originall=
y encoded in Unicode 3.2 for CP864 compatibility; in that codepage, the for=
ms of=C2=A0seen/sheen/sad/dad attach to the tail fragment<br></p><p>=E2=80=
=94 forms with included tail:=C2=A00x92, 0x95, 0x98, 0x8A<br></p><p>=E2=80=
=94 forms without tail (attaching to tail fragment like in CP864):=C2=A00xF=
3, 0xF4, 0xF5, 0xF6<br></p></div><p><br></p></div><p>If someone tried to ma=
ke a Win32 console implementation and tried to implement both Windows 9x Ar=
abic terminal character set compatibility and wide string API (ReadConsoleO=
utputW) compatibility simultaneously, then they would run into the issue th=
at there is currently no standardized mapping to handle that scenario. What=
 should Windows 9x Arabic console compatible implementations do in that cas=
e?<br></p></div><p><br></p></div></div></div></blockquote></div><p><br></p>=
</div></div></div></blockquote></div><p><br></p></div></div></div></blockqu=
ote></div><p><br></p></blockquote><p><br></p></div></blockquote></div></div=
></div></div></blockquote></div><p><br></p></blockquote></div>

--000000000000b9f284065129bcae--