| Newsgroups |
gmane.text.unicode.general |
| Message-ID |
<[email protected]> |
--2JUQTNLMGPDCIFGWGPHMWnhgwp
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8
Dnia 07 maja 2026 20:46 Peter Constable via Unicode < [email protected]=
nicode.org > napisa=C5=82(a): > This is not of any interest, because=
the current Microsoft company is not even close ideologically to the past =
version of Microsoft that originally made the Arabic terminals, in fact the=
y're not even ideologically compatible with each other. Of course it m=
ight not be of interest to you. You don=E2=80=99t seem interested in others=
=E2=80=99 reasoning about this unless it aligns to how you are thinking abo=
ut it. I didn't think that the evidence of user community in the propos=
al was insufficient at first, but once that was explained to me, I accepted=
that reasoning. In fact, I evaluate users' feedback all the time when =
I develop fonts, other software, and proposals, and I make changes to fit n=
ew evidence. When you brought up a hypothetical Microsoft voting result, I =
didn't think you were making any argument at all, because back then you=
haven't explained why specifically Microsoft would vote that way or ho=
w that specifically relates to the contents of the proposal, making it not =
a constructive claim at first. One point about current MS not being even c=
lose =E2=80=9Cideologically=E2=80=9D to past MS (very long past) that _ I _=
think is of interest is that, for UTC to decide to add new legacy characte=
rs, one consideration is what the new characters will entail for product su=
pport and interoperability. There=E2=80=99s a cost / benefit analysis to be=
done. In the 1980s, there was a reason why legacy implementations made sen=
se. But today, the cost / benefit analysis doesn=E2=80=99t weight at all in=
favour of encoding new legacy characters: Microsoft has been supporting Un=
icode in products for over 30 years, and in all that time there hasn=E2=80=
=99t been any identifiable customer need for encoding these additional lega=
cy Arabic text elements as separate characters =E2=80=89 =E2=80=94 =E2=80=
=89 i.e., benefits are extremely low to zero. But the costs for the new cha=
racters would certainly not be zero. You might think this is a false argu=
ment. Others might think otherwise. By false arguments I meant things like=
claiming that the characters can be represented with ZWJ/ZWNJ sequences or=
that the claims that Microsoft's documentation of legacy Arabic encodi=
ngs is available at https://learn.microsoft.com/en=02us/typography/legacy/=
legacy_arabic_fonts , which I had already debunked. Now that you explained =
the reasoning involving cost to benefit ratio, you've just expressed a =
legitimate argument, which is similar to what Asmus Freytag had said. Pe=
ter From: [email protected] <[email protected]> Sent: Thur=
sday, May 7, 2026 9:03 AM To: Peter Constable via Unicode <unicode@cor=
p.unicode.org>; Peter Constable <[email protected]>; Philippe Verdy &=
lt;[email protected]> Subject: RE: Odp: Pd: Missing legacy Arabic encod=
ing This is not of any interest, because the current Microsoft company is =
not even close ideologically to the past version of Microsoft that original=
ly made the Arabic terminals, in fact they're not even ideologically co=
mpatible with each other. There's no need to speculate on hypothetical =
voting scenarios involving giant companies. What matters are logical argume=
nts involving encoding policy, and the prevailing reason is that there is n=
o sufficient evidence of user community that would need to interchange text=
from those platforms into UCS-2 terminals. It is still necessary to debunk=
false arguments in order to prevent future decisions from being made incor=
rectly, because even if they wouldn't affect the outcome of this propos=
al, they could still improperly influence the evaluation of future proposal=
s. Dnia 07 maja 2026 17:36 Peter Constable via Unicode < unicode@co=
rp.unicode.org > napisa=C5=82(a): Just in case it might be of interest,=
if a motion on this proposal were raised in a UTC meeting, I suspect Micro=
soft would vote against encoding. =C2=A0 =C2=A0 Peter =C2=A0 From: Unicode=
< [email protected] > On Behalf Of piotrunio-2004=
@wp.pl via Unicode Sent: May 6, 2026 11:09 AM To: Philippe Verdy via U=
nicode < [email protected] >; Philippe Verdy < verdyp@gma=
il.com > Subject: Re: Odp: Pd: Missing legacy Arabic encoding The Rea=
dConsoleOutputW function, by definition, captures the tiles into the lpBuff=
er, which is a random access array of CHAR_INFO structure, whose horizontal=
and vertical size is specified in dwBufferSize. Since lpBuffer is random a=
ccess, this implies one CHAR_INFO structure per character tile. The API the=
refore fundamentally imposes a strict memory layout that cannot be violated=
. The fact that some Unix-like terminals such as Windows Terminal may suppo=
rt features outside the scope of the CHAR_INFO structure for compatibility =
with ANSI escape codes or WSL programs does not invalidate the compatibilit=
y considerations for legacy DOS/Win16/Win32 programs that require all chara=
cter tiles to fit in the CHAR_INFO structure for random access, because 4 b=
yte CHAR_INFO structure of Win32 is intended to be a fully backwards compat=
ible extension of the 2 byte VGA text mode tile structure of DOS/Win16. Dn=
ia 06 maja 2026 19:58 Philippe Verdy via Unicode < [email protected]=
ode.org > napisa=C5=82(a): Windows can use other ways to map 16-bit cod=
es in its *legacy* Console buffer (using old CHAR_INFO structure), it can p=
erfectly internally use compatibility characters, or PUAs of the BMP, and s=
till present an API that exposes connforming sequences. You're talking =
about an old implementation that was built even=C2=A0 long before the Arabi=
c script was extended (and newer scripts using contextual joining behaviors=
, that have never been part of the BMP, shcih as Adlam, and other scripts l=
ike Mongolian that also may need such sequences with ZWJ/ZWNJ controls, or =
with other formatting characters like those specific to Mongolian like FVS1=
...FVS4 and MVS, or those common to many Bhramic scripts, that the *legacy*=
Console did not support. The *legacy* console was not built to support mo=
re than one plane (including many CJK cgaracters). The newer console can! =
Le=C2=A0mer. 6 mai 2026 =C3=A0=C2=A018:37, [email protected] < pi=
[email protected] > a =C3=A9crit=C2=A0: Have you read the L2/26-077 pro=
posal? Using ZWJ or ZWNJ would not work for the compatibility purposes at a=
ll as already explained in the proposal. This is because ZWJ or ZWNJ would =
take the space of one character tile in the CHAR_INFO structure. Suppose th=
at you're trying to map 0xD0 from FP164 to a sequence of U+FE7C U+200D =
U+064B ( =EF=B9=BC=E2=80=8D=D9=8B ). The legacy application fills the 80=C3=
=9725 screen with all 0xD0 tiles. You subsequently try to capture the tiles=
with a Win32 program by using ReadConsoleOutputA into an 80=C3=9725 buffer=
of 2000 tiles. This succeeds and captures 0xD0 into all the tiles. You the=
n try to capture the tiles using ReadConsoleOutputW into an 80=C3=9725 buff=
er. Each sequence U+FE7C U+200D U+064B would take a sequence of three CHAR_=
INFO structures to store, meaning 6000 such structures for the whole screen=
. But the 80=C3=9725 buffer has only room for 2000 instances of the structu=
re (one per character tile). Since CHAR_INFO stores 16-bit character code, =
by that same logic the compatibility characters would have to be in BMP for=
it to work. In Windows 95 Vietnamese and Windows 95 Thai, there are instan=
ces where one character tile takes multiple CHAR_INFO structures, causing v=
isual width to be smaller than logical width, and when that happens, the re=
maining space at the end of the line is left blank, allowing for CP1258/CP8=
74 combining characters in those systems to map 1:1 to their Unicode equiva=
lents. Windows 3.1/95/98/ME Arabic don't work that way and don't us=
e combining characters or ZWJ sequences, so visual width is always equivale=
nt to logical width, each character tile maps 1:1 to a CHAR_INFO structure =
and all characters may fill the entire line, which would be impossible if s=
ome of those characters were mapped to composition sequences or non-BMP cha=
racters. Since there is currently no sufficient evidence of user community =
that would need to use those mappings, there are no plans for those charact=
ers to be added to Unicode, and therefore the only solution for the ReadCon=
soleOutputW to work properly in this case is to use agreed upon private use=
mappings for those compatibility characters. Dnia 06 maja 2026 18:06 Phi=
lippe Verdy via Unicode < [email protected] > napisa=C5=82=
(a): You actually don't need any new compatibility characters for Arabi=
c contextual forms, or for other contextual forms in other joining scripts =
(like Adlam, or even Mongolian whichbis a LTR script). You just have to pr=
epend or append a ZWJ or ZWNJ formatting control to the unified letter if y=
ou want to override its default contextual presentation form. Le mar. 5 m=
ai 2026, 00:48, Asmus Freytag via Unicode < [email protected] &g=
t; a =C3=A9crit=C2=A0: The issue at hand is the distinction between a theor=
etical gap and real-life problem. You have demonstrated that there are spe=
cifications that, if chained in the right way, can lead to ambiguities or g=
aps in interchange. What we don't have is an actual use case with real=
-life consequences for a set of existing users, not hypothetical ones. Whe=
n it comes to encoding decisions based on existing documents, there is a st=
rong presumption that once sufficiently many documents exist that contain a=
character, that this character will be needed in digitizing these document=
s, whether immediately, or eventually (e.g. in the case of future scholarly=
studies). Also, the texts themselves exist, barring accidents, in permanen=
ce. Therefore, it is justified to consider irrevocably allocating a charact=
er that will map to this source in perpetuity, even though each encoded cha=
racter carries a small cost for implementers. However, when it comes to l=
egacy characters, there's an additional cost that is imposed, and that =
is based on the fact that characters that are encoded solely for compatibil=
ity will usually violate one or more of the other encoding principles, some=
thing that incrementally complicates the standard. Even for people who neve=
r intend to use that character. Therefore, the SEW is on solid ground whe=
n it demands not only a hypothetical scenario, but evidence of actual impac=
t on actual users. Not only whether some application could invoke an API, b=
ut whether such applications exist and are used today to access documents e=
ncoded using the legacy characters in a way that is compromised irreparably=
by not having an encoding for them. A./ On 5/4/2026 10:24 AM, piotrun=
[email protected] via Unicode wrote: In UTC 187 Minutes, " Asmus Freytag n=
oted that the fact that lists of things existed in the past does not make t=
hese things plain text. Ned Holbrook pointed out that the purported issue o=
ccurs in a closed system, not in public interchange. ". However, the ar=
guments in the proposal do not merely hinge on the encodings being lists of=
characters, but specifically points out methods to interchange text, inclu=
ding an example of copying terminal output and pasting to Notepad, where th=
e copying invokes the mapping of the current terminal codepage to UCS-2 (as=
is CHAR_INFO compatible) and the pasting writes it into plain text. Win32 =
is also not a closed system, as Win32 can capture the tiles of the output o=
f Windows 3.1 Arabic DOS/Win16 programs and Windows 95/98/ME Arabic DOS/Win=
16/Win32 programs, but Win32 can also interact with public text interchange=
systems by reading and writing to files and network. I'm not saying th=
at Unicode absolutely must include those characters, but those kinds of mis=
leading claims are causing users to misunderstand what the proposal is abou=
t, and I don't want Unicode to be relying on uninformed decisions to ev=
aluate proposals. Dnia 18 kwietnia 2026 13:36 [email protected] via Un=
icode< [email protected] > napisa=C5=82(a): The SEW subsequen=
tly explained that the actual reason is due to insufficient evidence of use=
r community that would need to use the resulting mapping. Despite Win32 bei=
ng a highly popular platform with plenty of backwards compatibility and nat=
ive UCS-2 terminal support, the specific use cases of installing codepages =
into Windows NT and using terminal tiles from Windows 3.1/95/98/ME are not =
sufficiently documented, making it difficult for any user communities to fo=
rm around it. So it seems like the idea of standardizing legacy Arabic term=
inal BMP mappings is a dead end for now. Dnia 17 kwietnia 2026 22:59 piot=
[email protected] via Unicode< [email protected] > napisa=C5=
=82(a): The Recommendations in L2/26-100 claim that Microsoft's documen=
tation of legacy Arabic encodings is available at learn.microsoft.com http=
s://learn.microsoft.com/en-us/typography/legacy/legacy_arabic_fonts . Howev=
er, that article only demonstrates two encodings of TrueType fonts, which a=
re used in Windows 3.1 but are completely different from the eight terminal=
encodings. Unlike the TrueType encodings which represent internal shaping =
mappings and are not used for text interchange, the terminal encodings have=
been demonstrated to be directly used in text interchange through int 10h =
and ReadConsoleOutputA/WriteConsoleOutputA as already demonstrated in L2/26=
-077. The Recommendations also claim that the proposal does not demonstrate=
any need for interchange or encoding, but the proposal actually demonstrat=
ed such a need due to the logical extension of the Win32 terminal API to th=
e functions ReadConsoleOutputW/WriteConsoleOutputW, which are in Windows NT=
and may be used on the output of previously ran programs (including those =
that used the legacy Arabic terminal encodings), which given the CHAR_INFO =
structure, therefore implies a need for all the tiles to map to BMP for int=
erchange. I'm not objecting to the SEW's conclusion of "Users a=
re expected to use PUA.", which can indeed be used to provide a mapping=
even if not standardized, but the reasoning given was flawed. Dnia 09 sty=
cznia 2026 17:25 [email protected]< [email protected] > napis=
a=C5=82(a): The following Win32 C code will output 256 characters in system=
console codepage into the character grid, capture those character tiles in=
UCS-2 if possible, and then output the current console codepage number. #=
include <windows.h> #include <stdio.h> int main(){ HANDLE hC=
onsole=3DGetStdHandle(STD_OUTPUT_HANDLE); CHAR_INFO screen[256]; COORD si=
ze=3D{16,16,}; COORD pos=3D{0,0,}; SMALL_RECT rect=3D{0,0,15,15,}; for(i=
nt i=3D0;i<256;i++){ screen[i].Attributes=3D0xF0; screen[i].Char.Ascii=
Char=3Di; } WriteConsoleOutputA(hConsole,screen,size,pos,&rect); CHA=
R_INFO screenu[256]; if(ReadConsoleOutputW(hConsole,screenu,size,pos,&=
rect)){ for(int i=3D0;i<256;i++) printf("%04X ",screenu[i].Char=
.UnicodeChar); } else{ printf("error %08X\n",GetLastError()); }=
printf("codepage %u",GetConsoleOutputCP()); } In most cases, whe=
never a legacy Win32 codepage is used, the application can run on Windows N=
T to capture the UCS-2 mapping of those character cells to the BMP (althoug=
h for CJK codepages a more complex setup would be necessary due to thousand=
s of fullwidth characters with 2-byte sequences). However, in Arabic versi=
ons of Windows 9x (95/98/ME) the resulting character set has many presentat=
ion forms that are not in Unicode. This is the result when running on Windo=
ws ME:=C2=A0 i.imgur.com https://i.imgur.com/QFm3SkI.png =C2=A0in 10=C3=972=
0 font, i.imgur.com https://i.imgur.com/KUbLQ0A.png =C2=A0in 10=C3=9718 fo=
nt (same result also appears in Windows 95/98). 5=C3=9712, 7=C3=9712, 8=C3=
=9712, 10=C3=9718, 10=C3=9720, and 12=C3=9716 bitmap fonts have been attest=
ed with that character set (VGAOEM.FON, 8514OEM.FON, DOSAPP.FON). The 10=C3=
=9720 font has slightly different mapping than the other sizes: 0x93 is =C3=
=B6 instead of =C3=B4, and 0x97 is missing (causing the following character=
s on the same line to be drawn at the wrong position). It also claims to be=
using codepage 720, but many characters differ from their CP720 mappings, =
including the bundled=C2=A0CP_720.NLS mappings (for example, =D9=80 (U+064=
0 ARABIC TATWEEL) is 0x95 in CP720, but in the console 0x95 is =D8=B4 inst=
ead, and the tatweel is at 0xFF). On Windows 9x,=C2=A0ReadConsoleOutputW is=
not supported so the UCS-2 mappings of the console character tiles cannot =
be captured (error 0x00000078 ERROR_CALL_NOT_IMPLEMENTED). When that progr=
am runs on Arabic versions of Windows NT, the visual output is of the CP437=
character set if one of the bundled bitmap fonts is used ( i.imgur.com htt=
ps://i.imgur.com/RxjtxMH.png ), or the CP720 set if Lucida Console is used,=
with the Arabic letters either having glitchy font substitution (NT 4.0, N=
T 5.0/2000) or the .notdef glyph (NT 5.1/XP and up). In fact, it seems that=
the only Arabic bitmap fonts that occur in Windows NT are CP1256 fonts, wh=
ich are not used in terminals. So this appears to be one of those permanent=
Windows compatibility regressions that occured when Windows 9x ended, wher=
e the terminals can no longer render legacy Arabic text. Even if the user m=
anaged to use registry hacks to set the font to Courier New or Simplified A=
rabic Fixed, it would still use the CP720 mapping which is not compatible w=
ith the Windows 9x set. It appears that in the Windows 9x Arabic terminal =
character set, 244 characters ( =E2=80=87 =EF=BA=80=EF=BA=81=EF=BA=82=EF=BA=
=83=EF=BA=84=EF=BA=85=EF=BA=87=EF=BA=88=EF=BA=8A=EF=BA=8B=EF=BA=8D=EF=BA=8E=
=EF=BA=8F=EF=BA=91=EF=BA=93=E2=96=BA=E2=97=84=E2=86=95=EF=BA=95=C2=B6=C2=A7=
=EF=BA=97=EF=BA=99=E2=86=91=E2=86=93=E2=86=92=E2=86=90=EF=BA=9B=EF=B9=B0 =
=E2=96=B2=E2=96=BC !"#$%&'()*+,-./0123456789:;<=3D>?@ABC=
DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ =EF=BA=9D=EF=BA=
=9F=EF=BA=A1 =C3=A9=C3=A2 =EF=BA=A3 =C3=A0 =EF=BA=A5 =C3=A7=C3=AA=C3=AB=C3=
=A8=C3=AF=C3=AE =EF=BA=A7=EF=BA=A9=EF=BA=AB=EF=BA=AD=EF=BA=AF =C3=B4 =EF=BA=
=B3 =C3=BB=C3=B9 =EF=BA=B7=EF=BA=BB=C2=A3=EF=BA=BF=EF=BB=81=EF=BB=85=EF=BB=
=89=EF=BB=8A=EF=BB=8B=EF=BB=8C=EF=BB=8D=EF=BB=8E=EF=BB=8F=EF=BB=90=EF=BB=91=
=EF=BB=93=EF=BB=95=EF=BB=97=EF=BB=99=EF=BB=9B=C2=AB=C2=BB=EF=B9=B1=E2=96=92=
=EF=B9=B2=E2=94=82=E2=94=A4=EF=B9=B4=EF=B9=B6=EF=B9=B7=EF=B9=B8=D9=A0=D9=A1=
=D9=A2=D9=A3=EF=B9=B9=EF=B9=BA=E2=94=90=E2=94=94=E2=94=B4=E2=94=AC=E2=94=9C=
=E2=94=80=E2=94=BC=EF=B9=BB=EF=B9=BE=D9=A4=D9=A5=D9=A6=D9=A7=D9=A8=D9=A9=D8=
=8C=EF=B9=BF=EF=B1=9E=EF=B1=9F=EF=B1=A0=EF=B3=B2=EF=B1=A1=EF=B3=B3=EF=B1=A2=
=E2=94=98=E2=94=8C=D8=9B=D8=9F=C2 =C2=B5 =EF=BB=A9=EF=BB=AB=EF=BB=AC=EF=BB=
=AD=EF=BB=AF=EF=BB=B0=EF=BB=B1=EF=BB=B2=EF=BB=B3=EF=B3=B4=EF=B9=BC=EF=B9=BD=
=EF=BA=B1=EF=BA=B5=EF=BA=B9=EF=BA=BD=EF=B9=B3=C2=B0=C2=B7=E2=96=A0=D9=80 ) =
are already in Unicode, but 12 characters are not in Unicode: =E2=80=A2 6 o=
f them are pieces of lam-alef ligatures (0xDD, 0xDE, 0xF9, 0xFB, 0xFC, 0xFD=
) =E2=80=A2 2 of them are shadda with fathatan ligatures without or with ta=
tweel (0xD0, 0xD1) =E2=80=94 in some legacy Microsoft fonts, shadda with fa=
thatan is mapped to private use U+E818 =E2=80=A2 4 of them are disunificati=
ons of seen/sheen/sad/dad occuring either with or without tail =E2=80=94=C2=
=A0 =EF=B9=B3 (U+FE73 ARABIC TAIL FRAGMENT) was originally encoded in Unico=
de 3.2 for CP864 compatibility; in that codepage, the forms of=C2=A0seen/sh=
een/sad/dad attach to the tail fragment =E2=80=94 forms with included tail:=
=C2=A00x92, 0x95, 0x98, 0x8A =E2=80=94 forms without tail (attaching to tai=
l fragment like in CP864):=C2=A00xF3, 0xF4, 0xF5, 0xF6 If someone tried to=
make a Win32 console implementation and tried to implement both Windows 9x=
Arabic terminal character set compatibility and wide string API (ReadConso=
leOutputW) compatibility simultaneously, then they would run into the issue=
that there is currently no standardized mapping to handle that scenario. W=
hat should Windows 9x Arabic console compatible implementations do in that =
case?=0D
--2JUQTNLMGPDCIFGWGPHMWnhgwp
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset=UTF-8
<p><br></p><div class=3D"nh_extra"><blockquote><p style=3D"padding-bottom: =
12px;"><strong>Dnia 07 maja 2026 20:46</strong> <a href=3D"mailto:unicode@c=
orp.unicode.org" rel=3D"noopener noreferrer nofollow" target=3D"_blank"><sp=
an style=3D"margin-left: 4px;">Peter Constable via Unicode</span></a><span =
style=3D"margin-left: 4px;"> < [email protected] ></span> napi=
sa=C5=82(a):</p><div id=3D"gwp0ca10f88"><div id=3D"gwp0ca10f88h"><style></s=
tyle><div style=3D"overflow-wrap: break-word;" data-color-mode=3D"light" da=
ta-message-body=3D"true" class=3D"gwp0ca10f88b" lang=3D"EN-US"><div class=
=3D"gwp0ca10f88_WordSection1"><p class=3D"gwp0ca10f88_MsoNormal">> This =
is not of any interest, because the current Microsoft company is not even c=
lose ideologically to the past version of Microsoft that originally made th=
e Arabic terminals, in fact they're not even ideologically compatible with =
each other.</p><p class=3D"gwp0ca10f88_MsoNormal"><br></p><p class=3D"gwp0c=
a10f88_MsoNormal">Of course it might not be of interest to you. You don=E2=
=80=99t seem interested in others=E2=80=99 reasoning about this unless it a=
ligns to how you are thinking about it.</p></div></div></div></div></blockq=
uote></div><p>I didn't think that the evidence of user community in the pro=
posal was insufficient at first, but once that was explained to me, I accep=
ted that reasoning. In fact, I evaluate users' feedback all the time when I=
develop fonts, other software, and proposals, and I make changes to fit ne=
w evidence. When you brought up a hypothetical Microsoft voting result, I d=
idn't think you were making any argument at all, because back then you have=
n't explained why specifically Microsoft would vote that way or how that sp=
ecifically relates to the contents of the proposal, making it not a constru=
ctive claim at first.</p><p class=3D"gwp0ca10f88_MsoNormal"><br></p><blockq=
uote><p class=3D"gwp0ca10f88_MsoNormal">One point about current MS not bein=
g even close =E2=80=9Cideologically=E2=80=9D to past MS (very long past) th=
at _<em>I</em>_ think is of interest is that, for UTC to decide to add new =
legacy characters, one consideration is what the new characters will entail=
for product support and interoperability. There=E2=80=99s a cost / benefit=
analysis to be done. In the 1980s, there was a reason why legacy implement=
ations made sense. But today, the cost / benefit analysis doesn=E2=80=99t w=
eight at all in favour of encoding new legacy characters: Microsoft has bee=
n supporting Unicode in products for over 30 years, and in all that time th=
ere hasn=E2=80=99t been any identifiable customer need for encoding these a=
dditional legacy Arabic text elements as separate characters<span style=3D"=
font-family: Arial, sans-serif;">=E2=80=89</span>=E2=80=94<span style=3D"fo=
nt-family: Arial, sans-serif;">=E2=80=89</span>i.e., benefits are extremely=
low to zero. But the costs for the new characters would certainly not be z=
ero.</p><p class=3D"gwp0ca10f88_MsoNormal"><br></p><p class=3D"gwp0ca10f88_=
MsoNormal"><em>You</em> might think this is a false argument. Others might =
think otherwise.</p></blockquote><p class=3D"gwp0ca10f88_MsoNormal"><br></p=
><p>By false arguments I meant things like claiming that the characters can=
be represented with ZWJ/ZWNJ sequences or that the claims that Microsoft's=
documentation of legacy Arabic encodings is available at <a href=3D"https:=
//learn.microsoft.com/en=02us/typography/legacy/legacy_arabic_fonts" rel=3D=
"noopener noreferrer nofollow" target=3D"_blank">https://learn.microsoft.co=
m/en=02us/typography/legacy/legacy_arabic_fonts</a>, which I had already de=
bunked. Now that you explained the reasoning involving cost to benefit rati=
o, you've just expressed a legitimate argument, which is similar to what As=
mus Freytag had said.</p><p><br></p><div class=3D"nh_extra"><blockquote><di=
v id=3D"gwp0ca10f88"><div id=3D"gwp0ca10f88h"><div style=3D"overflow-wrap: =
break-word;" lang=3D"EN-US" class=3D"gwp0ca10f88b" data-message-body=3D"tru=
e" data-color-mode=3D"light"><div class=3D"gwp0ca10f88_WordSection1"><p cla=
ss=3D"gwp0ca10f88_MsoNormal"><br><br></p><p class=3D"gwp0ca10f88_MsoNormal"=
>Peter</p><p class=3D"gwp0ca10f88_MsoNormal"><br></p><div style=3D"border-r=
ight: none; border-bottom: none; border-left: none; border-image: initial; =
border-top: 1pt solid rgb(225, 225, 225); padding: 3pt 0in 0in;"><p class=
=3D"gwp0ca10f88_MsoNormal"><span style=3D"font-size: 11pt; font-family: Cal=
ibri, sans-serif;"><strong>From:</strong> [email protected] <piotruni=
[email protected]> <br><strong>Sent:</strong> Thursday, May 7, 2026 9:03 AM<b=
r><strong>To:</strong> Peter Constable via Unicode <[email protected]=
.org>; Peter Constable <[email protected]>; Philippe Verdy <verdyp=
@gmail.com><br><strong>Subject:</strong> RE: Odp: Pd: Missing legacy Ara=
bic encoding</span></p></div><p class=3D"gwp0ca10f88_MsoNormal"><br></p><p>=
This is not of any interest, because the current Microsoft company is not e=
ven close ideologically to the past version of Microsoft that originally ma=
de the Arabic terminals, in fact they're not even ideologically compatible =
with each other. There's no need to speculate on hypothetical voting scenar=
ios involving giant companies. What matters are logical arguments involving=
encoding policy, and the prevailing reason is that there is no sufficient =
evidence of user community that would need to interchange text from those p=
latforms into UCS-2 terminals. It is still necessary to debunk false argume=
nts in order to prevent future decisions from being made incorrectly, becau=
se even if they wouldn't affect the outcome of this proposal, they could st=
ill improperly influence the evaluation of future proposals.</p><p><br></p>=
<div><blockquote style=3D"margin-top: 5pt; margin-bottom: 5pt;"><p><span st=
yle=3D"font-family: Aptos, sans-serif;"><strong>Dnia 07 maja 2026 17:36</st=
rong></span><a href=3D"mailto:[email protected]" rel=3D"noopener nor=
eferrer" target=3D"_blank">Peter Constable via Unicode</a> < <a href=3D"=
mailto:[email protected]" rel=3D"noopener noreferrer" target=3D"_bla=
nk">[email protected]</a> > napisa=C5=82(a):</p><div id=3D"gwp0ca=
10f88_gwpb1352f8b"><div id=3D"gwp0ca10f88_gwpb1352f8bh"><div><div><p class=
=3D"gwp0ca10f88_gwpb1352f8bmsonormal"><span style=3D"font-size: 11pt;" lang=
=3D"EN-CA">Just in case it might be of interest, if a motion on this propos=
al were raised in a UTC meeting, I suspect Microsoft would vote against enc=
oding.</span></p><p class=3D"gwp0ca10f88_gwpb1352f8bmsonormal"><span style=
=3D"font-size: 11pt;" lang=3D"EN-CA"> </span></p><p class=3D"gwp0ca10f=
88_gwpb1352f8bmsonormal"><span style=3D"font-size: 11pt;" lang=3D"EN-CA">&n=
bsp;</span></p><p class=3D"gwp0ca10f88_gwpb1352f8bmsonormal"><span style=3D=
"font-size: 11pt;" lang=3D"EN-CA">Peter</span></p><p class=3D"gwp0ca10f88_g=
wpb1352f8bmsonormal"><span style=3D"font-size: 11pt;" lang=3D"EN-CA"> =
</span></p><div style=3D"border-right: none; border-bottom: none; border-le=
ft: none; border-top: 1pt solid rgb(225, 225, 225); padding: 3pt 0in 0in; b=
order-image: initial;"><p class=3D"gwp0ca10f88_gwpb1352f8bmsonormal"><span =
style=3D"font-size: 11pt; font-family: Calibri, sans-serif;"><strong>From:<=
/strong> Unicode <</span><a href=3D"mailto:[email protected].=
org" rel=3D"noopener noreferrer" target=3D"_blank"><span style=3D"font-size=
: 11pt; font-family: Calibri, sans-serif;">[email protected]=
</span></a><span style=3D"font-size: 11pt; font-family: Calibri, sans-serif=
;">> <strong>On Behalf Of </strong></span><a href=3D"mailto:piotrunio-20=
[email protected]" rel=3D"noopener noreferrer" target=3D"_blank"><span style=3D"font=
-size: 11pt; font-family: Calibri, sans-serif;">[email protected]</span>=
</a><span style=3D"font-size: 11pt; font-family: Calibri, sans-serif;"> via=
Unicode<br><strong>Sent:</strong> May 6, 2026 11:09 AM<br><strong>To:</str=
ong> Philippe Verdy via Unicode <</span><a href=3D"mailto:[email protected]=
nicode.org" rel=3D"noopener noreferrer" target=3D"_blank"><span style=3D"fo=
nt-size: 11pt; font-family: Calibri, sans-serif;">[email protected]<=
/span></a><span style=3D"font-size: 11pt; font-family: Calibri, sans-serif;=
">>; Philippe Verdy <</span><a href=3D"mailto:[email protected]" rel=
=3D"noopener noreferrer" target=3D"_blank"><span style=3D"font-size: 11pt; =
font-family: Calibri, sans-serif;">[email protected]</span></a><span style=
=3D"font-size: 11pt; font-family: Calibri, sans-serif;">><br><strong>Sub=
ject:</strong> Re: Odp: Pd: Missing legacy Arabic encoding</span></p></div>=
<p class=3D"gwp0ca10f88_gwpb1352f8bmsonormal"><br></p><p>The ReadConsoleOut=
putW function, by definition, captures the tiles into the lpBuffer, which i=
s a random access array of CHAR_INFO structure, whose horizontal and vertic=
al size is specified in dwBufferSize. Since lpBuffer is random access, this=
implies one CHAR_INFO structure per character tile. The API therefore fund=
amentally imposes a strict memory layout that cannot be violated. The fact =
that some Unix-like terminals such as Windows Terminal may support features=
outside the scope of the CHAR_INFO structure for compatibility with ANSI e=
scape codes or WSL programs does not invalidate the compatibility considera=
tions for legacy DOS/Win16/Win32 programs that require all character tiles =
to fit in the CHAR_INFO structure for random access, because 4 byte CHAR_IN=
FO structure of Win32 is intended to be a fully backwards compatible extens=
ion of the 2 byte VGA text mode tile structure of DOS/Win16.</p><p><br></p>=
<div><blockquote style=3D"margin-top: 5pt; margin-bottom: 5pt;"><p><span st=
yle=3D"font-family: Aptos, sans-serif;" lang=3D"EN-CA"><strong>Dnia 06 maja=
2026 19:58</strong></span><a href=3D"mailto:[email protected]" rel=
=3D"noopener noreferrer" target=3D"_blank">Philippe Verdy via Unicode</a> &=
lt; <a href=3D"mailto:[email protected]" rel=3D"noopener noreferrer"=
target=3D"_blank">[email protected]</a> > napisa=C5=82(a):</p><d=
iv id=3D"gwp0ca10f88_gwpb1352f8b_gwp6bd5644c"><div id=3D"gwp0ca10f88_gwpb13=
52f8b_gwp6bd5644ch"><div><div><p class=3D"gwp0ca10f88_gwpb1352f8bmsonormal"=
>Windows can use other ways to map 16-bit codes in its *legacy* Console buf=
fer (using old CHAR_INFO structure), it can perfectly internally use compat=
ibility characters, or PUAs of the BMP, and still present an API that expos=
es connforming sequences. You're talking about an old implementation that w=
as built even long before the Arabic script was extended (and newer s=
cripts using contextual joining behaviors, that have never been part of the=
BMP, shcih as Adlam, and other scripts like Mongolian that also may need s=
uch sequences with ZWJ/ZWNJ controls, or with other formatting characters l=
ike those specific to Mongolian like FVS1...FVS4 and MVS, or those common t=
o many Bhramic scripts, that the *legacy* Console did not support.<br>The *=
legacy* console was not built to support more than one plane (including man=
y CJK cgaracters). The newer console can!</p></div><p><br></p><div><div><p =
class=3D"gwp0ca10f88_gwpb1352f8bmsonormal">Le mer. 6 mai 2026 =C3=A0&n=
bsp;18:37, <a href=3D"mailto:[email protected]" rel=3D"noopener noreferr=
er" target=3D"_blank">[email protected]</a> <<a href=3D"mailto:piotru=
[email protected]" rel=3D"noopener noreferrer" target=3D"_blank">piotrunio-200=
[email protected]</a>> a =C3=A9crit :</p></div><blockquote style=3D"border-to=
p: none; border-right: none; border-bottom: none; border-left: 1pt solid rg=
b(204, 204, 204); padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt; bord=
er-image: initial;"><p>Have you read the L2/26-077 proposal? Using ZWJ or Z=
WNJ would not work for the compatibility purposes at all as already explain=
ed in the proposal. This is because ZWJ or ZWNJ would take the space of one=
character tile in the CHAR_INFO structure. Suppose that you're trying to m=
ap 0xD0 from FP164 to a sequence of U+FE7C U+200D U+064B (<span style=3D"fo=
nt-family: "Times New Roman", serif;" lang=3D"AR-SA" dir=3D"RTL">=
=EF=B9=BC=E2=80=8D=D9=8B</span>). The legacy application fills the 80=C3=97=
25 screen with all 0xD0 tiles. You subsequently try to capture the tiles wi=
th a Win32 program by using ReadConsoleOutputA into an 80=C3=9725 buffer of=
2000 tiles. This succeeds and captures 0xD0 into all the tiles. You then t=
ry to capture the tiles using ReadConsoleOutputW into an 80=C3=9725 buffer.=
Each sequence U+FE7C U+200D U+064B would take a sequence of three CHAR_INF=
O structures to store, meaning 6000 such structures for the whole screen. B=
ut the 80=C3=9725 buffer has only room for 2000 instances of the structure =
(one per character tile). Since CHAR_INFO stores 16-bit character code, by =
that same logic the compatibility characters would have to be in BMP for it=
to work. In Windows 95 Vietnamese and Windows 95 Thai, there are instances=
where one character tile takes multiple CHAR_INFO structures, causing visu=
al width to be smaller than logical width, and when that happens, the remai=
ning space at the end of the line is left blank, allowing for CP1258/CP874 =
combining characters in those systems to map 1:1 to their Unicode equivalen=
ts. Windows 3.1/95/98/ME Arabic don't work that way and don't use combining=
characters or ZWJ sequences, so visual width is always equivalent to logic=
al width, each character tile maps 1:1 to a CHAR_INFO structure and all cha=
racters may fill the entire line, which would be impossible if some of thos=
e characters were mapped to composition sequences or non-BMP characters. Si=
nce there is currently no sufficient evidence of user community that would =
need to use those mappings, there are no plans for those characters to be a=
dded to Unicode, and therefore the only solution for the ReadConsoleOutputW=
to work properly in this case is to use agreed upon private use mappings f=
or those compatibility characters.</p><p><br></p><div><blockquote style=3D"=
margin-top: 5pt; margin-bottom: 5pt;"><p><span style=3D"font-family: Aptos,=
sans-serif;" lang=3D"EN-CA"><strong>Dnia 06 maja 2026 18:06</strong></span=
><a href=3D"mailto:[email protected]" rel=3D"noopener noreferrer" ta=
rget=3D"_blank">Philippe Verdy via Unicode</a> < <a href=3D"mailto:unico=
[email protected]" rel=3D"noopener noreferrer" target=3D"_blank">unicode@=
corp.unicode.org</a> > napisa=C5=82(a):</p><div id=3D"gwp0ca10f88_gwpb13=
52f8b_gwp6bd5644c_m_-2650882641749569987gwpb67625a5"><div id=3D"gwp0ca10f88=
_gwpb1352f8b_gwp6bd5644c_m_-2650882641749569987gwpb67625a5h"><div><div><p>Y=
ou actually don't need any new compatibility characters for Arabic contextu=
al forms, or for other contextual forms in other joining scripts (like Adla=
m, or even Mongolian whichbis a LTR script).</p><div><p class=3D"gwp0ca10f8=
8_gwpb1352f8bmsonormal"><br></p></div><div><p class=3D"gwp0ca10f88_gwpb1352=
f8bmsonormal">You just have to prepend or append a ZWJ or ZWNJ formatting c=
ontrol to the unified letter if you want to override its default contextual=
presentation form.</p></div><div><p class=3D"gwp0ca10f88_gwpb1352f8bmsonor=
mal"><br></p></div></div><p><br></p><div><div><p class=3D"gwp0ca10f88_gwpb1=
352f8bmsonormal">Le mar. 5 mai 2026, 00:48, Asmus Freytag via Unicode <<=
a href=3D"mailto:[email protected]" rel=3D"noopener noreferrer" targ=
et=3D"_blank">[email protected]</a>> a =C3=A9crit :</p></div=
><blockquote style=3D"border-top: none; border-right: none; border-bottom: =
none; border-left: 1pt solid rgb(204, 204, 204); padding: 0in 0in 0in 6pt; =
margin: 5pt 0in 5pt 4.8pt; border-image: initial;"><div><p>The issue at han=
d is the distinction between a theoretical gap and real-life problem.</p><p=
><br></p><p>You have demonstrated that there are specifications that, if ch=
ained in the right way, can lead to ambiguities or gaps in interchange.</p>=
<p><br></p><p>What we don't have is an actual use case with real-life conse=
quences for a set of existing users, not hypothetical ones.</p><p><br></p><=
p>When it comes to encoding decisions based on existing documents, there is=
a strong presumption that once sufficiently many documents exist that cont=
ain a character, that this character will be needed in digitizing these doc=
uments, whether immediately, or eventually (e.g. in the case of future scho=
larly studies). Also, the texts themselves exist, barring accidents, in per=
manence. Therefore, it is justified to consider irrevocably allocating a ch=
aracter that will map to this source in perpetuity, even though each encode=
d character carries a small cost for implementers.<br><br>However, when it =
comes to legacy characters, there's an additional cost that is imposed, and=
that is based on the fact that characters that are encoded solely for comp=
atibility will usually violate one or more of the other encoding principles=
, something that incrementally complicates the standard. Even for people wh=
o never intend to use that character.<br><br>Therefore, the SEW is on solid=
ground when it demands not only a hypothetical scenario, but evidence of a=
ctual impact on actual users. Not only whether some application could invok=
e an API, but whether such applications exist and are used today to access =
documents encoded using the legacy characters in a way that is compromised =
irreparably by not having an encoding for them.</p><p><br></p><p>A./</p><p>=
<br></p><p><br></p><p>On 5/4/2026 10:24 AM, <a href=3D"mailto:piotrunio-200=
[email protected]" rel=3D"noopener noreferrer" target=3D"_blank">[email protected]=
</a> via Unicode wrote:</p><blockquote style=3D"margin-top: 5pt; margin-bot=
tom: 5pt;"><p>In UTC 187 Minutes, "<span style=3D"font-size: 13.5pt; font-f=
amily: "Times New Roman", serif; color: black;" lang=3D"EN-CA">As=
mus Freytag noted that the fact that lists of things existed in the past do=
es not make these things plain text. Ned Holbrook pointed out that the purp=
orted issue occurs in a closed system, not in public interchange.</span>". =
However, the arguments in the proposal do not merely hinge on the encodings=
being lists of characters, but specifically points out methods to intercha=
nge text, including an example of copying terminal output and pasting to No=
tepad, where the copying invokes the mapping of the current terminal codepa=
ge to UCS-2 (as is CHAR_INFO compatible) and the pasting writes it into pla=
in text. Win32 is also not a closed system, as Win32 can capture the tiles =
of the output of Windows 3.1 Arabic DOS/Win16 programs and Windows 95/98/ME=
Arabic DOS/Win16/Win32 programs, but Win32 can also interact with public t=
ext interchange systems by reading and writing to files and network. I'm no=
t saying that Unicode absolutely must include those characters, but those k=
inds of misleading claims are causing users to misunderstand what the propo=
sal is about, and I don't want Unicode to be relying on uninformed decision=
s to evaluate proposals.</p><p><br></p><div><blockquote style=3D"margin-top=
: 5pt; margin-bottom: 5pt;"><p><span style=3D"font-family: Aptos, sans-seri=
f;" lang=3D"EN-CA"><strong>Dnia 18 kwietnia 2026 13:36</strong></span><a hr=
ef=3D"mailto:[email protected]" rel=3D"noopener noreferrer" target=
=3D"_blank">[email protected] via Unicode< [email protected] &=
gt;</a> napisa=C5=82(a):</p><div id=3D"gwp0ca10f88_gwpb1352f8b_gwp6bd5644c_=
m_-2650882641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884a"><div id=
=3D"gwp0ca10f88_gwpb1352f8b_gwp6bd5644c_m_-2650882641749569987gwpb67625a5_m=
_5371084887079304582gwpbf2d884ah"><div><p>The SEW subsequently explained th=
at the actual reason is due to insufficient evidence of user community that=
would need to use the resulting mapping. Despite Win32 being a highly popu=
lar platform with plenty of backwards compatibility and native UCS-2 termin=
al support, the specific use cases of installing codepages into Windows NT =
and using terminal tiles from Windows 3.1/95/98/ME are not sufficiently doc=
umented, making it difficult for any user communities to form around it. So=
it seems like the idea of standardizing legacy Arabic terminal BMP mapping=
s is a dead end for now.</p><p><br></p><div><blockquote style=3D"margin-top=
: 5pt; margin-bottom: 5pt;"><p><span style=3D"font-family: Aptos, sans-seri=
f;" lang=3D"EN-CA"><strong>Dnia 17 kwietnia 2026 22:59</strong></span><a hr=
ef=3D"mailto:[email protected]" rel=3D"noopener noreferrer" target=
=3D"_blank">[email protected] via Unicode< [email protected] &=
gt;</a> napisa=C5=82(a):</p><div id=3D"gwp0ca10f88_gwpb1352f8b_gwp6bd5644c_=
m_-2650882641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884a_gwp2281a=
7f8"><div id=3D"gwp0ca10f88_gwpb1352f8b_gwp6bd5644c_m_-2650882641749569987g=
wpb67625a5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8h"><div><p>The Recom=
mendations in L2/26-100 claim that Microsoft's documentation of legacy Arab=
ic encodings is available at <a href=3D"https://learn.microsoft.com/en-us/t=
ypography/legacy/legacy_arabic_fonts" =3D"" rel=3D"noopener noreferrer" tar=
get=3D"_blank">https://learn.microsoft.com/en-us/typography/legacy/legacy_a=
rabic_fonts</a>. However, that article only demonstrates two encodings of T=
rueType fonts, which are used in Windows 3.1 but are completely different f=
rom the eight terminal encodings. Unlike the TrueType encodings which repre=
sent internal shaping mappings and are not used for text interchange, the t=
erminal encodings have been demonstrated to be directly used in text interc=
hange through int 10h and ReadConsoleOutputA/WriteConsoleOutputA as already=
demonstrated in L2/26-077. The Recommendations also claim that the proposa=
l does not demonstrate any need for interchange or encoding, but the propos=
al actually demonstrated such a need due to the logical extension of the Wi=
n32 terminal API to the functions ReadConsoleOutputW/WriteConsoleOutputW, w=
hich are in Windows NT and may be used on the output of previously ran prog=
rams (including those that used the legacy Arabic terminal encodings), whic=
h given the CHAR_INFO structure, therefore implies a need for all the tiles=
to map to BMP for interchange. I'm not objecting to the SEW's conclusion o=
f "Users are expected to use PUA.", which can indeed be used to provide a m=
apping even if not standardized, but the reasoning given was flawed.</p><p>=
<br></p><div><blockquote style=3D"margin-top: 5pt; margin-bottom: 5pt;"><p>=
<span style=3D"font-family: Aptos, sans-serif;" lang=3D"EN-CA"><strong>Dnia=
09 stycznia 2026 17:25</strong></span><a href=3D"mailto:piotrunio-2004@wp.=
pl" rel=3D"noopener noreferrer" target=3D"_blank">[email protected]< =
[email protected] ></a> napisa=C5=82(a):</p><div id=3D"gwp0ca10f88_gw=
pb1352f8b_gwp6bd5644c_m_-2650882641749569987gwpb67625a5_m_53710848870793045=
82gwpbf2d884a_gwp2281a7f8_gwpa05276c7"><div id=3D"gwp0ca10f88_gwpb1352f8b_g=
wp6bd5644c_m_-2650882641749569987gwpb67625a5_m_5371084887079304582gwpbf2d88=
4a_gwp2281a7f8_gwpa05276c7h"><div><div id=3D"gwp0ca10f88_gwpb1352f8b_gwp6bd=
5644c_m_-2650882641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884a_gw=
p2281a7f8_gwpa05276c7_gwpa8b5f718"><div id=3D"gwp0ca10f88_gwpb1352f8b_gwp6b=
d5644c_m_-2650882641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884a_g=
wp2281a7f8_gwpa05276c7_gwpa8b5f718h"><div><div id=3D"gwp0ca10f88_gwpb1352f8=
b_gwp6bd5644c_m_-2650882641749569987gwpb67625a5_m_5371084887079304582gwpbf2=
d884a_gwp2281a7f8_gwpa05276c7_gwpa8b5f718_gwpa8b5f718"><div id=3D"gwp0ca10f=
88_gwpb1352f8b_gwp6bd5644c_m_-2650882641749569987gwpb67625a5_m_537108488707=
9304582gwpbf2d884a_gwp2281a7f8_gwpa05276c7_gwpa8b5f718_gwpa8b5f718h"><div><=
div id=3D"gwp0ca10f88_gwpb1352f8b_gwp6bd5644c_m_-2650882641749569987gwpb676=
25a5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8_gwpa05276c7_gwpa8b5f718_g=
wpa8b5f718_gwpa8b5f718"><div id=3D"gwp0ca10f88_gwpb1352f8b_gwp6bd5644c_m_-2=
650882641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8_=
gwpa05276c7_gwpa8b5f718_gwpa8b5f718_gwpa8b5f718h"><div><p>The following Win=
32 C code will output 256 characters in system console codepage into the ch=
aracter grid, capture those character tiles in UCS-2 if possible, and then =
output the current console codepage number.</p><p><br></p><p style=3D"margi=
n-bottom: 12pt;">#include <windows.h><br>#include <stdio.h><br>=
int main(){<br>HANDLE hConsole=3DGetStdHandle(STD_OUTPUT_HANDLE);<br>CHAR_I=
NFO screen[256];<br>COORD size=3D{16,16,};<br>COORD pos=3D{0,0,};<br>SMALL_=
RECT rect=3D{0,0,15,15,};<br>for(int i=3D0;i<256;i++){<br>screen[i].Attr=
ibutes=3D0xF0;<br>screen[i].Char.AsciiChar=3Di;<br>}<br>WriteConsoleOutputA=
(hConsole,screen,size,pos,&rect);<br>CHAR_INFO screenu[256];<br>if(Read=
ConsoleOutputW(hConsole,screenu,size,pos,&rect)){<br>for(int i=3D0;i<=
;256;i++) printf("%04X ",screenu[i].Char.UnicodeChar);<br>}<br>else{<br>pri=
ntf("error %08X\n",GetLastError());<br>}<br>printf("codepage %u",GetConsole=
OutputCP());<br>}</p><p>In most cases, whenever a legacy Win32 codepage is =
used, the application can run on Windows NT to capture the UCS-2 mapping of=
those character cells to the BMP (although for CJK codepages a more comple=
x setup would be necessary due to thousands of fullwidth characters with 2-=
byte sequences).</p><p><br></p><p>However, in Arabic versions of Windows 9x=
(95/98/ME) the resulting character set has many presentation forms that ar=
e not in Unicode. This is the result when running on Windows ME: <a hr=
ef=3D"https://i.imgur.com/QFm3SkI.png" =3D"" rel=3D"noopener noreferrer" ta=
rget=3D"_blank">https://i.imgur.com/QFm3SkI.png</a> in 10=C3=9720 font=
, <a href=3D"https://i.imgur.com/KUbLQ0A.png" =3D"" rel=3D"noopener norefer=
rer" target=3D"_blank">https://i.imgur.com/KUbLQ0A.png</a> in 10=C3=97=
18 font (same result also appears in Windows 95/98). 5=C3=9712, 7=C3=9712, =
8=C3=9712, 10=C3=9718, 10=C3=9720, and 12=C3=9716 bitmap fonts have been at=
tested with that character set (VGAOEM.FON, 8514OEM.FON, DOSAPP.FON). The 1=
0=C3=9720 font has slightly different mapping than the other sizes: 0x93 is=
=C3=B6 instead of =C3=B4, and 0x97 is missing (causing the following chara=
cters on the same line to be drawn at the wrong position). It also claims t=
o be using codepage 720, but many characters differ from their CP720 mappin=
gs, including the bundled CP_720.NLS mappings (for example, <span styl=
e=3D"font-family: "Times New Roman", serif;" lang=3D"AR-SA" dir=
=3D"RTL">=D9=80</span>(U+0640 ARABIC TATWEEL) is 0x95 in CP720, but in the =
console 0x95 is <span style=3D"font-family: "Times New Roman", se=
rif;" lang=3D"AR-SA" dir=3D"RTL">=D8=B4</span>instead, and the tatweel is a=
t 0xFF). On Windows 9x, ReadConsoleOutputW is not supported so the UCS=
-2 mappings of the console character tiles cannot be captured (error 0x0000=
0078 ERROR_CALL_NOT_IMPLEMENTED).</p></div></div></div><p><br></p><p>When t=
hat program runs on Arabic versions of Windows NT, the visual output is of =
the CP437 character set if one of the bundled bitmap fonts is used (<a href=
=3D"https://i.imgur.com/RxjtxMH.png" =3D"" rel=3D"noopener noreferrer" targ=
et=3D"_blank">https://i.imgur.com/RxjtxMH.png</a>), or the CP720 set if Luc=
ida Console is used, with the Arabic letters either having glitchy font sub=
stitution (NT 4.0, NT 5.0/2000) or the .notdef glyph (NT 5.1/XP and up). In=
fact, it seems that the only Arabic bitmap fonts that occur in Windows NT =
are CP1256 fonts, which are not used in terminals. So this appears to be on=
e of those permanent Windows compatibility regressions that occured when Wi=
ndows 9x ended, where the terminals can no longer render legacy Arabic text=
. Even if the user managed to use registry hacks to set the font to Courier=
New or Simplified Arabic Fixed, it would still use the CP720 mapping which=
is not compatible with the Windows 9x set.</p></div></div><p><br></p></div=
></div></div></div><div id=3D"gwp0ca10f88_gwpb1352f8b_gwp6bd5644c_m_-265088=
2641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8_gwpa0=
5276c7_gwpa8b5f718"><div id=3D"gwp0ca10f88_gwpb1352f8b_gwp6bd5644c_m_-26508=
82641749569987gwpb67625a5_m_5371084887079304582gwpbf2d884a_gwp2281a7f8_gwpa=
05276c7_gwpa8b5f718h"><div><p>It appears that in the Windows 9x Arabic term=
inal character set, 244 characters (<span style=3D"font-family: Arial, sans=
-serif;" lang=3D"EN-CA">=E2=80=87</span><span style=3D"font-family: "T=
imes New Roman", serif;" lang=3D"AR-SA" dir=3D"RTL">=EF=BA=80=EF=BA=81=
=EF=BA=82=EF=BA=83=EF=BA=84=EF=BA=85=EF=BA=87=EF=BA=88=EF=BA=8A=EF=BA=8B=EF=
=BA=8D=EF=BA=8E=EF=BA=8F=EF=BA=91=EF=BA=93=E2=96=BA=E2=97=84=E2=86=95=EF=BA=
=95=C2=B6=C2=A7=EF=BA=97=EF=BA=99=E2=86=91=E2=86=93=E2=86=92=E2=86=90=EF=BA=
=9B=EF=B9=B0</span><span style=3D"font-family: Arial, sans-serif;" lang=3D"=
EN-CA">=E2=96=B2=E2=96=BC</span> !"#$%&'()*+,-./0123456789:;<=3D>=
?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~<span style=
=3D"font-family: "Times New Roman", serif;" lang=3D"AR-SA" dir=3D=
"RTL">=EF=BA=9D=EF=BA=9F=EF=BA=A1</span>=C3=A9=C3=A2<span style=3D"font-fam=
ily: "Times New Roman", serif;" lang=3D"AR-SA" dir=3D"RTL">=EF=BA=
=A3</span>=C3=A0<span style=3D"font-family: "Times New Roman", se=
rif;" lang=3D"AR-SA" dir=3D"RTL">=EF=BA=A5</span>=C3=A7=C3=AA=C3=AB=C3=A8=
=C3=AF=C3=AE<span style=3D"font-family: "Times New Roman", serif;=
" lang=3D"AR-SA" dir=3D"RTL">=EF=BA=A7=EF=BA=A9=EF=BA=AB=EF=BA=AD=EF=BA=AF<=
/span>=C3=B4<span style=3D"font-family: "Times New Roman", serif;=
" lang=3D"AR-SA" dir=3D"RTL">=EF=BA=B3</span>=C3=BB=C3=B9<span style=3D"fon=
t-family: "Times New Roman", serif;" lang=3D"AR-SA" dir=3D"RTL">=
=EF=BA=B7=EF=BA=BB=C2=A3=EF=BA=BF=EF=BB=81=EF=BB=85=EF=BB=89=EF=BB=8A=EF=BB=
=8B=EF=BB=8C=EF=BB=8D=EF=BB=8E=EF=BB=8F=EF=BB=90=EF=BB=91=EF=BB=93=EF=BB=95=
=EF=BB=97=EF=BB=99=EF=BB=9B=C2=AB=C2=BB=EF=B9=B1=E2=96=92=EF=B9=B2=E2=94=82=
=E2=94=A4=EF=B9=B4=EF=B9=B6=EF=B9=B7=EF=B9=B8=D9=A0=D9=A1=D9=A2=D9=A3=EF=B9=
=B9=EF=B9=BA=E2=94=90=E2=94=94=E2=94=B4=E2=94=AC=E2=94=9C=E2=94=80=E2=94=BC=
=EF=B9=BB=EF=B9=BE=D9=A4=D9=A5=D9=A6=D9=A7=D9=A8=D9=A9=D8=8C=EF=B9=BF=EF=B1=
=9E=EF=B1=9F=EF=B1=A0=EF=B3=B2=EF=B1=A1=EF=B3=B3=EF=B1=A2=E2=94=98=E2=94=8C=
=D8=9B=D8=9F=C2=A4=EF=BB=9D=EF=BB=9F=EF=BB=A1=EF=BB=A3=EF=BB=A5=EF=BB=A7</s=
pan>=C2=B5<span style=3D"font-family: "Times New Roman", serif;" =
lang=3D"AR-SA" dir=3D"RTL">=EF=BB=A9=EF=BB=AB=EF=BB=AC=EF=BB=AD=EF=BB=AF=EF=
=BB=B0=EF=BB=B1=EF=BB=B2=EF=BB=B3=EF=B3=B4=EF=B9=BC=EF=B9=BD=EF=BA=B1=EF=BA=
=B5=EF=BA=B9=EF=BA=BD=EF=B9=B3=C2=B0=C2=B7=E2=96=A0=D9=80</span>) are alrea=
dy in Unicode, but 12 characters are not in Unicode:</p><p>=E2=80=A2 6 of t=
hem are pieces of lam-alef ligatures (0xDD, 0xDE, 0xF9, 0xFB, 0xFC, 0xFD)</=
p><p>=E2=80=A2 2 of them are shadda with fathatan ligatures without or with=
tatweel (0xD0, 0xD1)</p><p>=E2=80=94 in some legacy Microsoft fonts, shadd=
a with fathatan is mapped to private use U+E818</p><p>=E2=80=A2 4 of them a=
re disunifications of seen/sheen/sad/dad occuring either with or without ta=
il</p><p>=E2=80=94 <span style=3D"font-family: "Times New Roman&q=
uot;, serif;" lang=3D"AR-SA" dir=3D"RTL">=EF=B9=B3</span>(U+FE73 ARABIC TAI=
L FRAGMENT) was originally encoded in Unicode 3.2 for CP864 compatibility; =
in that codepage, the forms of seen/sheen/sad/dad attach to the tail f=
ragment</p><p>=E2=80=94 forms with included tail: 0x92, 0x95, 0x98, 0x=
8A</p><p>=E2=80=94 forms without tail (attaching to tail fragment like in C=
P864): 0xF3, 0xF4, 0xF5, 0xF6</p></div><p><br></p></div><p>If someone =
tried to make a Win32 console implementation and tried to implement both Wi=
ndows 9x Arabic terminal character set compatibility and wide string API (R=
eadConsoleOutputW) compatibility simultaneously, then they would run into t=
he issue that there is currently no standardized mapping to handle that sce=
nario. What should Windows 9x Arabic console compatible implementations do =
in that case?</p></div><p><br></p></div></div></div></blockquote></div><p><=
br></p></div></div></div></blockquote></div><p><br></p></div></div></div></=
blockquote></div><p><br></p></blockquote><p><br></p></div></blockquote></di=
v></div></div></div></blockquote></div><p><br></p></blockquote></div></div>=
</div></div></blockquote></div><p><br></p></div></div></div></div></blockqu=
ote></div><p><br></p></div></div></div></div></blockquote></div><p><br></p>
--2JUQTNLMGPDCIFGWGPHMWnhgwp--