Fw: Konsole displays some characters wrong
Norman Diamond <[email protected]>
| Newsgroups | gmane.linux.suse.m17n |
|---|---|
| Message-ID | <[email protected]> |
OK, it could be 20 years too late to prevent ambiguous-width Romaji, katakana, and space characters from entering Unicode. However, the Unicode consortium continues to add characters. It isn't too late to add some necessary characters. A half-width downarrow can be added with the same width as half-width space, Romaji, and katakana. A full-width downarrow can be added with same same width as full-width space, Romaji, and katakana. Ambiguous-width space, Romaji, and katakana can be added to help unify the performance of applications when the text doesn't happen to include arrows. Sincerely, Norman Diamond Takashi Iwai wrote: > Norman Diamond wrote: >> >> I think the Unicode consortium should be taught that full-width and > half-width downarrow can be distinguished just as full-width Romaji can be > distinguished from half-width Romaji, without even needing CJK. Even a > full-width plain blank space is distinguished from a half-width plain blank > space. > > It's 20 years too late :) > >> However, anyone who edits these source files will likely have CJK fonts. > The comments are mostly in Japanese. Strings have mostly been moved to data > files, but changes to the code are generally accompanied by adding or editing > strings that are also Japanese. > > There is globaladvance attribute in the fontconfig and we used to set > it false for CJK for some time. The attribute itself is marked as > deprecated nowadays, but this might still work. You need to give it a > try. > > > Takashi > >> Yours sincerely, >> Norman Diamond >> >> >> >> Fuminobu TAKEYAMA replied to me: >> >> >> I do not agree with your statement that downarrow should be > displayed as >> > half-width. It is still >> >> the same character that it used to be in ShiftJIS and EUC > encodings, and >> > the adoption of Unicode >> >> and UTF-8 does not change the width of the glyph. >> > >> > The ambiguous width problem have been discussed for a long time. >> > There is no way to distinguish CJK down arrow and non-CJK down arrow. >> > >> > >> >> If I the character becomes half-width when I edit source files in > Konsole, >> > but it's still >> >> full-width when other people edit the same source files in a > Gnome terminal >> > or even in Windows, >> >> it will result in people scrambling source files. >> > >> > If they edit GNOME terminal without CJK fonts, arrows are half-width. >> > Even if we resolve this Konsole bug, your problem will not be > resolved. >> > >> > You still need to keep in mind that down arrow (and some other > characters) have >> > ambiguous width. >> > >> > >> > >> >> I do not know who decided that downarrow should have an ambiguous > width. >> >> If the Unicode consortium decided that, they're causing > problems. >> > >> > They did: >> > http://www.unicode.org/reports/tr11/tr11-28.html#Ambiguous >> > http://ja.wikipedia.org/wiki/東アジアの文字幅 >> > >> > >> > Anyway, let's fix this Konsole's bug. >> > >> > It is quite difficult to propose a new Unicode specification that can > be >> > acceptable for both CJK and non-CJK people. >> > >> > Fuminobu TAKEYAMA >> > >> > On 2015/06/13 10:35, Norman Diamond wrote: >> >> Since the font selection dialog's preview uses Qt widgets > instead of >> > whatever Konsole uses, we >> >> know that the full-width downarrow character is displayed as > full-width by >> > these Linux tools. >> >> Furthermore in Suse 11.0 Konsole itself also displayed the > full-width >> > downarrow character as >> >> full-width. So I think this bug is in the newer version of > Konsole itself, >> > not in the rest of >> >> Linux. I can't say if the preview window should be removed > because it >> > doesn't really provide a >> >> preview, but it helped narrow down this bug. >> >> >> >> I do not agree with your statement that downarrow should be > displayed as >> > half-width. It is still >> >> the same character that it used to be in ShiftJIS and EUC > encodings, and >> > the adoption of Unicode >> >> and UTF-8 does not change the width of the glyph. >> >> >> >> If I the character becomes half-width when I edit source files in > Konsole, >> > but it's still >> >> full-width when other people edit the same source files in a > Gnome terminal >> > or even in Windows, >> >> it will result in people scrambling source files. >> >> >> >> Full-width ａ and half-width a do not have ambiguous widths. > Full-width ア >> > and half-width ｱ do not >> >> have ambiguous widths. I do not know who decided that downarrow > should >> > have an ambiguous width. >> >> If the Unicode consortium decided that, they're causing > problems. >> >> >> >> Yours sincerely, Norman Diamond >> >> >> >> >> >> ----- Forwarded Message ----- >> >>> From: Fuminobu TAKEYAMA <[email protected]> To: >> > [email protected] Date: 2015/6/13, Sat >> >>> 01:30 Subject: Re: [opensuse-m17n] Konsole displays some > characters >> > wrong >> >>> >> >>>>> I still think there's a bug. >> >>> >> >>> Yes, this is clearly a Japanese specific bug. Could you file > a bug >> > report for openSUSE and >> >>> KDE? >> >>> >> >>> I saw a patch to fix the same problem with emacs about 5 > years ago. So >> > I think this problem can >> >>> be resolved by some hacks. >> >>> >> >>> >> >>>>> Konsole knows that the glyph for 字 is full-width, so > Konsole >> > should know that the glyph for >> >>>>> downarrow is full-width. >> >>> >> >>> Down arrow is known as ambiguous-width character. Monospaced > font >> > should provide half-width >> >>> down arrow but some Japanese fonts such as IPA Gothic and MS > Gothic do >> > full-width down arrow >> >>> from a kind of historical reason. >> >>> >> >>> >> >>>>> The version of Konsole that was included in Suse 11.0 > did know >> > that downarrow is >> >>>>> full-width. I never had to look at what the default > font was >> > because it just worked. >> >>> >> >>> Konsole's character layouting was changed recently as far > as I know >> > so that some >> >>> (flexible-width) bold fonts can be placed correctly. >> >>> >> >>> I guess it causes this problem. >> >>> >> >>> >> >>>> In the 固定幅フォントを選択 - Konsole panel, the bottom part is a > text box. I >> > can type or paste text in >> >>>> the text box. > I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they >> > display correctly in that >> >>>> text box. Konsole corrupts the display in its main > window, and the >> > preview window does not >> >>>> preview the >> >>> corruption. >> >>> >> >>> Unfortunately, the preview is for font selection dialog and > Qt widgets. >> > It is not preview of >> >>> Konsole. >> >>> >> >>> Maybe, the preview should be removed. >> >>> >> >>> >> >>> Fuminobu TAKEYAMA >> >>> >> >>> On 2015/06/12 7:05, Norman Diamond wrote: >> >>>> There is a worse bug. >> >>>> >> >>>> In the 固定幅フォントを選択 - Konsole panel, the bottom part is a > text box. I >> > can type or paste text in >> >>>> the text box. > I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they >> > display correctly in that >> >>>> text box. Konsole corrupts the display in its main > window, and the >> > preview window does not >> >>>> preview the corruption. >> >>>> >> >>>> Yours sincerely,Norman Diamond >> >>>> >> >>>> >> >>>> Norman Diamond wrote: >> >>>> >> >>>>> I click 設定 > 現在のプロファイルを編集(E)... > 外観 In フォント >> プレビュー >> > it says Monospace Oh now I see, next >> >>>>> to フォントサイズ where it lets me change the size, > there's a >> > clickable button to select a font. >> >>>>> In that panel it says モノスペース instead of Monospace. >> >>>>> >> >>>>> Thank you for the workaround. I will experiment with > some >> > fonts. I would like to >> >>>>> recommend that the preview box should include some > full-width >> > characters so that users can >> >>>>> judge more easily. If a font doesn't include > glyphs for >> > full-width characters then users >> >>>>> will see immediately that they shouldn't > experiment with >> > such a font. >> >>>>> >> >>>>> I still think there's a bug. >> >>>>> >> >>>>> Konsole knows that the glyph for 字 is full-width, so > Konsole >> > should know that the glyph for >> >>>>> downarrow is full-width. >> >>>>> >> >>>>> The version of Konsole that was included in Suse 11.0 > did know >> > that downarrow is >> >>>>> full-width. I never had to look at what the default > font was >> > because it just worked. >> >>>>> >> >>>>> Yours sincerely, Norman Diamond >> >>>>> >> >>>>> >> >>>>> Fuminobu TAKEYAMA wrote: >> >>>>> >> >>>>>> Ok, Monospace is an alias of IPAGothic on your > environment. >> >>>>>> >> >>>>>> The font, IPAGothic provides double-width > down-allow glyph >> > although Konsole assumes that >> >>>>>> the glyph should have the same width as glyphs > such as >> > 'a' and '0'. >> >>>>>> >> >>>>>> A solution is to use another font such as DejaVu > Sans Mono. >> > You can change fonts from: >> >>>>>> Settings > Edit Current Profile... > > Appearance >> >>>>>> >> >>>>>> If you do not need Japanese fonts, please > uninstall >> > ipa-gothic-fonts. So another font is >> >>>>>> to be selected for Monospace. >> >>>>>> >> >>>>>> >> >>>>>> Fuminobu TAKEYAMA >> >>>>>> >> >>>>>> >> >>>>>> On 2015/06/11 6:42, Norman Diamond wrote: >> >>>>>>> Yes the font is Monospace. I did not change > it from >> > the default. A >> >>>>>> Konsole settings panel displayed the name > Monospace and >> > allowed me to >> >>>>> change the >> >>>>>> font size but didn't allow me to select a > different >> > font. >> >>>>>>> >> >>>>>>> For this e-mail I use Yahoo's web > interface from >> > Internet Explorer >> >>>>> >> >>>>>> under Windows, so I translated the output of > fc-match -v >> > and fc-match -v Monospace from >> >>>>>> UTF-8 to Microsoft's Unicode encoding. > Yahoo's web >> > interface will scramble the text that >> >>>>>> I will paste below, so I ALSO attach >> >>>>> the >> >>>>>> outputs as files. >> >>>>>>> >> >>>>>>> >> >>>>>>> Output of fc-match -v >> >>>>>>> >> >>>>>>> Pattern has 41 elts (size 48) family: > "IPA >> > Pゴシック"(s) "IPAPGothic"(s) familylang: >> >>>>>>> "ja"(s) "en"(s) style: >> > "Regular"(w) stylelang: "en"(w) fullname: >> > "IPAPGothic"(w) "IPA P >> >>>>>>> ゴシック"(w) fullnamelang: "en"(w) > >> > "ja"(w) slant: 0(i)(s) weight: 80(i)(s) width: >> >>>>>>> 100(i)(s) size: 12(f)(s) pixelsize: > 12.5(f)(s) foundry: >> > "unknown"(w) antialias: >> >>>>>>> True(w) hintstyle: 0(i)(w) hinting: True(w) >> > verticallayout: False(s) autohint: True(w) >> >>>>>>> globaladvance: True(s) file: >> > "/usr/share/fonts/truetype/ipagp.ttf"(w) index: 0(i)(w) >> >>>>>>> outline: True(w) scalable: True(w) dpi: > 75(f)(s) rgba: >> > 5(i)(w) scale: 1(f)(s) charset: >> >>>>>>> [snip] (w) lang: >> >>>>>> >> >>>>> >> > > aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s) >> >>>>>>> >> >>>>> >> > fontversion: 198574(i)(s) >> >>>>>>> capability: "otlayout:hani otlayout:kana > >> > otlayout:latn"(w) fontformat: "TrueType"(w) >> >>>>>>> embeddedbitmap: True(w) decorative: False(s) > lcdfilter: >> > 0(i)(w) namelang: "ja"(s) >> >>>>>>> prgname: "fc-match"(s) hash: >> >>>>>> >> >>>>> >> > > "sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w) >> >>>>>>> postscriptname: "IPAPGothic"(w) >> > force_hintstyle: "none"(w) force_autohint: False(w) >> >>>>>>> force_bw: False(w) force_bw_monospace: > False(w) >> >>>>>>> >> >>>>>>> Output of fc-match -v Monospace >> >>>>>>> >> >>>>>>> Pattern has 42 elts (size 48) family: >> > "IPAゴシック"(s) "IPAGothic"(s) familylang: > "ja"(s) >> >>>>>>> "en"(s) style: > "Regular"(w) >> > stylelang: "en"(w) fullname: "IPAGothic"(w) >> > "IPAゴシック"(w) >> >>>>>>> fullnamelang: "en"(w) > "ja"(w) >> > slant: 0(i)(s) weight: 80(i)(s) width: 100(i)(s) size: >> >>>>>>> 12(f)(s) pixelsize: 12.5(f)(s) spacing: > 90(i)(w) >> > foundry: "unknown"(w) antialias: >> >>>>>>> True(w) hintstyle: 0(i)(w) hinting: True(w) >> > verticallayout: False(s) autohint: True(w) >> >>>>>>> globaladvance: True(s) file: >> > "/usr/share/fonts/truetype/ipag.ttf"(w) index: 0(i)(w) >> >>>>>>> outline: True(w) scalable: True(w) dpi: > 75(f)(s) rgba: >> > 5(i)(w) scale: 1(f)(s) charset: >> >>>>>>> [snip] (w) lang: >> >>>>>> >> >>>>> >> > > aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s) >> >>>>>>> >> >>>>> >> > fontversion: 198574(i)(s) >> >>>>>>> capability: "otlayout:hani otlayout:kana > >> > otlayout:latn"(w) fontformat: "TrueType"(w) >> >>>>>>> embeddedbitmap: False(w) decorative: False(s) > >> > lcdfilter: 0(i)(w) namelang: "ja"(s) >> >>>>>>> prgname: "fc-match"(s) hash: >> >>>>>> >> >>>>> >> > > "sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w) >> >>>>>>> postscriptname: "IPAGothic"(w) >> > force_hintstyle: "none"(w) force_autohint: False(w) >> >>>>>>> force_bw: False(w) force_bw_monospace: > False(w) >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>>> >> >>>>>>>> ----- Original Message ----- >> >>>>>>>>> From: Fuminobu TAKEYAMA > <[email protected]> >> > To: [email protected] Cc: Date: >> >>>>>>>>> 2015/6/10, Wed 19:43 Subject: Re: >> > [opensuse-m17n] Konsole displays some >> >>>>> characters >> >>>>>> wrong >> >>>>>>>>> >> >>>>>>>>> Hi, >> >>>>>>>>> >> >>>>>>>>> Which font do you use on Konsole? >> >>>>>>>>> >> >>>>>>>>> Then, can you share the result of > fc-match -v? >> > If the font is "Monospace" then run >> >>>>>>>>> the following >> >>>>>> command: >> >>>>>>>>> $ fc-match -v Monospace >> >>>>>>>>> >> >>>>>>>>> # you can snip off > "charset:" section >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> Fuminobu TAKEYAMA >> >>>>>>>>> >> >>>>>>>>> On 2015/06/09 11:41, Norman Diamond > wrote: >> >>>>>>>>>> In openSUSE 13.2, in Konsole, >> > downarrow↓displays↓wrong. The file encoding, >> >>>>>>>>>> Konsole terminal's encoding, > and >> >>>>> >> >>>>>> vim's >> >>>>>>>> encoding >> >>>>>>>>> are all UTF-8. >> >>>>>>>>>> >> > Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three. vim counts characters > properly >> >>>>>>>>>> but the display is >> >>>>> wrong. >> >>>>>>>>>> more, cat, etc., display wrong. >> >>>>>>>>>> >> >>>>>>>>>> I used Suse 11.0 for six years. > Konsole >> > had no problem >> >>>>> with >> >>>>>>>> downarrows. >> >>>>>>>>>> >> >>>>>>>>>> Yours sincerely, Norman Diamond >> >>>>>>>>>> >> >>>>>>>>> -- To unsubscribe, e-mail: >> >>>>> [email protected] >> >>>>>>>>> To contact the owner, e-mail: >> >>>>> [email protected] >> >>>>>>>>> >> >>>>>>>> >> >>>>>> -- To unsubscribe, e-mail: >> > [email protected] To contact the owner, >> >>>>>> e-mail: [email protected] >> >>>>>> >> >>>>> -- To unsubscribe, e-mail: >> > [email protected] To contact the owner, >> >>>>> e-mail: [email protected] >> >>>>> >> >>> -- To unsubscribe, e-mail: > [email protected] To >> > contact the owner, e-mail: >> >>> [email protected] >> >>> >> >>> >> >>> >> >>> >> > -- >> > To unsubscribe, e-mail: [email protected] >> > To contact the owner, e-mail: [email protected] >> > >> -- >> To unsubscribe, e-mail: [email protected] >> To contact the owner, e-mail: [email protected] >> > -- > To unsubscribe, e-mail: [email protected] > To contact the owner, e-mail: [email protected] > -- To unsubscribe, e-mail: [email protected] To contact the owner, e-mail: [email protected]