Re: Konsole displays some characters wrong

 > I do not agree with your statement that downarrow should be displayed as half-width.  It is still
 > the same character that it used to be in ShiftJIS and EUC encodings, and the adoption of Unicode
 > and UTF-8 does not change the width of the glyph.

The ambiguous width problem have been discussed for a long time.
There is no way to distinguish CJK down arrow and non-CJK down arrow.


 > If I the character becomes half-width when I edit source files in Konsole, but it's still
 > full-width when other people edit the same source files in a Gnome terminal or even in Windows,
 > it will result in people scrambling source files.

If they edit GNOME terminal without CJK fonts, arrows are half-width.
Even if we resolve this Konsole bug, your problem will not be resolved.

You still need to keep in mind that down arrow (and some other characters) have
ambiguous width.



 > I do not know who decided that downarrow should have an ambiguous width.
 > If the Unicode consortium decided that, they're causing problems.

They did:
http://www.unicode.org/reports/tr11/tr11-28.html#Ambiguous
http://ja.wikipedia.org/wiki/東アジアの文字幅


Anyway, let's fix this Konsole's bug.

It is quite difficult to propose a new Unicode specification that can be
acceptable for both CJK and non-CJK people.

Fuminobu TAKEYAMA

On 2015/06/13 10:35, Norman Diamond wrote:
> Since the font selection dialog's preview uses Qt widgets instead of whatever Konsole uses, we
> know that the full-width downarrow character is displayed as full-width by these Linux tools.
> Furthermore in Suse 11.0 Konsole itself also displayed the full-width downarrow character as
> full-width.  So I think this bug is in the newer version of Konsole itself, not in the rest of
> Linux.  I can't say if the preview window should be removed because it doesn't really provide a
> preview, but it helped narrow down this bug.
>
> I do not agree with your statement that downarrow should be displayed as half-width.  It is still
> the same character that it used to be in ShiftJIS and EUC encodings, and the adoption of Unicode
> and UTF-8 does not change the width of the glyph.
>
> If I the character becomes half-width when I edit source files in Konsole, but it's still
> full-width when other people edit the same source files in a Gnome terminal or even in Windows,
> it will result in people scrambling source files.
>
> Full-width ａ and half-width a do not have ambiguous widths.  Full-width ア and half-width ｱ do not
> have ambiguous widths.  I do not know who decided that downarrow should have an ambiguous width.
> If the Unicode consortium decided that, they're causing problems.
>
> Yours sincerely, Norman Diamond
>
>
> ----- Forwarded Message -----
>> From: Fuminobu TAKEYAMA <[email protected]> To: [email protected] Date: 2015/6/13, Sat
>> 01:30 Subject: Re: [opensuse-m17n] Konsole displays some characters wrong
>>
>>>> I still think there's a bug.
>>
>> Yes, this is clearly a Japanese specific bug. Could you file a bug report for openSUSE and
>> KDE?
>>
>> I saw a patch to fix the same problem with emacs about 5 years ago. So I think this problem can
>> be resolved by some hacks.
>>
>>
>>>> Konsole knows that the glyph for 字 is full-width, so Konsole should know that the glyph for
>>>> downarrow is full-width.
>>
>> Down arrow is known as ambiguous-width character. Monospaced font should provide half-width
>> down arrow but some Japanese fonts such as IPA Gothic and MS Gothic do full-width down arrow
>> from a kind of historical reason.
>>
>>
>>>> The version of Konsole that was included in Suse 11.0 did know that downarrow is
>>>> full-width.  I never had to look at what the default font was because it just worked.
>>
>> Konsole's character layouting was changed recently as far as I know so that some
>> (flexible-width) bold fonts can be placed correctly.
>>
>> I guess it causes this problem.
>>
>>
>>> In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box. I can type or paste text in
>>> the text box. I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they display correctly in that
>>> text box. Konsole corrupts the display in its main window, and the preview window does not
>>> preview the
>> corruption.
>>
>> Unfortunately, the preview is for font selection dialog and Qt widgets. It is not preview of
>> Konsole.
>>
>> Maybe, the preview should be removed.
>>
>>
>> Fuminobu TAKEYAMA
>>
>> On 2015/06/12 7:05, Norman Diamond wrote:
>>> There is a worse bug.
>>>
>>> In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box. I can type or paste text in
>>> the text box. I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they display correctly in that
>>> text box. Konsole corrupts the display in its main window, and the preview window does not
>>> preview the corruption.
>>>
>>> Yours sincerely,Norman Diamond
>>>
>>>
>>> Norman Diamond wrote:
>>>
>>>> I click 設定 > 現在のプロファイルを編集(E)... > 外観 In フォント > プレビュー it says Monospace Oh now I see, next
>>>> to フォントサイズ where it lets me change the size, there's a clickable button to select a font.
>>>> In that panel it says モノスペース instead of Monospace.
>>>>
>>>> Thank you for the workaround.  I will experiment with some fonts.  I would like to
>>>> recommend that the preview box should include some full-width characters so that users can
>>>> judge more easily.  If a font doesn't include glyphs for full-width characters then users
>>>> will see immediately that they shouldn't experiment with such a font.
>>>>
>>>> I still think there's a bug.
>>>>
>>>> Konsole knows that the glyph for 字 is full-width, so Konsole should know that the glyph for
>>>> downarrow is full-width.
>>>>
>>>> The version of Konsole that was included in Suse 11.0 did know that downarrow is
>>>> full-width.  I never had to look at what the default font was because it just worked.
>>>>
>>>> Yours sincerely, Norman Diamond
>>>>
>>>>
>>>> Fuminobu TAKEYAMA wrote:
>>>>
>>>>> Ok, Monospace is an alias of IPAGothic on your environment.
>>>>>
>>>>> The font, IPAGothic provides double-width down-allow glyph although Konsole assumes that
>>>>> the glyph should have the same width as glyphs such as 'a' and '0'.
>>>>>
>>>>> A solution is to use another font such as DejaVu Sans Mono. You can change fonts from:
>>>>> Settings > Edit Current Profile... > Appearance
>>>>>
>>>>> If you do not need Japanese fonts, please uninstall ipa-gothic-fonts. So another font is
>>>>> to be selected for Monospace.
>>>>>
>>>>>
>>>>> Fuminobu TAKEYAMA
>>>>>
>>>>>
>>>>> On 2015/06/11 6:42, Norman Diamond wrote:
>>>>>> Yes the font is Monospace.  I did not change it from the default.  A
>>>>> Konsole settings panel displayed the name Monospace and allowed me to
>>>> change the
>>>>> font size but didn't allow me to select a different font.
>>>>>>
>>>>>> For this e-mail I use Yahoo's web interface from Internet Explorer
>>>>
>>>>> under Windows, so I translated the output of fc-match -v and fc-match -v Monospace from
>>>>> UTF-8 to Microsoft's Unicode encoding.  Yahoo's web interface will scramble the text that
>>>>> I will paste below, so I ALSO attach
>>>> the
>>>>> outputs as files.
>>>>>>
>>>>>>
>>>>>> Output of fc-match -v
>>>>>>
>>>>>> Pattern has 41 elts (size 48) family: "IPA Pゴシック"(s) "IPAPGothic"(s) familylang:
>>>>>> "ja"(s) "en"(s) style: "Regular"(w) stylelang: "en"(w) fullname: "IPAPGothic"(w) "IPA P
>>>>>> ゴシック"(w) fullnamelang: "en"(w) "ja"(w) slant: 0(i)(s) weight: 80(i)(s) width:
>>>>>> 100(i)(s) size: 12(f)(s) pixelsize: 12.5(f)(s) foundry: "unknown"(w) antialias:
>>>>>> True(w) hintstyle: 0(i)(w) hinting: True(w) verticallayout: False(s) autohint: True(w)
>>>>>> globaladvance: True(s) file: "/usr/share/fonts/truetype/ipagp.ttf"(w) index: 0(i)(w)
>>>>>> outline: True(w) scalable: True(w) dpi: 75(f)(s) rgba: 5(i)(w) scale: 1(f)(s) charset:
>>>>>> [snip] (w) lang:
>>>>>
>>>> aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
>>>>>>
>>>>
fontversion: 198574(i)(s)
>>>>>> capability: "otlayout:hani otlayout:kana otlayout:latn"(w) fontformat: "TrueType"(w)
>>>>>> embeddedbitmap: True(w) decorative: False(s) lcdfilter: 0(i)(w) namelang: "ja"(s)
>>>>>> prgname: "fc-match"(s) hash:
>>>>>
>>>> "sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
>>>>>> postscriptname: "IPAPGothic"(w) force_hintstyle: "none"(w) force_autohint: False(w)
>>>>>> force_bw: False(w) force_bw_monospace: False(w)
>>>>>>
>>>>>> Output of fc-match -v Monospace
>>>>>>
>>>>>> Pattern has 42 elts (size 48) family: "IPAゴシック"(s) "IPAGothic"(s) familylang: "ja"(s)
>>>>>> "en"(s) style: "Regular"(w) stylelang: "en"(w) fullname: "IPAGothic"(w) "IPAゴシック"(w)
>>>>>> fullnamelang: "en"(w) "ja"(w) slant: 0(i)(s) weight: 80(i)(s) width: 100(i)(s) size:
>>>>>> 12(f)(s) pixelsize: 12.5(f)(s) spacing: 90(i)(w) foundry: "unknown"(w) antialias:
>>>>>> True(w) hintstyle: 0(i)(w) hinting: True(w) verticallayout: False(s) autohint: True(w)
>>>>>> globaladvance: True(s) file: "/usr/share/fonts/truetype/ipag.ttf"(w) index: 0(i)(w)
>>>>>> outline: True(w) scalable: True(w) dpi: 75(f)(s) rgba: 5(i)(w) scale: 1(f)(s) charset:
>>>>>> [snip] (w) lang:
>>>>>
>>>> aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
>>>>>>
>>>>
fontversion: 198574(i)(s)
>>>>>> capability: "otlayout:hani otlayout:kana otlayout:latn"(w) fontformat: "TrueType"(w)
>>>>>> embeddedbitmap: False(w) decorative: False(s) lcdfilter: 0(i)(w) namelang: "ja"(s)
>>>>>> prgname: "fc-match"(s) hash:
>>>>>
>>>> "sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
>>>>>> postscriptname: "IPAGothic"(w) force_hintstyle: "none"(w) force_autohint: False(w)
>>>>>> force_bw: False(w) force_bw_monospace: False(w)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: Fuminobu TAKEYAMA <[email protected]> To: [email protected] Cc: Date:
>>>>>>>> 2015/6/10, Wed 19:43 Subject: Re: [opensuse-m17n] Konsole displays some
>>>> characters
>>>>> wrong
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Which font do you use on Konsole?
>>>>>>>>
>>>>>>>> Then, can you share the result of fc-match -v? If the font is "Monospace" then run
>>>>>>>> the following
>>>>> command:
>>>>>>>> $ fc-match -v Monospace
>>>>>>>>
>>>>>>>> # you can snip off "charset:" section
>>>>>>>>
>>>>>>>>
>>>>>>>> Fuminobu TAKEYAMA
>>>>>>>>
>>>>>>>> On 2015/06/09 11:41, Norman Diamond wrote:
>>>>>>>>> In openSUSE 13.2, in Konsole, downarrow↓displays↓wrong. The file encoding,
>>>>>>>>> Konsole terminal's encoding, and
>>>>
>>>>> vim's
>>>>>>> encoding
>>>>>>>> are all UTF-8.
>>>>>>>>> Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three. vim counts characters properly
>>>>>>>>> but the display is
>>>> wrong.
>>>>>>>>> more, cat, etc., display wrong.
>>>>>>>>>
>>>>>>>>> I used Suse 11.0 for six years.  Konsole had no problem
>>>> with
>>>>>>> downarrows.
>>>>>>>>>
>>>>>>>>> Yours sincerely, Norman Diamond
>>>>>>>>>
>>>>>>>> -- To unsubscribe, e-mail:
>>>> [email protected]
>>>>>>>> To contact the owner, e-mail:
>>>> [email protected]
>>>>>>>>
>>>>>>>
>>>>> -- To unsubscribe, e-mail: [email protected] To contact the owner,
>>>>> e-mail: [email protected]
>>>>>
>>>> -- To unsubscribe, e-mail: [email protected] To contact the owner,
>>>> e-mail: [email protected]
>>>>
>> -- To unsubscribe, e-mail: [email protected] To contact the owner, e-mail:
>> [email protected]
>>
>>
>>
>>
-- 
To unsubscribe, e-mail: [email protected]
To contact the owner, e-mail: [email protected]