Re: [PATCH] emoji on TUI emacs

> Date: Thu, 30 Jul 2026 18:52:18 +0200
> Cc: [email protected]
> From: Kai Ma <[email protected]>
> 
> 
> > The way to make it possible to edit text with Emoji on text-only
> > frames is by disabling auto-composition-mode.  Did you try that?  If
> > you tried that, and you still find that "almost unusable", please tell
> > the details of what you see and preferably also show a simple recipe
> > to reproduce the problems you see.
> 
> Yes. I'm aware of this workaround. Sorry for not mentioning it in the 
> original mail. There are two drawbacks for this workaround:
> 
> 1. This breaks textual layout.
> 
> 2. This makes it other composition no longer work.

There are disadvantages, since this is just a workaround, yes.  But
Emacs is no longer "unusable", and there are no artifacts on the
screen.  So it is an improvement.

> >> Previous discussions are usually about the width of a grapheme
> >> cluster.  But to my surprise, I found that the problem occurs even if
> >> the width is correct.  For example, ⚡ (#x26a1) composed with #fe0f
> >> has width 2,
> >>
> >>       (string-width "\x26a1\xfe0f") ;; ==> 2
> > Yes, Emoji sequences which end in Variation Selector controls are one
> > of a couple of cases which currently cannot be correctly displayed on
> > text-only frames when auto-composition-mode is enabled.  Please try
> > disabling auto-composition-mode.
> 
> That's why I think there's something wrong with the current handling of 
> the composition.

Not generally with compositions, but specifically with Emoji sequences
that have VS controls or several Emoji codepoints that the terminal
shows as a single glyph.

> >> and it still causes garbled lines.  Therefore, the incorrect width of
> >> an emoji is not the _only_ problem.
> > It isn't the width of the Emoji, it's the fact that
> >
> >    (string-width "\x26a1") ;; ==> 2
> >
> > IOW, adding the VS-15 control leaves the width unaltered.
> 
> I'm talking about the composition. The impression I got from previous 
> discussions and etc/PROBLEMS are that the problem is *solely* about 
> width calculation.  That is not true!

Not IME, no.  The only problem I'm aware of is due to a mismatch
between the width of the composed Emoji as Emacs computes it and the
width the terminal actually draws on display.

> The example in my original mail shows that even if the width is correct, 
> the rendering issue still happens.

What do you see if you insert only "\x26a1" ?  It has the same width.

> >> Looking into the source code, in term.c, append_composite_glyph only
> >> appends one glyph, but in append_glyph, it appends pixel_width glyphs.
> >> Therefore, the attached patch simply copies the existing approach.  I
> >> have tested it on files containing many emojis, margins, and bidi
> >> texts, and observed no problems so far, but I might miss something so
> >> review is appreciated.
> > I don't understand why these changes are needed, or what is wrong with
> > the original code.
> 
> There are clearly two coordinate systems, the terminal one for cells 
> (denoted T) and the emacs internal matrix one (denoted E). E is 
> generated by append_composite_glyph filling in the glyph row, and T is 
> generated by the tty_* functions.
> 
> Let's say the buffer contains an emoji ⚡ and there's a character 'a' 
> follows it. The current append_composite_glyph assigns coordinates: '⚡' 
> E(line,col) and 'a' E(line,col+1).
> 
> When it's being output to the terminal, cmplus is used to maintain the T 
> coordinates. cmplus is used (in term.c) to advance the tty cursor, but 
> 'len' argument of 'tty_write_glyphs' etc is determined from the glyph 
> matrix. See, for example, in update_text_area in dispnew.c:
> 
> if (desired_row->used[TEXT_AREA]) rif->write_glyphs (w, updated_row, 
> desired_row->glyphs[TEXT_AREA], TEXT_AREA, desired_row->used[TEXT_AREA]);
> 
> The last argument is the number of cells of this write.
> 
> In the current implementation, '⚡' at E(line,col) is emitted at 
> T(line,col) (assuming nothing precedes it), but 'a' at T(line,col+1), 
> despite the fact that _Emacs itself_ thinks ⚡ has width 2.
> 
> So IMO this is an internal invariant violation.

And yet when '⚡', i.e. the single codepoint #x26a1, is displayed,
there are no display problems and no artifacts, the cursor is shown in
double width, and the cursor motion works correctly.  I tested this on
2 different systems (GNU/Linux and Windows) and with 2 different
terminal emulators.  Do you see something different?  If this simpler
case works for you, how do you explain it?

> > In general, it is wrong to produce as many glyphs as pixel_width for a
> > composite glyph, because Emacs cannot know the actual width of the
> > composed glyph on display, as shown on text-only frames.  The value of
> > pixel_width in this case is a guess based on what characters were
> > composed.  So your changes might work for some codepoints and on some
> > specific terminal emulators, but they will not work correctly in
> > general.
> 
> Pardon my ignorance. I don't follow you here.
> 
> Let's say the composition has Emacs width N, printed at T(0,0), where 
> should the next glyph be? I think it's fair to assume T(0,N).

The problem is with the value of N.  How can Emacs know what is the
width N of a composed sequence of codepoints?  It's the terminal
emulator that decides how to display them and with what font glyphs,
and Emacs has no access to that information when it displays on a
text-only terminal.  On GUI frames, we access the font information and
determine the metrics of the glyphs that are displayed as result of
the composition, but on text-only frames we cannot do that.

Try the Emoji sequence "\x1faf6\x1f3fc", for example.  On terminals
that support Emoji, it displays as a single glyph whose width is 2,
but Emacs thinks there are 2 separate glyphs there whose combined
pixel_width is 2 + 2 = 4.  And sure enough, (string-width "\x1faf6\x1f3fc")
returns 4 on a text-only frame, because Emacs doesn't think these two
codepoints will be combined into one on display.  How do you suggest
to resolve these problems?

> >> Also, if it's considered a safe fix, is it possible to install this
> >> patch on emacs-31, considering how annoying the problem is?
> > Sorry, I don't think the changes are correct, let alone safe.  Please
> > try disabling auto-composition-mode instead.
> 
> Do you have other acceptable alternative solutions in mind?

Not at the moment, no.  But I'm open to ideas.  For example, we could
perhaps do better for Emoji sequences whose first codepoint has a
width of 2, by adding a padding glyphs.  But that's a partial solution
in any case.  And adding padding glyphs according to pixel_width of a
composite glyph is incorrect in general, because on text-mode frames
that value is in many cases incorrect.