Re: [PATCH] emoji on TUI emacs

Eli Zaretskii <[email protected]> writes:

>> Date: Wed, 5 Aug 2026 04:45:27 +0200
>> Cc: [email protected]
>> From: Kai Ma <[email protected]>
>> 
>> Attached are the three patches that make textual Emacs work with emoji 
>> on modern terminal emulators.
>
> Thanks.  Please find a few comments to the patch below.

Thanks for the review.  I have updated the patches accordingly.

>> For your enjoyment, a smaller file containing many different kinds of 
>> emoji sequences is also attached.
>> 
>> First, the test results. So far, there are 3 main issues with Emoji 
>> support in the textual terminals.
>> 
>> * Issue 1: Keycap sequences like 3️⃣ (#x33 #xfe0f #x20e3) are not 
>> supported. They are explicitly excluded in emoji-zwj.awk: "FIXME: add 
>> support for Emoji_Keycap_Sequence once we learn how to respect 
>> FE0F/VS-16 for ASCII characters." But I'm not very sure what is missing. 
>> Could someone enlighten me here?
>
> See the (very long) discussion in bug#39799.  The problem here is that
> Emacs can only support a composition of a sequence of characters if
> all of those characters have a glyph in the same font used for all of
> them.  But ASCII characters are always displayed by the default face's
> font, which is normally not an Emoji font, and so doesn't usually
> support VS-16.  So compositions that begin with an ASCII codepoint
> don't work.

Thanks for the background.  However, I didn't manage to find an
acceptable way to support both tty and graphical displays, since, IIUC,
both textual and graphical terminals reuse the same composition table.

So I think I'll leave this open in tty terminals as well.

> However, this doesn't have to affect TTY frames, see below.
>
>> (BTW, during testing I find some East_Asian_Neutral characters misalign 
>> because Emacs marks them as wide. I'm not sure about know the rationale. 
>> Patch 2 removes them so they get width 1, in accordance to the Unicode 
>> standard. But perhaps there's some background I'm missing?)
>
> We generally go by the data in the file EastAsianWidth.txt, but at
> least #x2690..#x2692 look almost double-width on my system.  Isn't
> that what you see?

#x2690..#x2692 look narrow here.

I do observe some symbols that appear double-width or near double-width,
e.g. on WezTerm,

#x1fa60 🩠  XIANGQI RED GENERAL

But they function as if they were single-width despite the appearance,
so IMHO it's correct to treat them as narrow.

>> A follow-up question is: should we make string-width recognize emoji 
>> sequences and return 2 instead of 1? Or at least update org-string-width.
>
> string-width considers compositions, so if it produces results
> different from the actual display, it means composition_gstring_width
> needs to be fixed, IMO.

Thanks.  It seems in lisp_string_width the auto_comp branch is limited
to only graphical terminals.  I removed #ifdef HAVE_WINDOW_SYSTEM and
made some minor changes for tty terminals.

string-width returns 2 after the patch for fully-matched composed emoji
sequences on tty terminals:

    (string-width (string #x2665))         ;; ==> 1
    (string-width (string #x2665 #xfe0f))  ;; ==> 2
    (string-width (string #x1f3f4 #x200d #x2620 #xfe0f))
                                           ;; ==> 2

And I can confirm org-table alignment works properly now.

However, the column number change due to C-f, C-b, etc. is still the
total sum of CHARACTER_WIDTH (not necessarily 2).  Do we want to change
it as well?

>> +static bool
>> +composite_glyph_is_emoji_sequence (struct it *it)
>> +{
>> +  Lisp_Object gstring;
>> +  int first, second;
>> +
>> +  /* Skip static compositions.  */
>> +  if (it->cmp_it.ch < 0)
>> +    return false;
>> +
>> +  gstring = composition_gstring_from_id (it->cmp_it.id);
>> +  if (it->cmp_it.from + 1 >= it->cmp_it.to)
>> +    return false;
>> +  first = XFIXNUM (LGSTRING_CHAR (gstring, it->cmp_it.from));
>> +  second = XFIXNUM (LGSTRING_CHAR (gstring, it->cmp_it.from + 1));
>> +
>> +  /* Base + VS16/Modifier.  */
>> +  if (second == 0xFE0F || (0x1F3FB <= second && second <= 0x1F3FF))
>> +    {
>> +      Lisp_Object tail = Vauto_composition_emoji_eligible_codepoints;
>> +
>> +      FOR_EACH_TAIL (tail)
>>  	{
>> -	  glyph->resolved_level = 0;
>> -	  glyph->bidi_type = UNKNOWN_BT;
>> +	  Lisp_Object item = XCAR (tail);
>> +	  if (FIXNUMP (item) && XFIXNUM (item) == first)
>> +	    return true;
>
> Any reason not to use Fmember here?

Fixed.  That was my mistake.

> Also, I think we should have a separate variable for the TTY case, see
> below.

Fixed.

>> +  DEFVAR_BOOL ("tty-display-emoji-force-wide", tty_display_emoji_force_wide,
>> +    doc: /* Whether TTY frames should always render emoji sequences as
>> +wide glyphs.
>
> The first line of a doc string should always be a single complete
> sentence.  Also, "render" is not accurate here (it's the terminal that
> renders them).  So I suggest
>
>   Whether Emoji sequences on TTY frames should always be considered
>   wide.

Fixed. 

>> +When nil, emoji sequences whose first character's width is 1 may be
>> +rendered as a narrow glyph to be compatible with some non-compliant
>    ^^^^^^^^
> Again, "considered"

Fixed.

>> +  tty_display_emoji_force_wide = 1;
>
> Please use 'true' here instead of 1.

Fixed.

> I think we should leave auto-composition-emoji-eligible-codepoints
> alone, and introduce a separate variable for TTY frames.  The reason
> is a bit subtle, and is again related to the way Emacs selects fonts
> for non-ASCII characters on GUI frames.  Since we want the codepoints
> hard-coded above to be shown as their normal non-Emoji glyphs,
> preferably using the default face's font, we cannot add them to the
> 'emoji' script, thus forcing them to be displayed by fonts used for
> Emoji.  But OTOH, when they are followed by VS-16, we must force Emacs
> to use the Emoji font for them, or else there will be no composition.
> The comment above tells you that we have a trick in the code for that.
>
> But all of this is not relevant for TTY frames, because we don't care
> about the fonts, which are the responsibility of the terminal.  So I
> think we should have a separate variable which lists all of the
> codepoints, but used only in term.c, so as not to inadvertently affect
> GUI frames.  For TTY frames, we should simply have there all the
> codepoints in emoji-variation-sequences.txt.  Does this make sense?

Yes.  Added auto-composition-emoji-tty-eligible-codepoints that collects
codepoints from emoji-variation-sequences.txt.

>
>> -     print "                                           #'compose-gstring-for-graphic))))"
>> +     print "                                           #'compose-emoji-gstring))))"
>
> The name of the function compose-emoji-gstring is somewhat misleading.
> I'd prefer something like compose-gstring-and-emoji instead.

Fixed.

>> +(defvar auto-compose-emoji-for-terminal t
>> +  "Enable auto composition of Emoji sequences on text terminals.
>
> Once again, this is not about enabling such composition.  This is
> about Emacs being in sync with what the terminal actually does with
> Emoji sequences of more than one codepoint.  A non-nil value makes
> Emacs assume the terminal composes them.
>
> And I think when the user sets this to nil, he/she should also disable
> auto-mode, right?  Or at least to consider that?

Yes.  This was also the reason why t was the default value.  On a second
thought, this option seems to be pointless because there does not appear
to be a case where you enable auto-composition-mode but do not want to
compose emoji sequences.

In the updated patch, I removed this option to simplify things.

>> +(defun compose-emoji-gstring (gstring _direction)
>> +  "Compose Emoji sequences into a grapheme cluster.
>> +This function is redirected to `compose-glyph-for-graphic' by
>> +`auto-compose-chars' on graphical display."
>
> There's something wrong with the last sentence.  "Redirected to" is
> incorrect, I think, and there's no such thing as
> compose-glyph-for-graphic.

Fixed.