bug#81506: [PATCH] Extend w32 window events to expose IME composition string interface

[PATCH 3] Implement preedit text rendering for the IME window on w32

> This is the second patch.
> I fixed the static buffer issue from yesterday by switching to on-demand
allocation.>
> For w32term.c, I removed the direct execution of Lisp code and instead
communicate by writing to inev's args, following the preedit handling
approach in xterm.c.
> I kept the w32-ime-preedit flag to toggle the default IME window.
> The hook-based approach has been completely removed. Everything now works
through event reading.

This is a version based on the previous patch. This version adds the
following features:

- When composing CJK strings, the IME window no longer jitters along with
the cursor, keeping the behavior consistent with the native Windows input
method behavior.
- When composing CJK strings, while Emacs renders the preedit text, the
cursor can respond to arrow keys — specifically <left> and <right> behavior.

If you have any questions or concerns about the code, please feel free to
contact me.

I have set up a dedicated example repository to document the development of
this patch from scratch:
https://github.com/zHaOdANiuu/emacs-w32-draw-preedit.patch

Later on, I will attempt to implement the font size feature described in
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68339
, in order to improve the overall Windows input method experience in Emacs.

dANiuu zHaO <[email protected]> 于2026年7月30日周四 00:57写道：

> This is the second patch.
> I fixed the static buffer issue from yesterday by switching to on-demand
> allocation.
> For w32term.c, I removed the direct execution of Lisp code and instead
> communicate by writing to inev's args, following the preedit handling
> approach in xterm.c.
> I kept the w32-ime-preedit flag to toggle the default IME window.
> The hook-based approach has been completely removed. Everything now works
> through event reading.
>
> dANiuu zHaO <[email protected]> 于2026年7月29日周三 22:33写道：
>
>> For the IME window font size setting, I will try to implement it, and
>> once completed I will separate it into a standalone patch to facilitate
>> review.
>> However, what I currently prefer is having Emacs handle the rendering of
>> preedit text, because I prefer the preedit style more.
>> That said, I also understand that this feature involves quite a lot of
>> changes and will require a lengthy review and testing process. In
>> comparison, the font-size approach would be much easier to review and test.
>>
>> I revised the patch based on yesterday's email feedback, including string
>> allocation and other items. I made quite a few improvements to the code. I
>> expect to send the second patch in a few hours.
>>
>> > I see that you added a hook, but I think we should discuss the use
>> > cases for such a hook.  Also, we generally avoid installing features
>> > on Windows that have no equivalents on GNU/Linux, so does Emacs have
>> > anything similar on X?
>>
>> I took a look at other files under the src directory today and found that
>> xterm.c, xfns.c, and their corresponding Lisp files all have preedit
>> handling. I removed the hook approach and switched to an event-reading
>> approach instead. However, I still kept a Lisp flag, intending to implement
>> the ImmSetCompositionFont approach mentioned above. That way there can be
>> two sets of IME display control, allowing users to choose between the
>> preedit style and the default IME window.
>>
>> *  .\src\xfns.c (xic_preedit_caret_callback)
>> *  .\src\xterm.c (x_maybe_clear_preedit)
>> * .\lisp\term\x-win.el (x-preedit-text)
>>
>> Based on the code in these files, I learned about `PREEDIT_TEXT_EVENT`.
>> I'm glad — I think I now know how to write this properly.
>>
>> Eli Zaretskii <[email protected]> 于2026年7月29日周三 20:11写道：
>>
>>> [Please always use Reply to All when replying, to keep the bug tracker
>>> CC'ed.]
>>>
>>> > From: dANiuu zHaO <[email protected]>
>>> > Date: Tue, 28 Jul 2026 23:56:48 +0800
>>> >
>>> > Thanks for your review. Since this is my first time contributing code,
>>> there are indeed many areas where I can
>>> > improve.
>>> >
>>> > Let me first address the question about the purpose of the changes.
>>> The relevant bug report is:
>>> >
>>> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68339
>>> >
>>> > Emacs on Windows already supports the IME window, but the IME preedit
>>> window does not scale with the font
>>> > size in the buffer.  This report describes exactly this issue — when
>>> using Microsoft Pinyin IME to type Chinese
>>> > on Windows, the font size in the preedit window is too small to be
>>> legible.  I also don't have a good approach
>>> > for using ImmSetCompositionFont to address this.  So instead, I
>>> extended the WM_IME_COMPOSITION event
>>> > to obtain the composition string from CJK input methods.  My idea is
>>> that after obtaining these strings, I can
>>> > pass them to Elisp to render using overlays, rather than relying on
>>> the IME's own window to draw them.  This is
>>> > a new feature.
>>> >
>>> > This hook reads the composition string from the WM_IME_COMPOSITION
>>> event, and its sole purpose is this
>>> > use case.
>>>
>>> If this is for making the font of the IME window more like the size of
>>> the default font, then why not use ImmSetCompositionFont, as suggested
>>> by bug#68339?  I know that I said there it was not possible, but I
>>> think I was wrong: using ImmGetCompositionFont, we could retrieve the
>>> LOGFONT structure for the font the IME window uses, and then modify
>>> just one member of the structure, lfHeight, by setting it to the
>>> negative of FRAME_FONT (f)->pixel_size, and call
>>> ImmSetCompositionFont.  AFAIU, this should be done when we receive the
>>> WM_IME_STARTCOMPOSITION message, with the same context we use for
>>> calling ImmSetCompositionWindow.  Can you try that?
>>>
>>> > Regarding whether there is an equivalent implementation on GNU/Linux
>>> and X — I honestly don't know.  I do
>>> > most of my work on Windows, so I'm not familiar with that.  I
>>> apologize for this.
>>>
>>> If we use this for setting the font of the IME window, then we are
>>> okay, because we already have a similar code in xfns.c for X.
>>> However, providing a hook is much more general than this, so this is
>>> one more reason not to go the hook way.
>>>
>>> > As for the discussion from a few days ago about supporting IME in
>>> terminal (-nw) sessions — that is a difficult
>>> > challenge.  I think it heavily depends on whether the terminal itself
>>> handles IME input properly.  I haven't even
>>> > looked at Microsoft's documentation on this topic yet.  So I believe
>>> it is unrelated to our previous discussion.
>>>
>>> Understood.
>>>
>>> > Regarding the code comments and style — those oversights were my
>>> mistake, and I will fix them in the next
>>> > patch.
>>> >
>>> > The static buffer size limit is indeed too small.  I will switch to
>>> using MAX_ALLOCA or a similar approach.
>>> >
>>> > > +/* Fixed buffer for receiving IME composition (preedit) string.
>>> > > +   IME composition strings are typically short, so 64 characters
>>> > > +   is sufficient for most cases. The buffer is sized to 2 * 64
>>> > > +   bytes because each wide character (wchar_t) occupies 2 bytes
>>> > > +   under UTF-16 encoding on Windows. */
>>> > > +static char composition_ime_buf[2 * 64];
>>> >
>>> > For the wchar_t buffer allocation, multiplying by 4 when converting to
>>> UTF-8 would indeed be better.  The IME
>>> > composition strings produced by the input method I personally use do
>>> not exceed 2 bytes when converted to
>>> > UTF-8, so I made that assumption and overlooked the fact that other
>>> CJK input methods may produce longer
>>> > strings.
>>> >
>>> > > +       wlen = WideCharToMultiByte (CP_UTF8, 0,
>>> > > +                                   wbuf,
>>> > > +                                   size / sizeof (wchar_t),
>>> > > +                                   composition_ime_buf,
>>> > > +                                   sizeof (composition_ime_buf),
>>> > > +                                   NULL, NULL);
>>> >
>>> > Regarding calling Lisp inside read_socket_hook — I also feel that
>>> calling Lisp there is somewhat inappropriate;
>>> > it is a bit slow.  As for safety, I haven't encountered issues in my
>>> own local testing, but more testing is needed to
>>> > verify.  So I would like to know: what is the recommended path within
>>> the project for notifying Lisp with data
>>> > obtained from window events?  I will follow whatever the recommended
>>> approach is.
>>> >
>>> > > +     case WM_IME_COMPOSITION:
>>> > > +       {
>>> > > +         int len = (int) msg.msg.lParam;
>>> > > +         char *buf = (char *) msg.msg.wParam;
>>> > > +         Lisp_Object str = make_string_from_utf8 (buf, len);
>>> > > +         CALLN (Frun_hook_with_args, Qw32_ime_composition_hook,
>>> str);
>>> >
>>> > Regarding the review of w32-ime-preedit and w32-ime-composition-hook —
>>> you are right that w32-ime-preedit
>>> > is not needed.  We can simply check whether w32-ime-composition-hook
>>> is nil.  I didn't think of this at all when
>>> > writing it.  I agree this is absolutely the right approach.
>>> >
>>> > Finally, thank you for your review.  I plan to send a second patch in
>>> the next few days.  I would like to get this
>>> > feature into Emacs — it will allow users on Windows to see the IME
>>> preedit window clearly without needing to
>>> > install a third-party input method package.
>>>
>>> Thanks.
>>>
>>

From 7806a8919b7cce74c7dbeb65583767a949cc18f3 Mon Sep 17 00:00:00 2001
From: zdn <[email protected]>
Date: Thu, 30 Jul 2026 19:06:09 +0800
Subject: [PATCH] Implement preedit text rendering for the IME window on w32

* lisp/term/w32-win.el (w32-clear-preedit-text, w32-preedit-text):
Draw w32 IME preedit text.
* src/w32fns.c (w32_wnd_proc): Extend the WM_IME_COMPOSITION event
and handle edge cases in WM_IME_STARTCOMPOSITION
* src/w32term.c (w32_read_socket): Handle WM_IME_COMPOSITION message
---
 lisp/term/w32-win.el | 32 ++++++++++++++++
 src/w32fns.c         | 87 +++++++++++++++++++++++++++++++++++++++-----
 src/w32term.c        | 46 +++++++++++++++++++++++
 3 files changed, 156 insertions(+), 9 deletions(-)

diff --git a/lisp/term/w32-win.el b/lisp/term/w32-win.el
index 385f488..c6d9e12 100644
--- a/lisp/term/w32-win.el
+++ b/lisp/term/w32-win.el
@@ -621,6 +621,38 @@ w32-find-non-USB-fonts
     (clear-font-cache)
     (and val (setq w32-non-USB-fonts val))))
 
+;; The preedit rendering code below is adapted from x-win.el,
+;; with the variable and function prefixes changed to w32 and a few minor modifications.
+(defvar w32-preedit-overlay nil
+  "The overlay currently used to display preedit text from a compose sequence.")
+
+(defun w32-clear-preedit-text ()
+  "Clear the pre-edit overlay and remove itself from `pre-command-hook'.
+This function should be installed in `pre-command-hook' whenever
+preedit text is displayed."
+  (when w32-preedit-overlay
+    (delete-overlay w32-preedit-overlay)
+    (setq w32-preedit-overlay nil))
+  (remove-hook 'pre-command-hook #'w32-clear-preedit-text))
+
+(defun w32-preedit-text (event)
+  "Display preedit text from a compose sequence in EVENT.
+EVENT is a preedit-text event."
+  (interactive "e")
+  (when w32-ime-preedit
+    (when w32-preedit-overlay
+      (delete-overlay w32-preedit-overlay)
+      (setq w32-preedit-overlay nil)
+      (remove-hook 'pre-command-hook #'w32-clear-preedit-text))
+    (when (nth 1 event)
+      (let ((string (propertize (nth 1 event) 'face '(:underline t))))
+        (setq w32-preedit-overlay (make-overlay (point) (point)))
+        (add-hook 'pre-command-hook #'w32-clear-preedit-text)
+        (overlay-put w32-preedit-overlay 'window (selected-window))
+        (overlay-put w32-preedit-overlay 'before-string string)))))
+
+(define-key special-event-map [preedit-text] 'w32-preedit-text)
+
 (provide 'w32-win)
 (provide 'term/w32-win)
 
diff --git a/src/w32fns.c b/src/w32fns.c
index 726fe52..fd2c5eb 100644
--- a/src/w32fns.c
+++ b/src/w32fns.c
@@ -271,6 +271,15 @@ #define CCHDEVICENAME 32
 /* Flag to selectively ignore WM_IME_CHAR messages.  */
 static int ignore_ime_char = 0;
 
+/* Fixed buffer for receiving IME composition (preedit) string.  */
+static wchar_t *composition_ime_data;
+
+/* Used to preserve the position of the IME window during the WM_IME_COMPOSITION event,
+   preventing other events from updating the w32_system_caret_x variable and
+   causing the IME window's x position to jitter,
+   in order to remain consistent with Windows' default behavior.  */
+static int composition_ime_pt_x = -1;
+
 /* W95 mousewheel handler */
 extern unsigned int msh_mousewheel;
 unsigned int msh_mousewheel = 0;
@@ -5058,15 +5067,35 @@ w32_wnd_proc (HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
 	  form.ptCurrentPos.x = w32_system_caret_x;
 	  form.ptCurrentPos.y = w32_system_caret_y;
 
-	  form.rcArea.left = WINDOW_TEXT_TO_FRAME_PIXEL_X (w, 0);
-	  form.rcArea.top = (WINDOW_TOP_EDGE_Y (w)
-			     + w32_system_caret_hdr_height);
-	  form.rcArea.right = (WINDOW_BOX_RIGHT_EDGE_X (w)
-			       - WINDOW_RIGHT_MARGIN_WIDTH (w)
-			       - WINDOW_RIGHT_FRINGE_WIDTH (w));
-	  form.rcArea.bottom = (WINDOW_BOTTOM_EDGE_Y (w)
-				- WINDOW_BOTTOM_DIVIDER_WIDTH (w)
-				- w32_system_caret_mode_height);
+	  if (w32_ime_preedit)
+	    {
+	      if (composition_ime_pt_x > 0)
+		form.ptCurrentPos.x = composition_ime_pt_x;
+	      /* After hiding the preedit window, the original position is
+		 a bit too high.  Subtracting half the font pixel size gives
+		 better vertical centering.  The extra 2 pixels prevent the
+		 window from touching the edge.  */
+	      form.ptCurrentPos.y -= -FRAME_FONT (f)->pixel_size / 2 - 2;
+	      /* Set rcArea left and top to negative values to hide the preedit window.
+		 Removing the ISC_SHOWUICOMPOSITIONWINDOW flag from lParam in
+		 WM_IME_SETCONTEXT has no effect.
+		 See: https://learn.microsoft.com/en-us/windows/win32/intl/wm-ime-setcontext
+		 So we move the constraint area offscreen instead.  */
+	      form.rcArea.left = -LONG_MAX;
+	      form.rcArea.top = -LONG_MAX;
+	    }
+	  else
+	    {
+	      form.rcArea.left = WINDOW_TEXT_TO_FRAME_PIXEL_X (w, 0);
+	      form.rcArea.top = (WINDOW_TOP_EDGE_Y (w)
+				 + w32_system_caret_hdr_height);
+	      form.rcArea.right = (WINDOW_BOX_RIGHT_EDGE_X (w)
+				   - WINDOW_RIGHT_MARGIN_WIDTH (w)
+				   - WINDOW_RIGHT_FRINGE_WIDTH (w));
+	      form.rcArea.bottom = (WINDOW_BOTTOM_EDGE_Y (w)
+				    - WINDOW_BOTTOM_DIVIDER_WIDTH (w)
+				    - w32_system_caret_mode_height);
+	    }
 
 	  /* Punt if the window was deleted behind our back.  */
 	  if (!BUFFERP (w->contents))
@@ -5088,8 +5117,48 @@ w32_wnd_proc (HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
 	 supported by the input method...  */
       goto dflt;
 
+    case WM_IME_COMPOSITION:
+      if (!w32_ime_preedit)
+	goto dflt;
+
+      if (lParam & GCS_RESULTSTR)
+	{
+	  composition_ime_pt_x = -1;
+	  my_post_msg (&wmsg, hwnd, msg, (WPARAM) "", 0);
+	}
+      else if (lParam & (GCS_COMPSTR | GCS_CURSORPOS))
+	{
+	  int size = 0;
+	  int cursor_pos = 0;
+	  HIMC context = get_ime_context_fn (hwnd);
+
+	  if (composition_ime_pt_x < 0)
+	    composition_ime_pt_x = w32_system_caret_x;
+
+	  free (composition_ime_data);
+	  composition_ime_data = NULL;
+
+	  size = get_composition_string_fn (context, GCS_COMPSTR, NULL, 0);
+	  composition_ime_data = malloc (size  + sizeof (cursor_pos));
+	  size = get_composition_string_fn (context, GCS_COMPSTR,
+					    composition_ime_data, size);
+	  cursor_pos = get_composition_string_fn (context, GCS_CURSORPOS,
+						  NULL, 0);
+	  /* offset end, write cursor position.  */
+	  memcpy ((char *) composition_ime_data + size,
+		  &cursor_pos, sizeof (cursor_pos));
+	  release_ime_context_fn (hwnd, context);
+
+	  my_post_msg (&wmsg, hwnd, msg,
+		       (WPARAM) composition_ime_data, size);
+	  break;
+	}
+
+      goto dflt;
+
     case WM_IME_ENDCOMPOSITION:
       ignore_ime_char = 0;
+      composition_ime_pt_x = -1;
       goto dflt;
 
       /* Simulate middle mouse button events when left and right buttons
diff --git a/src/w32term.c b/src/w32term.c
index 728f6ce..58e915b 100644
--- a/src/w32term.c
+++ b/src/w32term.c
@@ -5387,6 +5387,45 @@ w32_read_socket (struct terminal *terminal,
 	    }
 	  break;
 
+	case WM_IME_COMPOSITION:
+	  {
+	    f = w32_window_to_frame (dpyinfo, msg.msg.hwnd);
+	    Lisp_Object str = Qnil;
+	    int size = (int) msg.msg.lParam;
+
+	    if (size > 0)
+	      {
+		int cursor_pos;
+		int utf8_len;
+		char *utf8_buf;
+		wchar_t *wbuf = (wchar_t *) msg.msg.wParam;
+		utf8_len = WideCharToMultiByte (CP_UTF8, 0,
+						wbuf, size / sizeof(wchar_t),
+						NULL, 0,
+						NULL, NULL);
+		utf8_buf = alloca (utf8_len);
+		utf8_len = WideCharToMultiByte(CP_UTF8, 0,
+					       wbuf, size / sizeof(wchar_t),
+					       utf8_buf, utf8_len,
+					       NULL, NULL);
+		str = make_string_from_utf8 (utf8_buf, utf8_len);
+
+                /* offaet end, read cursor position.  */
+		memcpy (&cursor_pos, (char *) wbuf + size, sizeof (cursor_pos));
+                /* see xic_preedit_draw_callback function.  */
+		Fput_text_property (make_fixnum (min (SCHARS (str), max (0, cursor_pos))),
+				    make_fixnum (min (SCHARS (str), max (0, cursor_pos) + 1)),
+				    Qcursor, Qt, str);
+	      }
+
+	    inev.kind = PREEDIT_TEXT_EVENT;
+	    inev.arg = str;
+	    inev.modifiers = msg.dwModifiers;
+	    XSETFRAME (inev.frame_or_window, f);
+	    inev.timestamp = msg.msg.time;
+	  }
+	  break;
+
         case WM_APPCOMMAND:
 	  f = w32_window_to_frame (dpyinfo, msg.msg.hwnd);
 
@@ -8392,6 +8431,13 @@ syms_of_w32term (void)
 API.  */);
   w32_add_wrapped_menu_bar_lines = 1;
 
+  DEFVAR_BOOL ("w32-ime-preedit",
+	       w32_ime_preedit,
+	       doc: /* Non-nil means report IME preedit strings to
+`w32-ime-composition-hook'.  When nil, the IME preedit is not
+reported and the IME works as normal.  */);
+  w32_ime_preedit = 0;
+
   /* Tell Emacs about this window system.  */
   Fprovide (Qw32, Qnil);
 }
-- 
2.54.0