bug#81506: [PATCH] Extend w32 window events to expose IME composition string interface

dANiuu zHaO <[email protected]> writes:

> [PATCH 3] Implement preedit text rendering for the IME window on w32
>
>> This is the second patch.
>> I fixed the static buffer issue from yesterday by switching to on-demand allocation.> 
>> For w32term.c, I removed the direct execution of Lisp code and instead communicate by writing to inev's args, following the preedit handling
> approach in xterm.c.
>> I kept the w32-ime-preedit flag to toggle the default IME window.
>> The hook-based approach has been completely removed. Everything now works through event reading.
>
> This is a version based on the previous patch. This version adds the following features:
>
> - When composing CJK strings, the IME window no longer jitters along with the cursor, keeping the behavior consistent with the native Windows
> input method behavior.
> - When composing CJK strings, while Emacs renders the preedit text, the cursor can respond to arrow keys — specifically <left> and <right>
> behavior.
>
> If you have any questions or concerns about the code, please feel free to contact me.
>
> I have set up a dedicated example repository to document the development of this patch from scratch:
> https://github.com/zHaOdANiuu/emacs-w32-draw-preedit.patch
>
> Later on, I will attempt to implement the font size feature described in 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68339
> , in order to improve the overall Windows input method experience in Emacs.
>
> dANiuu zHaO <[email protected]> 于2026年7月30日周四 00:57写道：
>
>  This is the second patch.
>  I fixed the static buffer issue from yesterday by switching to on-demand allocation.
>  For w32term.c, I removed the direct execution of Lisp code and instead communicate by writing to inev's args, following the preedit handling
>  approach in xterm.c.
>  I kept the w32-ime-preedit flag to toggle the default IME window.
>  The hook-based approach has been completely removed. Everything now works through event reading.
>
>  dANiuu zHaO <[email protected]> 于2026年7月29日周三 22:33写道：
>
>  For the IME window font size setting, I will try to implement it, and once completed I will separate it into a standalone patch to facilitate
>  review.
>  However, what I currently prefer is having Emacs handle the rendering of preedit text, because I prefer the preedit style more.
>  That said, I also understand that this feature involves quite a lot of changes and will require a lengthy review and testing process. In
>  comparison, the font-size approach would be much easier to review and test.
>
>  I revised the patch based on yesterday's email feedback, including string allocation and other items. I made quite a few improvements to
>  the code. I expect to send the second patch in a few hours.
>
>  > I see that you added a hook, but I think we should discuss the use
>  > cases for such a hook.  Also, we generally avoid installing features
>  > on Windows that have no equivalents on GNU/Linux, so does Emacs have
>  > anything similar on X?
>
>  I took a look at other files under the src directory today and found that xterm.c, xfns.c, and their corresponding Lisp files all have preedit
>  handling. I removed the hook approach and switched to an event-reading approach instead. However, I still kept a Lisp flag, intending to
>  implement the ImmSetCompositionFont approach mentioned above. That way there can be two sets of IME display control, allowing users
>  to choose between the preedit style and the default IME window.
>
>  *  .\src\xfns.c (xic_preedit_caret_callback)
>  *  .\src\xterm.c (x_maybe_clear_preedit)
>  * .\lisp\term\x-win.el (x-preedit-text)
>
>  Based on the code in these files, I learned about `PREEDIT_TEXT_EVENT`. I'm glad — I think I now know how to write this properly.
>
>  Eli Zaretskii <[email protected]> 于2026年7月29日周三 20:11写道：
>
>  [Please always use Reply to All when replying, to keep the bug tracker CC'ed.]
>
>  > From: dANiuu zHaO <[email protected]>
>  > Date: Tue, 28 Jul 2026 23:56:48 +0800
>  > 
>  > Thanks for your review. Since this is my first time contributing code, there are indeed many areas where I can
>  > improve.
>  > 
>  > Let me first address the question about the purpose of the changes. The relevant bug report is:
>  > 
>  > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68339
>  > 
>  > Emacs on Windows already supports the IME window, but the IME preedit window does not scale with the font
>  > size in the buffer.  This report describes exactly this issue — when using Microsoft Pinyin IME to type Chinese
>  > on Windows, the font size in the preedit window is too small to be legible.  I also don't have a good approach
>  > for using ImmSetCompositionFont to address this.  So instead, I extended the WM_IME_COMPOSITION event
>  > to obtain the composition string from CJK input methods.  My idea is that after obtaining these strings, I can
>  > pass them to Elisp to render using overlays, rather than relying on the IME's own window to draw them.  This is
>  > a new feature.
>  > 
>  > This hook reads the composition string from the WM_IME_COMPOSITION event, and its sole purpose is this
>  > use case.
>
>  If this is for making the font of the IME window more like the size of
>  the default font, then why not use ImmSetCompositionFont, as suggested
>  by bug#68339?  I know that I said there it was not possible, but I
>  think I was wrong: using ImmGetCompositionFont, we could retrieve the
>  LOGFONT structure for the font the IME window uses, and then modify
>  just one member of the structure, lfHeight, by setting it to the
>  negative of FRAME_FONT (f)->pixel_size, and call
>  ImmSetCompositionFont.  AFAIU, this should be done when we receive the
>  WM_IME_STARTCOMPOSITION message, with the same context we use for
>  calling ImmSetCompositionWindow.  Can you try that?
>
>  > Regarding whether there is an equivalent implementation on GNU/Linux and X — I honestly don't know.  I do
>  > most of my work on Windows, so I'm not familiar with that.  I apologize for this.
>
>  If we use this for setting the font of the IME window, then we are
>  okay, because we already have a similar code in xfns.c for X.
>  However, providing a hook is much more general than this, so this is
>  one more reason not to go the hook way.
>
>  > As for the discussion from a few days ago about supporting IME in terminal (-nw) sessions — that is a difficult
>  > challenge.  I think it heavily depends on whether the terminal itself handles IME input properly.  I haven't even
>  > looked at Microsoft's documentation on this topic yet.  So I believe it is unrelated to our previous discussion.
>
>  Understood.
>
>  > Regarding the code comments and style — those oversights were my mistake, and I will fix them in the next
>  > patch.
>  > 
>  > The static buffer size limit is indeed too small.  I will switch to using MAX_ALLOCA or a similar approach.
>  > 
>  > > +/* Fixed buffer for receiving IME composition (preedit) string.
>  > > +   IME composition strings are typically short, so 64 characters
>  > > +   is sufficient for most cases. The buffer is sized to 2 * 64
>  > > +   bytes because each wide character (wchar_t) occupies 2 bytes
>  > > +   under UTF-16 encoding on Windows. */
>  > > +static char composition_ime_buf[2 * 64];
>  > 
>  > For the wchar_t buffer allocation, multiplying by 4 when converting to UTF-8 would indeed be better.  The IME
>  > composition strings produced by the input method I personally use do not exceed 2 bytes when converted to
>  > UTF-8, so I made that assumption and overlooked the fact that other CJK input methods may produce longer
>  > strings.
>  > 
>  > > +       wlen = WideCharToMultiByte (CP_UTF8, 0,
>  > > +                                   wbuf,
>  > > +                                   size / sizeof (wchar_t),
>  > > +                                   composition_ime_buf,
>  > > +                                   sizeof (composition_ime_buf),
>  > > +                                   NULL, NULL);
>  > 
>  > Regarding calling Lisp inside read_socket_hook — I also feel that calling Lisp there is somewhat inappropriate;
>  > it is a bit slow.  As for safety, I haven't encountered issues in my own local testing, but more testing is needed to
>  > verify.  So I would like to know: what is the recommended path within the project for notifying Lisp with data
>  > obtained from window events?  I will follow whatever the recommended approach is.
>  > 
>  > > +     case WM_IME_COMPOSITION:
>  > > +       {
>  > > +         int len = (int) msg.msg.lParam;
>  > > +         char *buf = (char *) msg.msg.wParam;
>  > > +         Lisp_Object str = make_string_from_utf8 (buf, len);
>  > > +         CALLN (Frun_hook_with_args, Qw32_ime_composition_hook, str);
>  > 
>  > Regarding the review of w32-ime-preedit and w32-ime-composition-hook — you are right that w32-ime-preedit
>  > is not needed.  We can simply check whether w32-ime-composition-hook is nil.  I didn't think of this at all when
>  > writing it.  I agree this is absolutely the right approach.
>  > 
>  > Finally, thank you for your review.  I plan to send a second patch in the next few days.  I would like to get this
>  > feature into Emacs — it will allow users on Windows to see the IME preedit window clearly without needing to
>  > install a third-party input method package.
>
>  Thanks.
>
>

Hello dANiuu and Eli,

I tested the latest version the patch on
https://github.com/zHaOdANiuu/emacs-w32-draw-preedit.patch

I works pretty well on emacs master branch, hope that this feature can
be merged into master branch and enabled by default, so that more user
can benefit from it.

dANiuu, Thank you for implementing this great feature on Windows Emacs
port. This feature greatly enhances the user experience of inputting
Chinese in Emacs


Best wishes,
-- 
Eason Huang