bug#81548: 31.0.91; eglot extremely slow with large source files

TL;DR: The problem is clangd is extremely slow to calculate Semantic
Tokens for this very large file.  There's little Eglot can do about
this, except disabling semtok for this file, which is a suitable
workaround.  Clangd devs might be interested in profiling or explaining
(and confirming?) why clangd is irresponsive to other requests while
doing the relatively low-priority work of calculating semantic tokens.

>> From: Peter Wu <[email protected]>
>> Date: Mon, 3 Aug 2026 20:57:29 +0000

Hello Peter, thanks for reporting this.

First of all, a word on the reproduction recipe, which is excellent.  It
is is still longer than it needs to be.  Steps 7/8/9 can be use a
simpler way to create an utterly configless Emacs and launch eglot in a
file eglot'.  Avoid eglot-ensure and the opaque use-package macro.

7->9. /path/to/emacs -Q path/to/file.c -f eglot

>> 0. make sure you have clangd installed. I'm on macOS 26.6 and use the
>> system stock clangd: Apple clangd version 21.0.0 (clang-2100.1.1.101)
>> 1. git clone https://github.com/wolfSSL/wolfssl
>> 2. cd wolfssl
>> 3. ./autogen.sh (assuming you have the autotools installed)
>> 4. ./configure --enable-certgen \
>>             --enable-certreq \
>>             --enable-certext \
>>             --enable-crl \
>>             --enable-asn \
>>             --enable-opensslextra
>> 5. generate a compile_commands.json (I used: uvx compiledb make -j8)

Thank you for the ./configure trick, my compile_commands.json is now
fixed and I can open the file with just the "normal" error.  I didn't
need to make the whole project though . After 'configure' only this is
needed:

   bear -- make wolfcrypt/test/test.o

which is much faster (I suppose compiledb also works).  CMake doesn't
work for this project very well, but it does reproduce the specific bug.
So I reiterate the 5 liner here for future reference:

    clangd --version && emacs --version  # clangd 22.sth/emacs 32 for me
    git clone --depth=3D1 https://github.com/wolfSSL/wolfssl.git
    cd wolfssl
    cmake -B build -DCMAKE_EXPORT_COMPILE_COMMANDS=3Don
    path/to/emacs/master/emacs -Q wolfcrypt/test/test.c -f eglot
    ... and witness every synchronous request like find-definition hang
    for minutes.

Now, to the actual bug, it's the "semantic fontification", aka semtok,
which is a way for servers to say this bit here is a variable, this is a
function call, this is an operator, etc.

The reason you don't see this in Emacs 30.2 is that that version didn't
have semantic tokens.

Eglot turns semtok on by default in all buffers.  You can prevent that.
This is a workaround, if a bit heavy handed (becasue you lose semantic
tokens in all files)

  (add-hook 'eglot-managed-mode-hook
      (lambda ()
        (eglot-semantic-tokens-mode -1)))

The reason why this takes so long is:

  1. This file is almost 1M lines long
  2. It has lots of macros?

When it starts managing a file, one of the steps Eglot does is ask the
LSP server for tokens.  This file has tons of "tokens", and Eglot asks
for all of them upfront.  Although LSP has a mechanism for asking for
less tokens, known as textDocument/semanticTokens/range, it seems absent
from the 'clangd' server (and Eglot doesn't support it anymore, anyway,
I'd have to bring that code back which isn't simple).

Anyway, this "get tokens" is **asynchronous** and runs in the
background, which is why you can continue to scroll the buffer and do
other things like finding functions.

Anyway, clangd takes a very long time to calculate these tokens and gets
very busy. If you let it finish you can then probably continue working
normally.  If you do edits to the buffer, Eglot sends these to the LSP
server who has to recalculate tokens.  Oh no, you may despair, but in
that case Eglot asks only for a "delta" of tokens which is pretty fast.
Which is why after the first 45 seconds everything works smoothly for
you.

Anyway, if you do not decide to wait and at a certain point you decide
to find a definition that request is **synchronous**, which is why Emacs
will seem stuck if the request isn't answered.  Emacs/Eglot will wait
until the server answers.

Emacs's thread isn't doing any work, it's just down a pipe waiting for
input, but clangd's threads are very busy and for some reason the rest
of clangd doesn't remain responsive while this happens.  It lets all
other requests die.  Maybe clangd devs can be brought in.  I'm trying my
luck with Natan Ridge, aka HighCommander4, maybe he has insight.  You
may want to report the bug to https://github.com/clangd/clangd/issues.

The only thing I can't explain from Eglot's side is why the
xref-find-definitions sync request by Eglot doesn't time out after 10
seconds: that should be the default.  I.e. it shouldn't have an infinite
timeout, it should error after a while.

Jo=C3=A3o