bug#81019: 31.0.50; [BUG] treesit: local parser keeps stale incremental tree across range moves

> On Aug 4, 2026, at 7:04=E2=80=AFPM, Rahul Martim Juliato =
<[email protected]> wrote:
>=20
> Yuan Fu <[email protected]> writes:
>=20
>>> On Jul 23, 2026, at 7:53=E2=80=AFAM, Rahul Martim Juliato =
<[email protected]> wrote:
>>>=20
>>> Eli Zaretskii <[email protected]> writes:
>>>=20
>>>>> From: St=C3=A9phane Marks <[email protected]>
>>>>> Date: Thu, 23 Jul 2026 14:12:10 +0200
>>>>> Cc: [email protected], [email protected], =
[email protected],=20
>>>>> [email protected]
>>>>>=20
>>>>> On Thu, Jul 23, 2026 at 8:10=E2=80=AFAM Eli Zaretskii =
<[email protected]> wrote:
>>>>>=20
>>>>>> From: St=C3=A9phane Marks <[email protected]>
>>>>>> Date: Thu, 23 Jul 2026 13:41:42 +0200
>>>>>> Cc: Rahul Martim Juliato <[email protected]>, =
[email protected], [email protected],=20
>>>>>>     [email protected]
>>>>>>=20
>>>>>> On Thu, Jul 23, 2026 at 3:14=E2=80=AFAM Eli Zaretskii =
<[email protected]> wrote:
>>>>>>=20
>>>>>>> From: Rahul Martim Juliato <[email protected]>
>>>>>>> Cc: Rahul Martim Juliato <[email protected]>,  Eli =
Zaretskii
>>>>>>> <[email protected]>,  [email protected],  [email protected],
>>>>>>> [email protected]
>>>>>>> Date: Tue, 21 Jul 2026 14:02:41 -0300
>>>>>>>=20
>>>>>>> St=C3=A9phane Marks <[email protected]> writes:
>>>>>>>=20
>>>>>>>> On Tue, Jul 21, 2026 at 10:39=E2=80=AFAM Rahul Martim Juliato <
>>>>>>>> [email protected]> wrote:
>>>>>>>>=20
>>>>>>>>> [...]
>>>>>>>>=20
>>>>>>>> My build uses tree-sitter/0.26.9 and removing the sit-for =
worked for the
>>>>>>>> test case I tried.  I think Yuan's forced reparse for =
markdown-inline does
>>>>>>>> the trick?
>>>>>>>>=20
>>>>>>>=20
>>>>>>> That kills my theory, thanks.  If it works on 0.26.9 then the =
library
>>>>>>> version has nothing to do with it, and my worry about older =
builds goes
>>>>>>> away with it.
>>>>>>>=20
>>>>>>> I do not think it is the forced reparse either though, at least =
not for
>>>>>>> the yank case.  Your June 11 commit changed the code block =
overlay:
>>>>>>>=20
>>>>>>> -      (let ((ov (make-overlay node-start node-end nil t nil)))
>>>>>>> +      (let ((ov (make-overlay node-start node-end nil nil t)))
>>>>>>>=20
>>>>>>> front-advance off, rear-advance on.  Yanking to extend a fenced =
block
>>>>>>> inserts at the rear, so with the old overlay the block did not =
grow, and
>>>>>>> markdown-ts-at-code-block-p, which reads those overlays, would =
not find
>>>>>>> one.  With rear-advance it grows on its own, no reparse and no =
redisplay
>>>>>>> needed.  The sit-for came along in that same commit.
>>>>>>>=20
>>>>>>> So your patch should go in, and I withdraw the hesitation from =
my last
>>>>>>> message.
>>>>>>=20
>>>>>> Thanks, but I'm afraid I'm now even more confused.  Is the =
version of
>>>>>> the Tree-sitter library (0.26.x vs older versions) an issue or =
isn't
>>>>>> it?
>>>>>>=20
>>>>>> And how does the presence of sit-for cause the problem here in =
the
>>>>>> first place?  (I know you tried to answer, but I don't think I
>>>>>> understand the answer, because it mainly described how the
>>>>>> introduction of sit-for happened historically, not why having it =
there
>>>>>> causes problems.)
>>>>>>=20
>>>>>> sit-for was a Band-Aid to work around an issue with treesit for =
which Yuan subsequently pushed a
>>>>> solution to
>>>>>> refresh embedded parsers.  Using sit-for forced a redisplay which =
in turn forced treesit to do what Yuan
>>>>> is now
>>>>>> doing automatically.  We propose removing the sit-for since it's =
now no longer necessary.
>>>>>=20
>>>>> You are saying that this bug report doesn't point to a real =
problem?
>>>>> IOW, will we have any problem if we do NOT remove the sit-for =
call?
>>>>>=20
>>>>> Just avoiding the performance implications of the redisplay.  =
Considering markdown-ts-mode is still considered
>>>>> "experimental," I think we should remove the sit-for.
>>>>=20
>>>> OK, but then what the Subject of this bug report is about?  Does
>>>> "local parser keep stale incremental tree across range moves" or
>>>> doesn't it?
>>>=20
>>> Not anymore. The issue has now been addressed in three places:
>>>=20
>>> * In the Tree-sitter library itself.
>>> * In `treesit.c` (Yuan added a workaround since we still support =
older
>>> Tree-sitter versions).
>>> * In `markdown-ts-mode`, via the `sit-for` workaround that we're now
>>> proposing to remove because the previous two are sufficient.
>>=20
>> Ok yeah, that matches my understanding. Tree-sitter fixed the bug, =
but
>> that fix is only in the latest version. I added a workaround fix to
>> emacs-31 so that we=E2=80=99re good for any tree-sitter version =
(maybe in the
>> future we can remove this). And IIRC Stephane or Rahul added the
>> sit-for workaround in the early on, which can be removed now.
>>=20
>> Yuan
>=20
> Since both commits are now in the Emacs 31 branch:
>=20
> * 9d1b0c2d7f6: Yuan Fu (2026-05-28), "Fix markdown-inline incremental
>  parsing issue" (bug#81019)
>=20
> * 296869994e4: St=C3=A9phane Marks (2026-07-19), "Remove 'sit-for' =
treesit
>  parser workaround" (bug#81019)
>=20
> I've verified that this resolves the issue I originally reported, so =
I'm
> closing this bug. If anyone can still reproduce the problem, it can be
> reopened.
>=20
> Thanks to everyone who helped investigate and fix it.

Especially you ;)

Yuan=