bug#81019: 31.0.50; [BUG] treesit: local parser keeps stale incremental tree across range moves

Yuan Fu <[email protected]> writes:

>> On Jul 23, 2026, at 7:53=E2=80=AFAM, Rahul Martim Juliato <rahuljuliato@=
gmail.com> wrote:
>>=20
>> Eli Zaretskii <[email protected]> writes:
>>=20
>>>> From: St=C3=A9phane Marks <[email protected]>
>>>> Date: Thu, 23 Jul 2026 14:12:10 +0200
>>>> Cc: [email protected], [email protected], [email protected],=
=20
>>>> [email protected]
>>>>=20
>>>> On Thu, Jul 23, 2026 at 8:10=E2=80=AFAM Eli Zaretskii <[email protected]> w=
rote:
>>>>=20
>>>>> From: St=C3=A9phane Marks <[email protected]>
>>>>> Date: Thu, 23 Jul 2026 13:41:42 +0200
>>>>> Cc: Rahul Martim Juliato <[email protected]>, [email protected],=
 [email protected],=20
>>>>>      [email protected]
>>>>>=20
>>>>> On Thu, Jul 23, 2026 at 3:14=E2=80=AFAM Eli Zaretskii <[email protected]> =
wrote:
>>>>>=20
>>>>>> From: Rahul Martim Juliato <[email protected]>
>>>>>> Cc: Rahul Martim Juliato <[email protected]>,  Eli Zaretskii
>>>>>> <[email protected]>,  [email protected],  [email protected],
>>>>>>  [email protected]
>>>>>> Date: Tue, 21 Jul 2026 14:02:41 -0300
>>>>>>=20
>>>>>> St=C3=A9phane Marks <[email protected]> writes:
>>>>>>=20
>>>>>>> On Tue, Jul 21, 2026 at 10:39=E2=80=AFAM Rahul Martim Juliato <
>>>>>>> [email protected]> wrote:
>>>>>>>=20
>>>>>>>> [...]
>>>>>>>=20
>>>>>>> My build uses tree-sitter/0.26.9 and removing the sit-for worked fo=
r the
>>>>>>> test case I tried.  I think Yuan's forced reparse for markdown-inli=
ne does
>>>>>>> the trick?
>>>>>>>=20
>>>>>>=20
>>>>>> That kills my theory, thanks.  If it works on 0.26.9 then the library
>>>>>> version has nothing to do with it, and my worry about older builds g=
oes
>>>>>> away with it.
>>>>>>=20
>>>>>> I do not think it is the forced reparse either though, at least not =
for
>>>>>> the yank case.  Your June 11 commit changed the code block overlay:
>>>>>>=20
>>>>>>  -      (let ((ov (make-overlay node-start node-end nil t nil)))
>>>>>>  +      (let ((ov (make-overlay node-start node-end nil nil t)))
>>>>>>=20
>>>>>> front-advance off, rear-advance on.  Yanking to extend a fenced block
>>>>>> inserts at the rear, so with the old overlay the block did not grow,=
 and
>>>>>> markdown-ts-at-code-block-p, which reads those overlays, would not f=
ind
>>>>>> one.  With rear-advance it grows on its own, no reparse and no redis=
play
>>>>>> needed.  The sit-for came along in that same commit.
>>>>>>=20
>>>>>> So your patch should go in, and I withdraw the hesitation from my la=
st
>>>>>> message.
>>>>>=20
>>>>> Thanks, but I'm afraid I'm now even more confused.  Is the version of
>>>>> the Tree-sitter library (0.26.x vs older versions) an issue or isn't
>>>>> it?
>>>>>=20
>>>>> And how does the presence of sit-for cause the problem here in the
>>>>> first place?  (I know you tried to answer, but I don't think I
>>>>> understand the answer, because it mainly described how the
>>>>> introduction of sit-for happened historically, not why having it there
>>>>> causes problems.)
>>>>>=20
>>>>> sit-for was a Band-Aid to work around an issue with treesit for which=
 Yuan subsequently pushed a
>>>> solution to
>>>>> refresh embedded parsers.  Using sit-for forced a redisplay which in =
turn forced treesit to do what Yuan
>>>> is now
>>>>> doing automatically.  We propose removing the sit-for since it's now =
no longer necessary.
>>>>=20
>>>> You are saying that this bug report doesn't point to a real problem?
>>>> IOW, will we have any problem if we do NOT remove the sit-for call?
>>>>=20
>>>> Just avoiding the performance implications of the redisplay.  Consider=
ing markdown-ts-mode is still considered
>>>> "experimental," I think we should remove the sit-for.
>>>=20
>>> OK, but then what the Subject of this bug report is about?  Does
>>> "local parser keep stale incremental tree across range moves" or
>>> doesn't it?
>>=20
>> Not anymore. The issue has now been addressed in three places:
>>=20
>> * In the Tree-sitter library itself.
>> * In `treesit.c` (Yuan added a workaround since we still support older
>>  Tree-sitter versions).
>> * In `markdown-ts-mode`, via the `sit-for` workaround that we're now
>>  proposing to remove because the previous two are sufficient.
>
> Ok yeah, that matches my understanding. Tree-sitter fixed the bug, but
> that fix is only in the latest version. I added a workaround fix to
> emacs-31 so that we=E2=80=99re good for any tree-sitter version (maybe in=
 the
> future we can remove this). And IIRC Stephane or Rahul added the
> sit-for workaround in the early on, which can be removed now.
>
> Yuan

Since both commits are now in the Emacs 31 branch:

* 9d1b0c2d7f6: Yuan Fu (2026-05-28), "Fix markdown-inline incremental
  parsing issue" (bug#81019)
=20=20
* 296869994e4: St=C3=A9phane Marks (2026-07-19), "Remove 'sit-for' treesit
  parser workaround" (bug#81019)

I've verified that this resolves the issue I originally reported, so I'm
closing this bug. If anyone can still reproduce the problem, it can be
reopened.

Thanks to everyone who helped investigate and fix it.

--=20
Rahul Martim Juliato