Re: Amazing work on bug fixing

Thanks, Stavros!

When the "Extended Thinking" mode of Gemini 3.1 Pro was launched, I decided
to give it a try, after previous attempts with Gemini 3.1 Pro in normal
mode as well as ChatGPT and Claude weren't very successful.
Extended Thinking is really a breakthrough.

I usually write the prompt as follows:

We're debugging the Maxima Computer Algebra System written in Lisp. There
> is a bug in the *[module name]* module that can be reproduced as follows:
> ----------
> *[Code example]*
> ----------
> As you can see, *[actual behavior]*, while we would expect *[correct
> behavior]*.
>
*[Any notable observations, e.g. similar cases that work correctly]*

----------
> Here's a trace:
> *[Trace of important functions or stack trace in case of a Lisp error]*
> ----------
> Your task is to thoroughly investigate this problem, explain in detail
> every step that causes it, and find a fix for it. Ideally, the fix should
> be minimally invasive.
> ----------
> Relevant sections of Maxima's source code:
> ----------
> *[Source code]*
>

When pasting source code:

   - Strip away everything that's clearly not involved in the problem.
   - Ideally, include every function that gets called during the
   reproduction of the bug.
   - Be patient, as pasting into Gemini's prompt textbox can sometimes take
   a lot of time (the browser may even ask you if you want to wait or close
   the page - wait).

Now Gemini will think for a while.
When it comes back with an answer:

   - Make sure that the selected model is still "Gemini 3.1 Pro
   (Extended)". It eats your quota really quickly (there's a limit that resets
   every 5 hours + a weekly limit), and then it will fall back to a simpler
   model, which almost never comes up with anything useful. You can also
   notice by how quickly it responds.
   - Be *extremely critical*. LLMs are incredibly good at selling, they
   will make it sound like it's 100% sure that they're right.
   - If you find the answer convincing, test the proposed fix against the
   problem and the test suite.

If it's an obvious case where you can clearly see that the fix is correct,
you're done.
If it's a more difficult case, let Gemini regenerate the answer a few times
to help you decide. See if it always finds the same fix or different ones.

In case you get mostly incorrect fixes:

   - *Don't continue the conversation for too long*, as it will eventually
   forget the first part of it, which contains the highly important source
   code. LLMs have a limited context window. If the conversation gets too
   long, it starts forgetting things said earlier.
   - Instead, edit your initial prompt and include a section like: "You
   previously proposed the following fixes, which didn't work: [...]". Include
   a short description for every proposed fix and why it wasn't working (e.g.
   it broke certain tests). If editing the initial prompt is not possible
   because you have sent follow-up prompts already, you can "fork" the
   conversation after the first reply, and then you will be able to edit the
   initial prompt in the forked conversation.

I hope this helps!

Best regards
David Scherfgen

Am Fr., 19. Juni 2026 um 16:19 Uhr schrieb Stavros Macrakis <
[email protected]>:

> David,
>
> Thank you very much for contributing so many bug fixes to Maxima!
>
> I think you've mentioned that you use an LLM to help you out. Could you
> tell us more about your setup and what you've learned about working
> effectively with it?
>
> Thanks!
>
>           -s
>
>

_______________________________________________
Maxima-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/maxima-discuss