Re: Amazing work on bug fixing

Allow me to also express appreciation for your debugging efforts, David, 
but also for the manner in which you use an LLM: Not as an "agent" but 
as a capable assistant, retaining control over the process. I have been 
using LLMs in a similar manner in the past three and a half years 
(through the APIs, mostly, less often through the Web) and it's been an 
incredible productivity booster. Agentic use, "vibe programming," can 
produce unpleasant surprises.


Viktor




On 2026-06-19 12:49, David Scherfgen via Maxima-discuss wrote:
> If the affected module's source code file is less than ~100 KB, I 
> don't put too much work into filtering and paste it completely.
>
> Gemini (and other comparable LLMs) already have a decent "baked-in" 
> understanding of the basic functions like SIMPLIFYA, supposedly 
> because lots of publicly available discussions (mailing list, big 
> reports, the code itself) were contained in the training data. You can 
> try asking your model about how exactly SIMPLIFYA works and what its 
> arguments X and Y mean, and instruct it not to use any external 
> sources. The answer should be quite good.
>
> I will do some more tests regarding pasting code into the prompt and 
> uploading it as an attachment.
>
> When I asked Gemini itself about this question (paste vs. upload), it 
> told me that a hybrid approach may be best. Upload the (zipped) 
> complete codebase, but also include highly relevant functions directly 
> in the prompt to steer the model's attention towards them.
>
> All this is really exciting!
>
>
> Stavros Macrakis <[email protected]> schrieb am Fr., 19. Juni 2026, 
> 18:15:
>
>     Thanks again!
>
>     I do have a paid subscription, and am able to set 3.1 Pro /
>     Extended in the UI.
>
>     It seems that you manually select the relevant sections of
>     Maxima's source code. How big a context do you include -- dozens
>     of functions, hundreds of lines? Presumably the source
>     "background" functions like add/simplus/simplifya/... isn't needed
>     in many cases, because it guesses at the functionality based on
>     the function name?
>
>     I have tried command-line Claude on whole directories (smaller
>     than Maxima), and it seems to do a pretty good job of
>     "understanding" what the code is trying to do and even
>     spontaneously noticing bugs (without bug reports). And then I can
>     ask about particular bits of functionality without specifying
>     which source file implements it. Is your experience that giving it
>     too much context consumes too many tokens, or just doesn't work well?
>
>               -s
>
>     On Fri, Jun 19, 2026 at 11:53 AM David Scherfgen
>     <[email protected]> wrote:
>
>         I use the web version. The model can be chosen in the
>         drop-down menu. I have a paid subscription. It's possible that
>         without, you can't effectively use Extended Thinking because
>         it will quickly reach your allowed limit.
>
>         You can upload files and attach them to the conversation, but
>         I found that it gives better results when pasting the source
>         code directly into the prompt. I'll have to test that again,
>         though.
>
>         Stavros Macrakis <[email protected]> schrieb am Fr., 19. Juni
>         2026, 17:22:
>
>             Thank you very much! That is a lot of great detail.
>
>             Do you use Gemini on the command line, the Web interface
>             (More uploads / Import code directly from Sourceforge, or
>             local git repository?), or via some IDE (Google AI
>             Studio?)? Do you have a GEMINI.md file? Is that where you
>             configure Extended Thinking?
>
>             Thanks again!!
>
>                           -s
>
>             On Fri, Jun 19, 2026 at 11:15 AM David Scherfgen
>             <[email protected]> wrote:
>
>                 Thanks, Stavros!
>
>                 When the "Extended Thinking" mode of Gemini 3.1 Pro
>                 was launched, I decided to give it a try, after
>                 previous attempts with Gemini 3.1 Pro in normal mode
>                 as well as ChatGPT and Claude weren't very successful.
>                 Extended Thinking is really a breakthrough.
>
>                 I usually write the prompt as follows:
>
>                     We're debugging the Maxima Computer Algebra System
>                     written in Lisp. There is a bug in the /[module
>                     name]/ module that can be reproduced as follows:
>                     ----------
>                     /[Code example]/
>                     ----------
>                     As you can see, /[actual behavior]/, while we
>                     would expect /[correct behavior]/.
>
>                     /[Any notable observations, e.g. similar cases
>                     that work correctly]/
>
>                     ----------
>                     Here's a trace:
>                     /[Trace of important functions or stack trace in
>                     case of a Lisp error]/
>                     ----------
>                     Your task is to thoroughly investigate this
>                     problem, explain in detail every step that causes
>                     it, and find a fix for it. Ideally, the fix should
>                     be minimally invasive.
>                     ----------
>                     Relevant sections of Maxima's source code:
>                     ----------
>                     /[Source code]/
>
>                 When pasting source code:
>
>                   * Strip away everything that's clearly not involved
>                     in the problem.
>                   * Ideally, include every function that gets called
>                     during the reproduction of the bug.
>                   * Be patient, as pasting into Gemini's prompt
>                     textbox can sometimes take a lot of time (the
>                     browser may even ask you if you want to wait or
>                     close the page - wait).
>
>                 Now Gemini will think for a while.
>                 When it comes back with an answer:
>
>                   * Make sure that the selected model is still "Gemini
>                     3.1 Pro (Extended)". It eats your quota really
>                     quickly (there's a limit that resets every 5 hours
>                     + a weekly limit), and then it will fall back to a
>                     simpler model, which almost never comes up with
>                     anything useful. You can also notice by how
>                     quickly it responds.
>                   * Be *extremely critical*. LLMs are incredibly good
>                     at selling, they will make it sound like it's 100%
>                     sure that they're right.
>                   * If you find the answer convincing, test the
>                     proposed fix against the problem and the test suite.
>
>                 If it's an obvious case where you can clearly see that
>                 the fix is correct, you're done.
>                 If it's a more difficult case, let Gemini regenerate
>                 the answer a few times to help you decide. See if it
>                 always finds the same fix or different ones.
>
>                 In case you get mostly incorrect fixes:
>
>                   * *Don't continue the conversation for too long*, as
>                     it will eventually forget the first part of it,
>                     which contains the highly important source code.
>                     LLMs have a limited context window. If the
>                     conversation gets too long, it starts forgetting
>                     things said earlier.
>                   * Instead, edit your initial prompt and include a
>                     section like: "You previously proposed the
>                     following fixes, which didn't work: [...]".
>                     Include a short description for every proposed fix
>                     and why it wasn't working (e.g. it broke certain
>                     tests). If editing the initial prompt is not
>                     possible because you have sent follow-up prompts
>                     already, you can "fork" the conversation after the
>                     first reply, and then you will be able to edit the
>                     initial prompt in the forked conversation.
>
>                 I hope this helps!
>
>                 Best regards
>                 David Scherfgen
>
>                 Am Fr., 19. Juni 2026 um 16:19 Uhr schrieb Stavros
>                 Macrakis <[email protected]>:
>
>                     David,
>
>                     Thank you very much for contributing so many bug
>                     fixes to Maxima!
>
>                     I think you've mentioned that you use an LLM to
>                     help you out. Could you tell us more about your
>                     setup and what you've learned about working
>                     effectively with it?
>
>                     Thanks!
>
>                           -s
>
>
>
> _______________________________________________
> Maxima-discuss mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/maxima-discuss

_______________________________________________
Maxima-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/maxima-discuss