Re: Structured Changelogs for ELPA packages

Philip Kaludercic <[email protected]> writes:

> Jean Louis <[email protected]> writes:
>
>> Hello Philip,
>>
>> I understand your request was about extracting, yet me I answered to
>> Andrew Hyatt, who said:
>>
>>> " I know this will be controversial, but if we want to maintain that
>>> changelogs are for humans and not machines, the solution is to use
>>> an LLM to produce a standardized changelog from the freeform
>>> changelog. I really don't see a downside to this; this kind of
>>> operation is pretty reliable. "
>>
>> So my message and testing was related to "producing standardized
>> changelog using the LLM" and I have demonstrated it is not
>> deterministic.
>
> My understanding, and Andrew can correct me here, is that he was
> thinking about using an LLM to take an arbitrary changelog/NEWS file and
> transform it into some standardized format like "Keep a Changelog".

Yes, exactly.  To make this super-explicit, here's an example using just
free software and open weight LLMs, and the `llm` package on ELPA.

(llm-chat (defconst ash/llm-ollama-gemma (make-llm-ollama :chat-model "gemma4:e2b"))
          (llm-make-chat-prompt 
           "A short history of the changes to this module:
Version 0.1 was the first version. Fixes to the roasting function were
made in 0.1.1. The issue was that the negative roasting values could be
obtained if there was just 1 roasting step. In Version 0.2, I made
several improvements so that now we can handle several different
roasting, subtypes, see variable `roaster-roast-type' for details."
           :context
           "Take the input and turn it into an org-list, with headings for major versions that have underneath them in the hierarchy, headings for minor versions.  Each heading has the version number, and a list of changes is below.  Example output:

* Version 1
- Initial version of the =foo= module.
** Version 1.1
- Fix typo in the =README.org=.
- Bugfix for command =foo-find-match=.
* Version 2
- Added the ability to do reverse computations.(llm-chat ash/llm-ollama-gemma
          (llm-make-chat-prompt 
           "A short history of the changes to this module:
Version 0.1 was the first version. Fixes to the roasting function were
made in 0.1.1. The issue was that the negative roasting values could be
obtained if there was just 1 roasting step. In Version 0.2, I made
several improvements so that now we can handle several different
roasting, subtypes, see variable `roaster-roast-type' for details."
           :context
           "Take the input and turn it into an org-list, with headings for major versions that have underneath them in the hierarchy, headings for minor versions.  Each heading has the version number, and a list of changes is below.  Example output:

* Version 1
- Initial version of the =foo= module.
** Version 1.1
- Fix typo in the =README.org=.
- Bugfix for command =foo-find-match=.
* Version 2
- Added the ability to do reverse computations.
- Created a tutorial with command =foo-tutorial=.
  "))
"* Version 0.1
- First version of the module.
** Version 0.1.1
- Fixes to the roasting function to prevent obtaining negative roasting values when only one roasting step was performed.

** Version 0.2
- Made several improvements to handle different roasting subtypes.
- Introduced the variable `roaster-roast-type` for details on roasting types."

- Created a tutorial with command =foo-tutorial=.
  "))

Running this produces the following output:

"* Version 0.1
- First version of the module.
** Version 0.1.1
- Fixes to the roasting function to prevent obtaining negative roasting values when only one roasting step was performed.

** Version 0.2
- Made several improvements to handle different roasting subtypes.
- Introduced the variable `roaster-roast-type` for details on roasting types."