Re: folder naming conventions and installed version identifier, was: Maxima searching all "binary" directories: conflicts & performance degradation

On 4/19/26 2:58 AM, David Scherfgen wrote:

> I have just pushed a minimally-invasive fix for the problem.
> Files that were found within the binary directory, but not within the 
> subdirectories corresponding to the current combination of Maxima and 
> Lisp versions, are ignored.
> The same goes for files within the "test_directory_cached" directory 
> that's created during the cache initialization (for checking whether 
> Lisp and OS behave as expected).
> I kindly ask you to verify that Maxima still finds files it's expected 
> to find. I tested lots of cases on my machine, and it worked perfectly.

Thanks for the updates. Works fine, but then I wasn’t having the 
reported bug issues either.

In my .maxima dir, I have a few git repos checked out. With the debug 
variables enabled, I see that it looks at all the files in the .git 
directory. We should probably avoid that; there’s nothing there that we 
really want.

>
> Best regards
> David Scherfgen
>
> Am So., 19. Apr. 2026 um 07:43 Uhr schrieb David Scherfgen 
> <[email protected]>:
>
>     Forgot to reply to mailing list ...
>     /
>     /
>     /
>     > Raymond Toy <[email protected]>:/
>     /> FWIW, I have some fasls there. Not many, but not zero. But as a
>     dev, I know how to add *.fasl to the search path./
>
>     Good to know. Then we should definitely keep supporting .fasl
>     files in the user directory, in my opinion.
>
>     /> Raymond Toy <[email protected]>:/
>     /> An alternative is on startup to search ~/.maxima and explicitly
>     add all the directories there to the search path [...]/
>
>     The problem with that is of course that new directories created
>     while Maxima is running wouldn't be searched - not so nice.
>
>     At this point, I think the best way forward is to implement an
>     exclusion logic because even if we add a new .maxima-binary
>     top-level directory, users who upgrade their Maxima would still
>     have the old directory structure, and old/incompatible .fasl files
>     would be loaded from there when they're not found in the new
>     .maxima-binary directory.
>
>     I'll try to come up with a minimally messy solution.
>
>     Maybe we can print a warning when the number of different
>     version-specific folders exceeds a certain threshold, e.g. 10, so
>     that users are made aware of potential space/performance problems?
>
>     Best regards
>     David Scherfgen
>
>     Raymond Toy <[email protected]> schrieb am So., 19. Apr. 2026,
>     01:24:
>
>         On 4/18/26 10:40 AM, David Scherfgen via Maxima-discuss wrote:
>
>>         Hi Robert,
>>
>>         Thanks for the feedback.
>>
>>         I completely agree regarding the 5.nnnpost version
>>         identifiers — that’s an excellent point, and it absolutely
>>         complicates both bug reporting and the file search logic.
>>         However, I have practically no knowledge of that part of the
>>         Maxima codebase (like configure.ac <http://configure.ac> or
>>         the build scripts), so it would be much better if someone
>>         else could tackle that specific issue.
>         I can help with that, but I’m confused on what the proposal
>         is. My |~/.maxima/binary| has things like 5_47post and
>         branch_5_46_base_475_g7550340ea. So the proposal is that
>         5_47post should include more of the git version?
>>
>>         Regarding the directory structure for the binary files, your
>>         feedback brings up two technical hurdles we need to consider:
>>
>>         *1. Do users actually need $USER_DIR/**/*.fasl?*
>>
>>         Currently, $file_search_lisp includes the wildcard entry
>>         $USER_DIR/**/*.fasl (where $USER_DIR is the user's Maxima
>>         user directory and .fasl can be different for other Lisps).
>>         My question for you and the list is: do users really expect
>>         to be able to drop their own compiled Lisp files (.fasl)
>>         /anywhere/ in their user directory (not inside binary) and
>>         have Maxima find them?
>         FWIW, I have some fasls there. Not many, but not zero. But as
>         a dev, I know how to add *.fasl to the search path.
>>
>>         If we can agree that the answer is "no, they shouldn't," then
>>         this whole problem becomes incredibly simple to solve. We can
>>         just remove that wildcard and only add the correct binary
>>         directory to the search path.
>>
>>         If the answer is "yes, they do expect that," then keeping the
>>         binary folder inside the top-level user maxima directory
>>         becomes tricky. That wildcard will inevitably sweep through
>>         and find all the old, incompatible .fasl files that we are
>>         explicitly trying to avoid. We would have to write explicit
>>         exclusion logic: /If a matched file is located somewhere
>>         within the binary folder, explicitly check that its path
>>         matches the current version/Lisp combination; otherwise,
>>         reject it./ This adds messy logic, which was my primary
>>         motivation for thinking about a separate folder like
>>         maxima-internal.
>         An alternative is on startup to search |~/.maxima| and
>         explicitly add all the directories there to the search path,
>         excluding |~/.maxima/binary/|. We’d add
>         |~/.maxima/binary/version/| separately. So we’d have
>         |~/.maxima/dir1/**/|, |~/.maxima/dir2/**/| and so on.
>
>         You had also previously mentioned that we search directories
>         multiple times once each for |*.lisp|, |*.fasl|, |*.mac|,
>         |*.wxm|. To reduce this, we could kind of go back to the old
>         scheme, but maybe make each entry be something like
>         |[".maxima/**", ["mac, "wxm","lisp", "fasl"]]|. Then, we’ll do
>         a directory with |".maxima/**/*.*"| and then for each entry
>         see if the extension (type) matches one in the list. This is
>         kind of how the original file search paths were done. I didn’t
>         do this because directory searches seemed pretty fast for all
>         the Lisps I tested on.
>
>>
>>         *2. The OS Distinction & Flattening the Directory*
>>         *
>>         *Regarding baking in the OS name: I understand the concern,
>>         but adding the OS is essentially a one-line code change that
>>         shouldn't introduce new points of failure. A simple
>>         distinction between Windows, UNIX and unknown can be achieved
>>         by looking at common *feature* members.
>>
>>         To address the concern about messy folders and deep nesting,
>>         what if we flattened the structure instead of nesting it?
>>         Rather than having binary/5_47post/sbcl/2_6_3/win32, we could
>>         merge the identifiers into a single directory name:
>>         binary-5_47post-sbcl-2.6.3-windows. This keeps the directory
>>         tree shallow, completely prevents cross-platform/version
>>         interference, and requires very little effort to implement.
>>
>>         As a related side note, I actually suspect that the bug
>>         Wolfgang recently ran into
>>         <https://sourceforge.net/p/maxima/bugs/4713/> may be related
>>         to his cross-OS workflow. It is just a suspicion as of now —
>>         we haven't found the root of the issue yet — but it does
>>         highlight why having a built-in OS separation might actually
>>         save us from tricky edge-case bugs.
>>
>>         Let me know what you think about the user expectation
>>         regarding .fasl files — that seems to be the biggest
>>         technical blocker to keeping the binary files inside the
>>         standard user directory.
>>
>>         Best regards
>>         David
>>
>>         Am Sa., 18. Apr. 2026 um 18:54 Uhr schrieb Robert Dodier
>>         <[email protected]>:
>>
>>             David, thanks for working on this stuff. I agree it's a
>>             good idea to
>>             solve it somehow.
>>
>>             About searching only the binary folder for the current
>>             version -- agreed.
>>
>>             About creating another top-level folder -- I'm against
>>             it, I think the
>>             binary folder should go inside the top-level per-user
>>             maxima folder
>>             (same as it is now).
>>
>>             About baking the OS name into the binary path -- I'm
>>             against it, I
>>             don't think it's common enough to justify the effort,
>>             which is mostly
>>             going to be trying to fix bugs after the fact when users
>>             report it's
>>             not working for them.
>>
>>             It's a mess to have multiple folders for different
>>             combinations of
>>             Maxima + Lisp versions, but I don't think Maxima should
>>             try to clean
>>             that up; I think we should leave it for the users to deal
>>             with. I
>>             expect the vast majority of users are going to have one
>>             Maxima + Lisp
>>             combination for each installed version -- I think that's
>>             tolerable.
>>
>>             You didn't mention it, but I've come to the conclusion
>>             that installed
>>             versions of Maxima should only identify themselves via
>>             version-specific identifiers, as it is now with
>>             uninstalled versions
>>             (e.g. branch_5_49_base_392_g85c1954 as it is right now);
>>             identifying
>>             itself as "5.nnnpost", e.g. 5.49post, makes it impossible to
>>             distinguish different installed versions. That is a serious
>>             consequence for bug reporting, and also for the file
>>             search problem. I
>>             guess we would have to modify configure.ac
>>             <http://configure.ac> or something to fix that.
>>
>>             All the best,
>>
>>             Robert
>>
>>
>>
>>         _______________________________________________
>>         Maxima-discuss mailing list
>>         [email protected]
>>         https://lists.sourceforge.net/lists/listinfo/maxima-discuss
>         &#8203;
>         _______________________________________________
>         Maxima-discuss mailing list
>         [email protected]
>         https://lists.sourceforge.net/lists/listinfo/maxima-discuss
>
&#8203;

_______________________________________________
Maxima-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/maxima-discuss