Re: folder naming conventions and installed version identifier, was: Maxima searching all "binary" directories: conflicts & performance degradation

Forgot to reply to mailing list ...

*> Raymond Toy <[email protected] <[email protected]>>:*
*> FWIW, I have some fasls there. Not many, but not zero. But as a dev, I
know how to add *.fasl to the search path.*

Good to know. Then we should definitely keep supporting .fasl files in the
user directory, in my opinion.

*> Raymond Toy <[email protected] <[email protected]>>:*
*> An alternative is on startup to search ~/.maxima and explicitly add all
the directories there to the search path [...]*

The problem with that is of course that new directories created while
Maxima is running wouldn't be searched - not so nice.

At this point, I think the best way forward is to implement an exclusion
logic because even if we add a new .maxima-binary top-level directory,
users who upgrade their Maxima would still have the old directory
structure, and old/incompatible .fasl files would be loaded from there when
they're not found in the new .maxima-binary directory.

I'll try to come up with a minimally messy solution.

Maybe we can print a warning when the number of different version-specific
folders exceeds a certain threshold, e.g. 10, so that users are made aware
of potential space/performance problems?

Best regards
David Scherfgen

Raymond Toy <[email protected]> schrieb am So., 19. Apr. 2026, 01:24:

> On 4/18/26 10:40 AM, David Scherfgen via Maxima-discuss wrote:
>
> Hi Robert,
>
> Thanks for the feedback.
>
> I completely agree regarding the 5.nnnpost version identifiers — that’s
> an excellent point, and it absolutely complicates both bug reporting and
> the file search logic. However, I have practically no knowledge of that
> part of the Maxima codebase (like configure.ac or the build scripts), so
> it would be much better if someone else could tackle that specific issue.
>
> I can help with that, but I’m confused on what the proposal is. My
> ~/.maxima/binary has things like 5_47post and
> branch_5_46_base_475_g7550340ea. So the proposal is that 5_47post should
> include more of the git version?
>
>
> Regarding the directory structure for the binary files, your feedback
> brings up two technical hurdles we need to consider:
>
> *1. Do users actually need $USER_DIR/**/*.fasl?*
>
> Currently, $file_search_lisp includes the wildcard entry
> $USER_DIR/**/*.fasl (where $USER_DIR is the user's Maxima user directory
> and .fasl can be different for other Lisps). My question for you and the
> list is: do users really expect to be able to drop their own compiled Lisp
> files (.fasl) *anywhere* in their user directory (not inside binary) and
> have Maxima find them?
>
> FWIW, I have some fasls there. Not many, but not zero. But as a dev, I
> know how to add *.fasl to the search path.
>
>
> If we can agree that the answer is "no, they shouldn't," then this whole
> problem becomes incredibly simple to solve. We can just remove that
> wildcard and only add the correct binary directory to the search path.
>
> If the answer is "yes, they do expect that," then keeping the binary
> folder inside the top-level user maxima directory becomes tricky. That
> wildcard will inevitably sweep through and find all the old, incompatible
> .fasl files that we are explicitly trying to avoid. We would have to
> write explicit exclusion logic: *If a matched file is located somewhere
> within the binary folder, explicitly check that its path matches the
> current version/Lisp combination; otherwise, reject it.* This adds messy
> logic, which was my primary motivation for thinking about a separate folder
> like maxima-internal.
>
> An alternative is on startup to search ~/.maxima and explicitly add all
> the directories there to the search path, excluding ~/.maxima/binary/.
> We’d add ~/.maxima/binary/version/ separately. So we’d have
> ~/.maxima/dir1/**/, ~/.maxima/dir2/**/ and so on.
>
> You had also previously mentioned that we search directories multiple
> times once each for *.lisp, *.fasl, *.mac, *.wxm. To reduce this, we
> could kind of go back to the old scheme, but maybe make each entry be
> something like [".maxima/**", ["mac, "wxm","lisp", "fasl"]]. Then, we’ll
> do a directory with ".maxima/**/*.*" and then for each entry see if the
> extension (type) matches one in the list. This is kind of how the original
> file search paths were done. I didn’t do this because directory searches
> seemed pretty fast for all the Lisps I tested on.
>
>
> *2. The OS Distinction & Flattening the Directory*
>
> Regarding baking in the OS name: I understand the concern, but adding the
> OS is essentially a one-line code change that shouldn't introduce new
> points of failure. A simple distinction between Windows, UNIX and unknown
> can be achieved by looking at common *feature* members.
>
> To address the concern about messy folders and deep nesting, what if we
> flattened the structure instead of nesting it? Rather than having
> binary/5_47post/sbcl/2_6_3/win32, we could merge the identifiers into a
> single directory name: binary-5_47post-sbcl-2.6.3-windows. This keeps the
> directory tree shallow, completely prevents cross-platform/version
> interference, and requires very little effort to implement.
>
> As a related side note, I actually suspect that the bug Wolfgang recently
> ran into <https://sourceforge.net/p/maxima/bugs/4713/> may be related to
> his cross-OS workflow. It is just a suspicion as of now — we haven't found
> the root of the issue yet — but it does highlight why having a built-in OS
> separation might actually save us from tricky edge-case bugs.
>
> Let me know what you think about the user expectation regarding .fasl
> files — that seems to be the biggest technical blocker to keeping the
> binary files inside the standard user directory.
>
> Best regards
> David
>
> Am Sa., 18. Apr. 2026 um 18:54 Uhr schrieb Robert Dodier <
> [email protected]>:
>
>> David, thanks for working on this stuff. I agree it's a good idea to
>> solve it somehow.
>>
>> About searching only the binary folder for the current version -- agreed.
>>
>> About creating another top-level folder -- I'm against it, I think the
>> binary folder should go inside the top-level per-user maxima folder
>> (same as it is now).
>>
>> About baking the OS name into the binary path -- I'm against it, I
>> don't think it's common enough to justify the effort, which is mostly
>> going to be trying to fix bugs after the fact when users report it's
>> not working for them.
>>
>> It's a mess to have multiple folders for different combinations of
>> Maxima + Lisp versions, but I don't think Maxima should try to clean
>> that up; I think we should leave it for the users to deal with. I
>> expect the vast majority of users are going to have one Maxima + Lisp
>> combination for each installed version -- I think that's tolerable.
>>
>> You didn't mention it, but I've come to the conclusion that installed
>> versions of Maxima should only identify themselves via
>> version-specific identifiers, as it is now with uninstalled versions
>> (e.g. branch_5_49_base_392_g85c1954 as it is right now); identifying
>> itself as "5.nnnpost", e.g. 5.49post, makes it impossible to
>> distinguish different installed versions. That is a serious
>> consequence for bug reporting, and also for the file search problem. I
>> guess we would have to modify configure.ac or something to fix that.
>>
>> All the best,
>>
>> Robert
>>
>
>
> _______________________________________________
> Maxima-discuss mailing [email protected]://lists.sourceforge.net/lists/listinfo/maxima-discuss
>
> &#8203;
> _______________________________________________
> Maxima-discuss mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/maxima-discuss
>

_______________________________________________
Maxima-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/maxima-discuss