Re: folder naming conventions and installed version identifier, was: Maxima searching all "binary" directories: conflicts & performance degradation

Good point. Should all folders whose names begin with a dot be ignored in
search, or only typically ones such as .git, .svn, .hg, .bzr, .vs, .vscode,
.idea, .xcode? Maybe introduce a list like $file_search_ignore that is
pre-populated to said folder names using wildcard paths?

Best regards
David Scherfgen

Raymond Toy <[email protected]> schrieb am So., 19. Apr. 2026, 22:19:

> On 4/19/26 2:58 AM, David Scherfgen wrote:
>
> I have just pushed a minimally-invasive fix for the problem.
> Files that were found within the binary directory, but not within the
> subdirectories corresponding to the current combination of Maxima and Lisp
> versions, are ignored.
> The same goes for files within the "test_directory_cached" directory
> that's created during the cache initialization (for checking whether Lisp
> and OS behave as expected).
> I kindly ask you to verify that Maxima still finds files it's expected to
> find. I tested lots of cases on my machine, and it worked perfectly.
>
> Thanks for the updates. Works fine, but then I wasn’t having the reported
> bug issues either.
>
> In my .maxima dir, I have a few git repos checked out. With the debug
> variables enabled, I see that it looks at all the files in the .git
> directory. We should probably avoid that; there’s nothing there that we
> really want.
>
>
> Best regards
> David Scherfgen
>
> Am So., 19. Apr. 2026 um 07:43 Uhr schrieb David Scherfgen <
> [email protected]>:
>
>> Forgot to reply to mailing list ...
>>
>> * > Raymond Toy <[email protected] <[email protected]>>:*
>> *> FWIW, I have some fasls there. Not many, but not zero. But as a dev, I
>> know how to add *.fasl to the search path.*
>>
>> Good to know. Then we should definitely keep supporting .fasl files in
>> the user directory, in my opinion.
>>
>> *> Raymond Toy <[email protected] <[email protected]>>:*
>> *> An alternative is on startup to search ~/.maxima and explicitly add
>> all the directories there to the search path [...]*
>>
>> The problem with that is of course that new directories created while
>> Maxima is running wouldn't be searched - not so nice.
>>
>> At this point, I think the best way forward is to implement an exclusion
>> logic because even if we add a new .maxima-binary top-level directory,
>> users who upgrade their Maxima would still have the old directory
>> structure, and old/incompatible .fasl files would be loaded from there when
>> they're not found in the new .maxima-binary directory.
>>
>> I'll try to come up with a minimally messy solution.
>>
>> Maybe we can print a warning when the number of different
>> version-specific folders exceeds a certain threshold, e.g. 10, so that
>> users are made aware of potential space/performance problems?
>>
>> Best regards
>> David Scherfgen
>>
>> Raymond Toy <[email protected]> schrieb am So., 19. Apr. 2026, 01:24:
>>
>>> On 4/18/26 10:40 AM, David Scherfgen via Maxima-discuss wrote:
>>>
>>> Hi Robert,
>>>
>>> Thanks for the feedback.
>>>
>>> I completely agree regarding the 5.nnnpost version identifiers — that’s
>>> an excellent point, and it absolutely complicates both bug reporting and
>>> the file search logic. However, I have practically no knowledge of that
>>> part of the Maxima codebase (like configure.ac or the build scripts),
>>> so it would be much better if someone else could tackle that specific issue.
>>>
>>> I can help with that, but I’m confused on what the proposal is. My
>>> ~/.maxima/binary has things like 5_47post and
>>> branch_5_46_base_475_g7550340ea. So the proposal is that 5_47post should
>>> include more of the git version?
>>>
>>>
>>> Regarding the directory structure for the binary files, your feedback
>>> brings up two technical hurdles we need to consider:
>>>
>>> *1. Do users actually need $USER_DIR/**/*.fasl?*
>>>
>>> Currently, $file_search_lisp includes the wildcard entry
>>> $USER_DIR/**/*.fasl (where $USER_DIR is the user's Maxima user
>>> directory and .fasl can be different for other Lisps). My question for
>>> you and the list is: do users really expect to be able to drop their own
>>> compiled Lisp files (.fasl) *anywhere* in their user directory (not
>>> inside binary) and have Maxima find them?
>>>
>>> FWIW, I have some fasls there. Not many, but not zero. But as a dev, I
>>> know how to add *.fasl to the search path.
>>>
>>>
>>> If we can agree that the answer is "no, they shouldn't," then this whole
>>> problem becomes incredibly simple to solve. We can just remove that
>>> wildcard and only add the correct binary directory to the search path.
>>>
>>> If the answer is "yes, they do expect that," then keeping the binary
>>> folder inside the top-level user maxima directory becomes tricky. That
>>> wildcard will inevitably sweep through and find all the old, incompatible
>>> .fasl files that we are explicitly trying to avoid. We would have to
>>> write explicit exclusion logic: *If a matched file is located somewhere
>>> within the binary folder, explicitly check that its path matches the
>>> current version/Lisp combination; otherwise, reject it.* This adds
>>> messy logic, which was my primary motivation for thinking about a separate
>>> folder like maxima-internal.
>>>
>>> An alternative is on startup to search ~/.maxima and explicitly add all
>>> the directories there to the search path, excluding ~/.maxima/binary/.
>>> We’d add ~/.maxima/binary/version/ separately. So we’d have
>>> ~/.maxima/dir1/**/, ~/.maxima/dir2/**/ and so on.
>>>
>>> You had also previously mentioned that we search directories multiple
>>> times once each for *.lisp, *.fasl, *.mac, *.wxm. To reduce this, we
>>> could kind of go back to the old scheme, but maybe make each entry be
>>> something like [".maxima/**", ["mac, "wxm","lisp", "fasl"]]. Then,
>>> we’ll do a directory with ".maxima/**/*.*" and then for each entry see
>>> if the extension (type) matches one in the list. This is kind of how the
>>> original file search paths were done. I didn’t do this because directory
>>> searches seemed pretty fast for all the Lisps I tested on.
>>>
>>>
>>> *2. The OS Distinction & Flattening the Directory*
>>>
>>> Regarding baking in the OS name: I understand the concern, but adding
>>> the OS is essentially a one-line code change that shouldn't introduce new
>>> points of failure. A simple distinction between Windows, UNIX and unknown
>>> can be achieved by looking at common *feature* members.
>>>
>>> To address the concern about messy folders and deep nesting, what if we
>>> flattened the structure instead of nesting it? Rather than having
>>> binary/5_47post/sbcl/2_6_3/win32, we could merge the identifiers into a
>>> single directory name: binary-5_47post-sbcl-2.6.3-windows. This keeps
>>> the directory tree shallow, completely prevents cross-platform/version
>>> interference, and requires very little effort to implement.
>>>
>>> As a related side note, I actually suspect that the bug Wolfgang
>>> recently ran into <https://sourceforge.net/p/maxima/bugs/4713/> may be
>>> related to his cross-OS workflow. It is just a suspicion as of now — we
>>> haven't found the root of the issue yet — but it does highlight why having
>>> a built-in OS separation might actually save us from tricky edge-case bugs.
>>>
>>> Let me know what you think about the user expectation regarding .fasl
>>> files — that seems to be the biggest technical blocker to keeping the
>>> binary files inside the standard user directory.
>>>
>>> Best regards
>>> David
>>>
>>> Am Sa., 18. Apr. 2026 um 18:54 Uhr schrieb Robert Dodier <
>>> [email protected]>:
>>>
>>>> David, thanks for working on this stuff. I agree it's a good idea to
>>>> solve it somehow.
>>>>
>>>> About searching only the binary folder for the current version --
>>>> agreed.
>>>>
>>>> About creating another top-level folder -- I'm against it, I think the
>>>> binary folder should go inside the top-level per-user maxima folder
>>>> (same as it is now).
>>>>
>>>> About baking the OS name into the binary path -- I'm against it, I
>>>> don't think it's common enough to justify the effort, which is mostly
>>>> going to be trying to fix bugs after the fact when users report it's
>>>> not working for them.
>>>>
>>>> It's a mess to have multiple folders for different combinations of
>>>> Maxima + Lisp versions, but I don't think Maxima should try to clean
>>>> that up; I think we should leave it for the users to deal with. I
>>>> expect the vast majority of users are going to have one Maxima + Lisp
>>>> combination for each installed version -- I think that's tolerable.
>>>>
>>>> You didn't mention it, but I've come to the conclusion that installed
>>>> versions of Maxima should only identify themselves via
>>>> version-specific identifiers, as it is now with uninstalled versions
>>>> (e.g. branch_5_49_base_392_g85c1954 as it is right now); identifying
>>>> itself as "5.nnnpost", e.g. 5.49post, makes it impossible to
>>>> distinguish different installed versions. That is a serious
>>>> consequence for bug reporting, and also for the file search problem. I
>>>> guess we would have to modify configure.ac or something to fix that.
>>>>
>>>> All the best,
>>>>
>>>> Robert
>>>>
>>>
>>>
>>> _______________________________________________
>>> Maxima-discuss mailing [email protected]://lists.sourceforge.net/lists/listinfo/maxima-discuss
>>>
>>> &#8203;
>>> _______________________________________________
>>> Maxima-discuss mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/maxima-discuss
>>>
>> &#8203;
>

_______________________________________________
Maxima-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/maxima-discuss