Re: Gsoc: Multithreading support inquiry

Michael,

AFAIK it is still an open issue. However, be warned that it's not an easy one (which is probably why it has been open for many years). For more information, you already found the correct place: ancient postings in the clisp-devel archives.
I believe lack of threads has been for some years the main feature that distinguishes CLISP from other Common Lisp implementations. Many packages like web servers presuppose the availability of threads (and I believe even SLIME uses them for some of its newer features).
With threads fully available (and robust), it would be easy to provide a CLISP backend for Bordeaux-Threads or one of the other meta-packages, which many other packages came to use as a compatibility layer.

What makes hash-tables with threads particularly challenging in CLISP IMHO?
 - Lock less is not everything. One also needs to consider:
 - Generally (as in Java), the interaction with the hash-table-iterator;
 - the suitability for CLISP's weak and semi-weak hash-tables;
 - the moving GC, whereas some other Lisp never move objects;
 - Then add threads on top of this.

I'll repeat myself, but often enough I've wondered whether CLISP specifically would not be better served by a combination of other approaches:
 - serve-fd (from cmucl) as an API to run Lisp code based on some unix event availability;
 - Some similar API to run Lisp code when a UNIX signal is received -- but run OUTSIDE the fragile context of the signal handler, somewhere in CLISP's virtual machine byte code interpreter loop;
 - Perhaps use libuv or something similar and dispatch I/O threads inside the CLISP bytecode interpreter loop (would require a rewrite of clisp I/O;
- or perhaps use libuv internally only for I/O but provide green threads dispatched by CLISP's bytecode interpreter at suitable places. 

Drawback: In the ancient times, CLISP' bytecode interpreter loop was heavily optimized. Every instruction you add to this loop causes a slowdown of everything. Is that really a good place to add some rare event handling and green threads dispatcher?
Other VM (for other languages) try to add such things at places than run less often, e.g. some solely switch at branch backwards instructions, so its outside of the main VM loop, yet occurs often enough that the responsiveness is good, except in long numeric code without backward jumps (e.g. loops). Others solely dispatch at function calls, so a long loop without subfunction calls prevents any other green thread from running...
IIRC, the MS-Windows check for Control-C or Control-Break in CLISP is run synchronously in the bytecode loop, but I may be wrong.

I'd add that Bruno has always been in favour of real threads, because it's the real thing, whereas I believe the green-threads approach is more amenable to success and more easily decomposed into distinct steps. Yet if you look, the general movement is away from green threads, for various reasons. E.g. Rust used them initially, then moved away, for reasons I forgot. Green threads has never been an official clisp project proposal.

Regards,
	Jörg

-----Original Message-----
From: Michael Buch <[email protected]> 
Sent: Tuesday, January 22, 2019 10:10 PM

Hi everyone,

I was browsing through some of the GNU project ideas from previous years and came across those of the CLisp project ( https://clisp.sourceforge.io/wanted.html). Specifically adding multithreading support to CLisp intrigued me due to my general interest in concurrent algorithms and data structures. I was wondering whether this project still needs work to be done and whether the lock-free hash table implementation remains one of the last missing pieces. I saw there were some discussions on this topic in previous mail threads but couldn't find out whether a working solution was provided or whether it is being worked on.

Another interesting, perhaps related avenue, is HTTP framework support for cl-http and AllegroServe mentioned in the release notes. It points to multithreading as the main blocker. Is there still use in providing support for these?

Regards,
Michael

_______________________________________________
clisp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/clisp-list

_______________________________________________
clisp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/clisp-list