Re: Parsec parser library...

Is having invocation history a bug or a feature?

I would expect it to make the closure every time you invoke the top level
parser in most cases, but it sounds like it stores it forever.

-- 
Martin Simmons
LispWorks Ltd
http://www.lispworks.com/



>>>>> On Sun, 26 Jul 2026 22:45:52 -0700, David McClain (as dbm at refined-audiometrics dot com) said:
> 
> I tracked down the programming style that led to thread-unsafe conditions. It is a very common programming idiom in Lisp and it begs some questions…
> 
> The Parseq library compiles PEG grammar rules into Let-Over-Lambda closures, where the LET var is used to track invocation history and detect the use of Left-Recursion. Parseq cannot do left recursion.
> 
> And there are good reasons for wanting Let-Over-Lambda:
> 
> (LET  ((STATE …))
>     (LAMBDA  (args…)
> 	….))
> 
> The LET bindings hold onto persistent state across invocations of the lambda closure. This is often a desirable thing to do. But as written, it only works in the face of single-threaded code. Those LET bindings are globally accessible and, if mutated, will cause race conditions, or worse, between multiple threads attempting to alter the persistent state.
> 
> So we need some kind of thread-local version of Let-Over-Lambda. In rare cases you might want the state to persist over all possible invocations. And in that case, go ahead and use LET-OVER-LAMBDA along with locking to serialize mutation among threads.
> 
> But just as often, you really don’t mean to have globally accessible state shared between threads, but rather as a history tracking device within one thread’s execution. And for that, LET-OVER-LAMBDA is a disaster in multi-threaded code.
> 
> In the case of the Parseq library, the solution is to keep state for each named rule in a dynamically bound hash-table, indexed by the name of the state. Each thread, on entry just rebinds a special var with a new hash-table for its own use. But this does not have the simple appearance of a LET-OVER-LAMBDA.
> 
> We need some kind of macrology to provide thread-safe LOL…
> 
> - DM 
> 
> 
> > On Jul 25, 2026, at 14:51, David McClain <[email protected]> wrote:
> > 
> > It looks like the Parsec library is not thread safe. 
> > 
> > I just ran into the most peculiar problem using it, wherein a formerly reliable number parsing system built with Parsec, claims to have detected Left-Recursion after about 2500 numbers were thrown at it. And this happens in each of several parallel threads all trying to parse numbers.
> > 
> > I can reliably parse literally millions of number strings with it, when executed from a single thread (the Editor, or the REPL). 
> > 
> > But I see erorrs happen when there are at least 4 parallel threads all doing the same kind of work - taking tabular entries from an incoming database, splitting each line at the delimiters, and then calling on READ-FROM-STRING to read the numbers contained in those strings.
> > 
> > The numbers are all just plain decimal numbers with fractions, like 310.1023344. Nothing unusual about them.
> > 
> > If I surround the calls to READ-FROM-STRING with a plain vanilla READTABLE, then no errors arise because it no longer uses my Parseq parser.
> > 
> > If I have just one thread using the Parseq parser, no problem. 
> > 
> > So I am led to conclude that something in the Parseq library is not thread-safe.
> > 
> > 
> 
> 

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
[email protected]
http://www.lispworks.com/support/lisp-hug.html