Re: handling state the database way

Hi Samer,

Considering the worthing, I think you wanted to send this to the list.
Re-included the list.  Hope that is ok.

On 02/20/2014 10:34 PM, Abdallah, Samer wrote:
> See below�
>
> On 17 Feb 2014, at 09:54, Jan Wielemaker <[email protected]> wrote:
>
>>
>> It is certainly a good way to explore a way to solve this mess.  There
>> are quite a few dimensions that do make sense though:
>>
>>   - Sharing between threads
>>   - Thread safety (for shared state)
>>   - Name space and scoping
>>   - Identity (sharing of variables)
>>
>> Except records vs. dynamic predicates, all mechanisms have their
>> own properties (could be formulated more cleanly).
>>
>>   - dynamic predicates
>>     Shared between threads, module-based name scoping, multiple
>>     clauses (can be interpreted in several ways).  Both store
>>     and load copy the value.
>>   - thread local predicates
>>     As above, but not shared between threads.
>>   - Non-backtrackable global variables
>>     Global name space (bug, should be module), storing copies,
>>     load shares.  One value per key, thread local.
>>   - Backtrackable global variables
>>     lobal name space (bug, should be module), both store and
>>     load shares.  One value per key, thread local, backtracks.
>>   - Non-backtrackable destructive assignment
>>     Local identity, otherwise as Non-backtrackable global variables.
>>     Have to be careful with term identity.
>>   - Backtrackable destructive assignment
>>     Local identity, otherwise as Backtrackable global variables.
>>     Have to be careful with term identity.
>>   - Prolog flags
>>     Global name space, copy on store and load, newly created
>>     threads copy from creator (copy-on-write implementation).
>>
>
> Thinking about Jan's comment a bit more, I have a few observations:
>
> 1. normal dynamic predicates are, I think, too entrenched in Prolog
>      to change. The rest, perhaps, are up grabs, but where to wield the knife�

I write a CICLOPS paper argueing for transactions to get cleaner update
semantics in multi-threaded environments.  Still should implement that.
Otherwise, I indeed think we should not change anything here.

> 2. Thread local vs shared is a useful distinction that could perhaps be
>      broadened to include global variables, however they turn out to be
> in the end.

Global variables are thread-specific. Making them shared is at this
moment not possible because Prolog terms cannot be shared between
threads. In some systems this would be simple for ground terms. Not for
SWI-Prolog.

> 3. All names should be module scoped, except maybe for Prolog flags.
>     Though there is I think an overlap between flags and settings. Perhaps we should
>     not be allowed to create new flags, but use the setting system for such
>     needs. Unfortunately, I see that user defined flags are part of  the ISO standard.

You refer to library(settings)? They serve a different role. The
settings library is intended for application configuration. The flags
are there to query things about the hosting Prolog system or tell it to
behave differently. AFAIK, user defined flags are not part of ISO. The
predicate create_prolog_flag/3 has been agreed between Vitor (YAP) and
me. It allows libraries to define flags and applications to tell/query
a library for specific behaviour.  AFAIK, flags are not module-aware
in any Prolog system.

> 4. The recorded database is weird. On the face of it, it has pretty much the
>     same functionality as dynamic predicates. All you have to do is replace
>     'Key' with 'Module' and there you are - for each module, an arbitrary collection
>      of terms which look and act pretty much like unit clauses. Also, the fact that you can
>     use an term of the key, but only the name and arity will be effective is an
>    invitation for trouble.

It is an old interface. If I take it out, I'm sure my mailbox will fill
up quickly :-) It still has some useful features: it maintains internal
sharing in terms, can represent cycles and attributed variables.

>     The only reason I can see for it being there is that it promises to be more
>     efficient for certain usage patterns, but if you ask me, it should be the job
>     of the compiler/runtime to figure out how best to optimise something, possibly
>    using information collected at run time. I suppose this was the reason behind
>     ditching index declarations a few versions ago. At the time it made me slightly
>     anxious (only slightly) but then I relaxed and thought, hey, it's the computer's
>     job to think about such things, not mine.
>
>     The fact that recorded facts have a reference for easy deletion might make
>     certain operations marginally simpler, but if they're so useful, why shouldn't we
>    be able to get a reference for asserted clauses too?

There is (traditional, pre-ISO) clause/3, asserta/2 and assertz/2. You
can use erase/1 to get rid of both records and clauses.

> 5. Backtrackable global variables have a place, or at least, they are the only way
>     I could implement Tarau's multi stream DCG idea in my previous email, which
>     can only work when there is no term copying on either setting or retrieving the
>     value of the variable. Indeed, the gap which Tarau's assumption mechanism
>     fills is that of a backtrackable assert/retract without copying.

They are basically a way to name a term, so you don't have to pass it
around. The price you pay is that you polute a global name space and you
must be careful about reentrance, etc.  But, they can indeed be
practical.

>     Also b_setval/2 does not completely undo itself - if the variable did not exist before,
>     it exists after doing and then undoing b_setval/2. The other side of this is that
>     there is no b_delete/1, a bactrackable version of nb_delete/1.

The API comes from hProlog. My original implementation had a
backtrackable name space for b_* and a non backtrackage for nb_*, but
this could not host CHR (and that is why I implemented globals). So,
there is only one global variable namespace, but there are two
assignment operations. For the other predicates, the b_* and nb_*
versions refer to the same implementation. I'm not proud of that. I
hoped ISO would come up with a globals standard. It has been discussed
in many meetings, but without result. It is like Richard's `green worm':
you basically do not want them, but if you have it it might be wise to
all have the same green worms.

> 6. The fact that nb_setval/2 copies but nb_getval/2 does not is odd and a potential
>      trap. If I understand correctly, nb_setval/2 is equivalent to
>	nb_setval(K,V) :- copy_term(V,V1), once(b_setval(K,V)).

Nope.  It is nb_setval(K,V) :- duplicate_term(V,V1), b_linkval(K,V).

note that once(b_setval(K,V)) wouldn't make the assignment permanent.
In fact, it is the same as just b_setval(K,V), as this predicate is
deterministic.

The duplicate_term/2 is needed to get rid of trailed assignments
in the term, to avoid this:

1 ?- A = term(X), ( X = b, nb_linkval(x, A), fail ; nb_getval(x,B)).
A = B, B = term(X).

While:

2 ?- A = term(X), ( X = b, nb_setval(x, A), fail ; nb_getval(x,B)).
A = term(X),
B = term(b).

> .   so perhaps nb_getval/2 should be equivalent to
>          nb_getval(K,V) :- b_getval(K,V1), copy_term(V1,V).

What would that help us?  nb_getval/2 and b_getval/2 are just the
same.

> 7. Destructive assignment just looks like trouble to me. It's neither a mutable
>     variable with a globally known address, nor an immutable variable, or even an
>     immutable reference to a mutable storage location. Looking at code that uses
>     destructive assignment, I can never quite tell what is going to happen.
>     Jan says, in an understated fashion, 'have to be careful with term identity'. I guess
>     the idea is to have something mutable but local in scope, but in that case, I think it
>     would be a lot better to be able to generate reference to a new mutable location,
>     a bit like Haskell's ST monad. Then at least the reference is immutable and
>     can be passed around, duplicated etc in a referentially transparent way. It could
>      be implemented as a blob such that the mutable location is reclaimed when the
> blob is garbage collected.

That could be a way. It is cleaner. It is also lot slower though. You
can avoid most of the copying term-identity problem by ensuring there is
an additional variable in the compound, so it never becomes ground and
copy_term/2 actually copies it (it shares ground subterms). Packaging
that nicely gives SICStus mutable terms, soo deeply admired by Richard
:-).

As far as I'm concerned, global variables and destructive assignment are
fine as they are, but should only be used for implementing things with a
nice and clean interface.  For example, from the library:

aggregate_all(count, Goal, Count) :- !,
	State = state(0, _),
	(   Goal,
	    arg(1, State, C0),
	    C1 is C0+1,
	    nb_setarg(1, State, C1),
	    fail
	;   arg(1, State, Count)
	).

This is not nice code, but it works and is fast. I also have an
implementation of findall/3 this way. It is faster on few solutions
because there is no need to catch errors and cleanup the store, but
slower on many solutions because you still have to copy the data and you
will get more garbage collections. It would be nice to have a copy that
would only copy if the term or assignments therein are created after a
certain point. Finding that the term is old enough is easy, but finding
assignments is hard (well, slow).

Hope this explains the background of all this stuff.

	Cheers --- Jan

_______________________________________________
SWI-Prolog mailing list
[email protected]
https://lists.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog