Re: predutils package available (Re: PAC library for develop version available)

On 10/03/2014 08:11 AM, Richard A. O'Keefe wrote:

> In order to handle circular dependencies between modules,
> it was necessary to be able to read and process all the
> linkage information without having to read all of a file.
> In QP it was quite important therefore to put :-meta_predicate
> declarations *for exported predicates* at the top.  Others
> just had to go before their first use.
>
> Jan can tell you whether this is still necessary or useful in
> SWI Prolog.

No and yes. Well, it is more subtle. SWI-Prolog does not need
meta-predicate declarations to be in any specific order for getting the
module qualification of terms correct. It doesn't even need the
declaration before the definition, but it does need it before the first
call. The module qualification is in the startup code of the
meta-predicate itself. The startup code also deals with stuff like
indexing and is generated lazily at the first call. This is achieved by
setting the initial startup code to S_VIRGIN. The implementation of
S_VIRGIN considers the clauses and predicate properties and replaces
itself with a sequence that realises the proper startup for this
predicate.

It is useful because the cross-referencer that drives the semantic
highlighting for the built-in editor works pretty much like Quintus:
it reads (only) the headers of the imported modules.

It is also useful to deal with goal-expansion. If a predicate has an
argument with the meta-specification `0`, expand_goal/2 will be called
for that argument (recursively). This means that if your code relies on
goal expansion, you need to make sure that meta predicate declarations
are known before the first usage.  I consider that pretty unfortunate.
I see various ways out:

   - Enforce Quintus-like ordering, emitting an error if these are
     not respected (i.e., the system will assume that a predicate
     is not a meta-predicate on first reference and raise a warning
     if it finds a meta-predicate declaration).
   - Implement multi-pass compilation, so the order doesn't matter
     anymore.  More below.
   - Drop goal_expansion/2 and introduce inline predicates.  ECLiPSe
     went that way.   Now, missing a declaration will only have some
     impact on performance (no inlining).  We could even dynamically
     repair that by rewriting the calling clause at the moment that
     we discover that we are making a call to the implementation of
     an inline predicate.

>> :- meta_predicate  my_flip(2, ?, ?).
>> my_flip(F, X, Y):- call(F, Y, X).
>
> That makes perfect sense in a multipass implementation.
> Quintus Prolog was a single pass implementation:
>   - while reading a term does not fail
>       expand it
>       for each clause in the expansion
>          apply module-related transformations
>          compile it
>          add it to memory (or the .qof file)
>
> An arguably *better* scheme would read an entire file
> into memory and check it and process it before storing
> any part of it (the way the Erlang compiler processes
> Erlang files), but that wasn't the way Prolog worked
> before Quintus, and customers depended on immediate-
> incremental processing.
>
> And that meant meta-predicates had to have their
> interface declared before any definition or call.
>
> The problem with putting the :- meta_predicate
> declaration immediately before the definition is
> "what if there is a *call* even earlier?"  How
> can you have mutually recursive meta-predicates if
> :- meta_predicate declarations can follow calls?
>
> Again, I do not speak for SWI Prolog here.  For all I
> know, now that we no longer have to live with 4 MB
> machines, Jan may have decided to process whole files.
> (ISO Prolog certainly jettisoned historic features in
> order to make that straightforward.)  Whether it's
> portable to take advantage of that is another matter.
>
>>
>> (Though I am not sure whether the  directive is necessary
>> in this particular case).
>
> To the best of my knowledge SWI does no meta-predicate
> inference.

What do you mean by `meta-predicate inference'?  It qualifies
arguments, its list_undefined/0 (called by make/0 and check/0)
use meta-predicate declaration to compute what can be called,
etc.  What would be missing?

>> Is this a bad practice ?  Or are you saying on  `header information`
>> for use of external predicates, which sounds reasonable principle.
>
> What I am saying is
>   - if a predicate is exported, include it in the :-meta_predicate
>     declaration at the beginning of the file.
>   - if a predicate is not exported, make sure its :-meda_predicate
>     declaration precedes the first use.  (And a good way to do
>     _that_ is to put it before any predicate definitions or
>     initialisation commands.)

I think that is a good style guide. For SWI-Prolog, you could opt to
place meta-predicate declarations of local meta-predicates before their
definition. Note that technically you don't need these declarations
anyway, but they help the code analysis tools (although code analysis
will find the simple cases itself thanks to code contributed by the
group of Guenter Kniesel).

> It was historic practice before Quintus Prolog to put
> :- mode declarations for all the predicates in a file
> at the beginning of that file.  More precisely, it was
> the practice to write a single :- mode declaration
> listing all the predicates, typically in alphabetic
> order, so that the user had a handy list of predicates.
>
> :- meta_predicate declarations are just :- mode
> declarations with a few extra options.

We have PlDoc for documenting the mode and modern editors typically
provide some overview of the content of a file.

> If we were designing Prolog these days, when people carry around
> lightweight laptops with 8 GB of memory, we'd probably do what
> Haskell and Erlang do, and that is parse and check a whole module
> before acting on any part of it
>
> SWI Prolog 7.x has cut the umbilical cord of backwards
> compatibility, so it would not be out of place for SWI 8.x
> to adopt a whole-file-processing approach.  It always did
> feel a little bit wrong that a declarative language should
> have any dependency on the order in which declarations and
> definitions were written.

One could surely argue for that.  There are also some counter
arguments.

   * Prolog programs may not be just code, but can also be
     (potentially huge) amounts of data.  We want as little
     as possible overhead loading that.
   * Especially when prototyping, it is so nice that you don't
     have to specify all dependencies upfront.  Some people
     like exhaustive use_module/2 declarations to specify
     exactly what they want, others prefer use_module/1,
     import the whole lot into `user` and rely on module
     defaulting or rely on auto loading.  Depending on
     the project, organization of the development team
     and probably many other factors, there are advantages
     in each of these approaches.  Anything except using
     use_module/2 however forces the system to make
     assumptions about the properties of referenced
     predicates.
   * It allows for local definition of source transformation
     and generation rules without the need for boilerplate.

I think that my long-term plan is to properly implement code inlining,
so we can do `:- inline maplist/2.` instead of the ugly
library(apply_macros). Inlining avoids visibility ambiguity issues of
goal expansion and guarantees consistent semantics. It also simplifies
source-level debugging. Possibly we can compile inlined code in such a
way that the original code remains accessible, so your call to maplist
becomes something like this internally:

   (   inlining
   ->  'generated_pred_XXX'(...)
   ;   maplist(XXX, ...)
   ).

This would make SWI-Prolog source code completely order independent,
while allowing for single-pass compilation and even dynamically fixing
your code if the definition of an inlined predicate has changed.

I think that working towards a safe fully dynamic system is a promising
route.  (SWI-)Prolog is already quite strong in this area, but currently
there are a few places where you need to get things in the right order
and this confuses people.

	Cheers --- Jan