Re: load_files freezes in a multi threading application using JPL (Java calling Prolog)

On 03/17/2014 03:20 PM, Sergio Castro wrote:
> Hi Jan,
>
> Thanks for following up on this. To be clear, the deadlock only occurs
> when the second file redefines a (static) predicate defined in the first
> file. Suppressing output in the load_files/2 calls doesn't prevent the
> deadlock. As we're debugging from within a JPL section, we have not been
> able so far to get the thread stack traces. We'll follow up on this bug
> if we manage to dig further information.

AFAIK, gdb doesn't work on the latest Maverics release, but xcode comes
with something that is almost the same.  Wait for the process to 
deadlock, get the process id, run

% gdb `which java` PID
(gdb) thread apply all backtrace

and you should have a stack trace of all threads.  Optionally, you
may isolate the Prolog ones and do (if <N> is a Prolog thread, which
you can see because it has PL_next_solution() on the stack)

(gdb) thread <N>
(gdb) call PL_backtrace(10,0)

This dumps the Prolog stack to stderr stream.  10 is the number of 
frames and the second argument is a bitwise-or of some flags for special
features.  There is some likelyhood that this will deadlock :-(

This will surely work on Linux.  It also works on MacOS < Maverics. With 
Maverics I hope it works with the Apple debugger.

Hope this helps

	Cheers --- Jan


>
> Cheers,
>
> Sergio
>
>
> Le Lundi 17 mars 2014 13h43, Jan Wielemaker <[email protected]> a écrit :
>
>     On 03/17/2014 12:02 PM, Sergio Castro wrote:
>      > Hi Jan,
>      >
>      > I did some more experiments that may shed some light on the
>     problem. I'm
>      > using two files with the same content:
>      >
>      > ---- hello.pl
>      > :- thread_local(hello/1).
>      > hello(X) :- write('Hello '), write(X), flush_output.
>      > ----
>      >
>      > ---- hello2.pl
>      > :- thread_local(hello/1).
>      > hello(X) :- write('Hello '), write(X), flush_output.
>      > ----
>      >
>      > Loading these two files results in no deadlock and the following
>     output:
>      >
>      > %
>      >
>     /Users/sergioc/.jpc/file_Users_sergioc_Documents_workspaces_heal_jpc_target_classes/org/jpc/pl/jpc.pl
>      > compiled 0.00 sec, 1 clauses
>      > before load_files
>      > % /Users/sergioc/.jpc/hello.pl compiled 0.00 sec, 2 clauses
>      > after load_files
>      > before load_files
>      > Warning: /Users/sergioc/.jpc/hello2.pl:5:
>      > Redefined static procedure hello/1
>      > Foreign predicate system:$record_clause/3 did not clear exception:
>      >
>     error(permission_error(modify,thread_local_procedure,hello/1),context(system:
>      > $record_clause/3,_G1162))
>      > % /Users/sergioc/.jpc/hello2.pl compiled 0.00 sec, 2 clauses
>      > after load_files
>      > OK
>      >
>      > If I comment out the thread_local/1 directives (that are being
>     used just
>      > to trigger an exception) I get a deadlock:
>      >
>      > %
>      >
>     /Users/sergioc/.jpc/file_Users_sergioc_Documents_workspaces_heal_jpc_target_classes/org/jpc/pl/jpc.pl
>      > compiled 0.00 sec, 1 clauses
>      > before load_files
>      > % /Users/sergioc/.jpc/hello.pl compiled 0.00 sec, 2 clauses
>      > after load_files
>      > before load_files
>      > Warning: /Users/sergioc/.jpc/hello2.pl:5:
>      > Redefined static procedure hello/1
>      > Previously defined at /Users/sergioc/.jpc/hello.pl:5
>      >
>      > It seems that the handling of the exception in the first case
>     might be
>      > doing some cleanup that overcomes the deadlock. Does this
>     provides you
>      > any hint to the cause of the deadlock?
>
>     Not really.  I tried with this code:
>
>     test :-
>              load_files(hello, []),
>              thread_create(load_files(hello2, []), Id, []),
>              thread_join(Id, Status),
>              writeln(Status).
>
>     That loads both scenarios just fine (with errors of course).  Does
>     code without errors work?  Does it work if you suppress output using
>     load_files(hello, [silent(true)]) (and no errors in the code)?  Anyway,
>     probably attaching a debugging to the deadlocked process and getting
>     the stacks of all threads is the most promising route to get a clue.
>
>          Cheers --- Jan
>
>      >
>      > Sergio
>      >
>      >
>      >
>      > Le Dimanche 16 mars 2014 15h52, Jan Wielemaker
>     <[email protected] <mailto:[email protected]>> a
>      > �crit :
>      >
>      >    On 03/14/2014 05:27 PM, Sergio Castro wrote:
>      >      > Hello,
>      >      >
>      >      > My scenario is the following: I implemented a simple
>     mechanism to
>      >      > overcome certain JPL limitations regarding multithreading
>      >      > applications (e.g., afaik a JPL query always has to be
>     opened, its
>      >      > results traversed, and closed in the same thread). So the
>     main idea
>      >      > is that I keep a dedicated thread in a query wrapper, and
>     all query
>      >      > operations are executed in the context of that thread.
>     Then the
>      >      > calling thread waits until the query thread completes,
>     since the
>      >      > objective of the query thread is just to circumvent the JPL
>      >      > multithreading limitations regarding the need of being
>     manipulated in
>      >      > the same Java thread.
>      >      >
>      >      > If possible, I would like feedback regarding the
>     correctness of this
>      >      > design. I am aware that this is of course less efficient than
>      >      > directly using a JPL query (since with my query wrapper
>     there is a
>      >      > separate Java thread for each active query), but I find it
>     useful for
>      >      > scenarios where I do not want to be constrained to
>     traverse a JPL
>      >      > query in the same thread. I am happy to share the code if
>     someone is
>      >      > interested btw.
>      >
>      >    I don't know. Technically, I guess this should be possible,
>     but I wonder
>      >    what requirement justifies this complication. Can you give a
>     scenario
>      >    where yiou would like to do this?
>      >
>      >      > However, I found a problem that appears in this setting.
>     If I execute
>      >      > the query below twice (using JPL), it freezes at the
>     second call to
>      >      > load_files/2 (this is a simplification of my problem to
>     keep the
>      >      > description simple, in my application I am in fact loading
>     files
>      >      > using Logtalk, and the calls do not happen exactly one after
>      >      > another):
>      >      >
>      >      > "writeln('before load_files'), flush_output,
>      >      > load_files('<filepath>/hello.pl', []), writeln('after
>     load_files'),
>      >      > flush_output"
>      >      >
>      >      > So the output I see is:
>      >      >
>      >      > before load_files % <filepath>/hello.pl compiled 0.00 sec,
>     2 clauses
>      >      > after load_files before load_files <FROZEN HERE>
>      >      >
>      >      > Note that the calls to load_files do not happen
>     concurrently (afaik
>      >      > files should not be loaded concurrently). However, they do
>     happen in
>      >      > different threads (synchronised between them). I also
>     verified that
>      >      > if I execute everything in the same thread, then
>     load_files does not
>      >      > freeze.
>      >
>      >    load_files/2 can be called concurrently.  Additional threads
>     trying to
>      >    load a file while this is in progress will wait for the initiating
>      >    thread.  The above works find in pure Prolog, so I assume the
>     culprit
>      >    is JPL or your wrapper.
>      >
>      >      > Any idea what may be the problem ?
>      >
>      >    Typically, attach e.g., gdb and see which threads are blocking
>     where.
>      >    That might give a clue.
>      >
>      >          Cheers --- Jan
>      >
>      >
>      >
>      >
>      >      >
>      >      > Thanks,
>      >      >
>      >      > Sergio -------------- next part -------------- HTML attachment
>      >      > scrubbed and removed
>     _______________________________________________
>      >      > SWI-Prolog mailing list [email protected]
>     <mailto:[email protected]>
>      >    <mailto:[email protected]
>     <mailto:[email protected]>>
>
>      >      > https://lists.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog
>      >      >
>      >
>      >
>
>
>

_______________________________________________
SWI-Prolog mailing list
[email protected]
https://lists.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog