Multiprocessing question.
quasi <[email protected]>
| Newsgroups | gmane.lisp.cmucl.general |
|---|---|
| Message-ID | <[email protected]> |
Hello Folks,
I am currently working on CMUCL to build a search system for an
airline database.
To improve performance, batches of requests are done in parallel. This
was my first time using the CMUCL MP. My starting point was
<URL:http://www.trakt7.net/cmucl%20and%20multiprocessing>
I have found the CMUCL is extremely stable and very fast to work
with. Thank you guys for this wonderful software.
But I have been having several hard to trace problems in 2 areas,
namely, MP and sockets. This is more likely due to my lack of
know-how. I would be very grateful if you could help.
Firstly, I am using trivial-sockets to make HTTP connections. We have
seen that whenever we try to connect to a system which is down, or
when my interface has gone down, CMUCL goes into continuous GC taking
up 99% CPU. This is when MP is initialised.
After we abort from such errors and try to run anything I get the
following message.
Error in function LISP::ASSERT-ERROR:
The assertion (NOT MULTIPROCESSING::*INHIBIT-SCHEDULING*) failed.
[Condition of type SIMPLE-ERROR]
Restarts:
0: [CONTINUE] Retry assertion.
1: [ABORT-REQUEST] Abort handling SLIME request.
2: [CONTINUE] Return NIL from load of "home:.cmucl-init".
3: [ABORT] Skip remaining initializations.
Backtrace:
0: (LISP::ASSERT-ERROR (NOT MULTIPROCESSING::*INHIBIT-SCHEDULING*) NIL NIL)
1: (MULTIPROCESSING:PROCESS-WAIT-WITH-TIMEOUT "Waiting for Final Results" 55 #<Closure Over Function "DEFUN AIR-SEARCH" {592F35A1}>)
Locals:
MULTIPROCESSING::ARGS = NIL
MULTIPROCESSING::PREDICATE = #<Closure Over Function "DEFUN AIR-SEARCH" {592F35A1}>
MULTIPROCESSING::TIMEOUT = 55
MULTIPROCESSING::WHOSTATE = "Waiting for Final Results"
To manage the data between the different parallel requests I am using
a global hash table with a lock. To wait for all the processes to
complete I have a counter locally and also one in the global hash
table which the processes on completion increment.
I use process-wait-with-timeout to keep checking the values of the
counter in the global hash table. Is this a good way for a process to
wait ?
I will be gratefull for any insights into MP.
thanks,
quasi
--
quasi
Utopia Unlimited!