Re: Process-resume hangs

On Mon, 9 Jan 2006, Dan Corkill wrote:

> Gary,
>
> Thanks for the prompt response and clear explanation.
>
>> (ccl:process-suspend ccl:*current-process*) can't work as implemented,
>> and I'm not sure if -should- work.  (If it's decided that it shouldn't,
>> it should probably detect that case and signal an error rather than
>> fouling things up as it currently does.)
>> 
>> The documentation claims that a process can suspend itself and be
>> resumed by another process; it also notes that PROCESS-SUSPEND can
>> be dangerous, since the suspended process may own a lock or other
>> resources, and this can (fairly easily) lead to deadlock.  It doesn't
>> note the fact that that act of suspending a process requires a lock,
>> and if a process suspends itself it continues to own this lock (that
>> was either not true when the documentation was first written, or
>> was simply overlooked.)
>
> The background for the bug report is from testing for the CL Gardeners 
> Portable-Threads initiative (http://wiki.alu.org/Portable_Threads). I've 
> added a check and error-signal for self-suspension at the portable-threads 
> interface layer.
>
> I've always been nervous about the heavy-handed nature of suspend-process, 
> and I am now soliciting opinions about removing hibernate/awaken-process from 
> the portable-threads interface.  I assume that the cost of a long-duration 
> wait for a semaphore in OpenMCL is not excessive in comparison to a suspended 
> process.
>

The overhead's roughly the same: waiting for a semaphore blocks (in
the OS) until the semaphore's available (it actually uses PROCESS-WAIT
to repeatedly block for up to a second, but that's mostly done to
update the "whostate" so that :PROC and any similar tools can say
that the thread's waiting on a semaphore.)  Suspending waits indefinitely
for a "resume" signal.  The differences are:

a) waiting on a semaphore is generally interruptible.
b) neither WAIT-ON-SEMAPHORE nor SIGNAL-SEMAPHORE is serialized at
    the implementation level; any thread that wants to can signal any
    semaphore that it wants to at any time.

The underlying mechanism used by PROCESS-SUSPEND/PROCESS-RESUME needs
to be there: when a thread enters the GC, it needs to have a reliable
way of guaranteeing that no other thread is running while it's analyizing
and reorganizing memory contents and it doesn't want those other threads
to wake up for any reason until it's done.

The GC used to try to free some unreferenced malloc()'ed memory while
other threads were suspended.  On some platforms, malloc() and free()
use a global lock, and things would come to a grinding halt if the GC
tried to call free() while a suspended thread owned the malloc lock.
(The GC was changed to do that free()ing after it had resumed suspended
threads, e.g., to be even more careful about what it does with threads
suspended.)  The moral being that even very careful, very low-level
code can find a way to lose.

I think that the basic argument in favor of providing PROCESS-SUSPEND/
RESUME is philosophical: you should have enough rope to hang yourself
with.  The documentation -does- note that there are all kinds of ways
to hang yourself.

> Thanks for clarifying the documentation claim that self-suspension is 
> supported in OpenMCL.

I'm reasonably sure that it worked without deadlock at some point
in the past.  (I fact, I think that a newly-created thread suspended
itself while waiting to be preset and run at one point in time.)  I'm
also sure that there are good reasons for the locking (TCR_LOCK) that
breaks self-suspension, and I think that those reasons outweigh whatever
benefit continuing to (nominally) support self-suspension would offer.

>
> -- Dan
>
>