(SETF (DOCUMENTATION <function> <T|FUNCTION>) <doc-string>) bugs

Gary Byers <[email protected]> Mon, 20 Jan 2003 20:14:30 -0700 (MST)
Newsgroups gmane.lisp.openmcl.bugs
Message-ID <[email protected]>
The (SETF DOCUMENTATION) methods specialized on the class FUNCTION and on
DOC-ID's (EQL T) and (EQL 'FUNCTION) call each other in 0.13.3.  (The
(EQL FUNCTION) method should call the (EQL T) method, and the (EQL T)
method should use (CALL-NEXT-METHOD) to associate the doc string
with the function object.  ENSURE-GENERIC-FUNCTION calls code which
does (SETF (DOCUMENTATION gf T) doc-string), so code which does
ENSURE-GENERIC-FUNCTION or DEFGENERIC with a :DOCUMENTATION argument
can run into this, as James Anderson reported.

That problem's bad enough, but it exposes a couple of other bugs.

Generic function calls are generally not tail-recursive: they
create a stack frame to contain their incoming arguments and call
the method function(s) of the effective method in another frame.
When a generic function call returns, it returns to code in
the kernel that discards the extra stack frame (with which
no function invocation is associated : OpenMCL's backtrace
just identifies these "kernel" frames as 'NIL NIL', and
should generally just hide them ...).  A tail-recursive
call from a method-function will normally reuse the stack
frame that it was invoked in, but the "extra" kernel frame
will only be popped off when that later frame is returned from.
A cycle of recursive generic function calls (as caused by the
(SETF DOCUMENTATION) bug will cause the stack to fill up
with these extra kernel frames, even if the generic function
calls are tail recursive.  The stack will eventually overflow
with a long chain of (meaningless) kernel frames as the only
information available to backtrace.

The scenario in James Anderson's bug report exposes another
problem: a thread's "control" (function invoation history)
and "value" (arguments/locals/temporaries) stacks are roughly
the same size (1MB, by default) and grow at nearly exactly
the same rate in this "spinning generic function call" situation.
therefore, both stacks are likely to  overflow at roughly the
same time.  (In fact, the break level is otten > 1 when a break
loop is reliably entered: the attempt to signal the first
stack overflow is often interrupted by an overflow on the
second stack.

When a stack overflows, the stack's limit is extended to include
some "reserved" space, to allow the user some elbow room to
poke around in a break loop.  When that break loop exits (normally
to throw to some surrounding restart or handler, an unwind-protect
cleanup form restores the stack limit to its original value.
Executing an unwind-protect cleanup can cause frames and values
to be pushed on both stacks, so the act of restoring one stack's
overflow limit can cause the other stack to overflow (making it
nearly impossible to return from the break loop.)

The original bug (the (SETF DOCUMENTATION) bug) is easy to fix;
changing how stack-overflow recovery works may be harder.  Generic
functions pretty much -have- to stack-allocate their arguments
(heap-allocating &rest lists is probably not a practical alternative)
and so interfere with tail-recursion elimination; it would seem
desirable for generic functions to leave some trace of their
invocation so that siturations like this one are easier to
recognize.