Re: Mysteries of unwind-protect

-----------------
From: "Tim Bradshaw (as tfb at tfeb dot org)" 
Date: Thu, 15 Jan 2026 15:41:39 

>In other words, the unwind forms in unwind-protect may well happen in an environment where things
>it might seem safe to assume are not, in fact, true, because there's been a trip through the
>debugger.


-----------------
This is what LLMs say, with Claude presenting the function: ts-immutable.
Maybe is 100% correct, or maybe there is some nonsense, but ts-immutable needs checking.

A single function is easy to copy-paste, but LLMs often provide many functions
All their very long explanations are only as good as the code actually works.


In a LispREPL-LLM environment, all functions would be verified automatically.
If there is any error, the LOOP will continue until the code is correct.


GEMINI:
This is a classic performance "gotcha" in Common Lisp when dealing with high-performance numeric
code. The compiler isn't actually losing track of the type of v (the array); rather, it is losing
the ability to keep the value of v0 (the float) in a hardware register across the boundary created
by unwind-protect. 


CLAUDE:
Why unwind-protect Causes Boxing
The key insight is that unwind-protect's cleanup form can execute in two very different contexts:

Normal exit: The protected form completes normally
Non-local exit: An error is signaled, throw is called, return-from jumps out, etc.


When a non-local exit occurs, the cleanup form runs during stack unwinding. At this point:
The dynamic environment may be partially unwound
Exception handlers may be active
The compiler cannot guarantee the same register allocation or stack frame layout
The code may need to be callable from the unwinder mechanism itself

To handle this safely, compilers typically generate cleanup code that:
Can be called from any context
Doesn't assume specific register states
Uses a more conservative calling convention

This often means boxing float values so they can be passed through the generic unwinding mechanism.


Why progn Works
With progn, everything executes in sequence with no special control flow. The compiler can:

Keep v0 in an unboxed floating-point register
Perform the incf using unboxed arithmetic
Store directly back to the array using unboxed operations
All in the normal stack frame with predictable register allocation

Is This Safe?
Yes, the compilers are correct to box here. It's not a limitation - it's correctness. The type of v
isn't in question; it's that the cleanup form must be executable in contexts where normal unboxed
optimization assumptions don't hold.

The Fundamental Issue
This confirms the real problem: unwind-protect with mutable closed-over variables forces heap
allocation and boxing because:

The cleanup must be a proper closure (heap-allocated)
Mutable variables need indirection via value cells
Value cells contain boxed (pointer-tagged) values

Verification: Immutable Variable

(defun ts-immutable ()
  (declare (optimize speed (safety 0)))
  (let ((v (make-array 1 :element-type 'double-float 
                       :initial-element 0.0d0)))
    (declare (type (simple-array double-float (1)) v))
    (let ((v0 (aref v 0)))
      (declare (type double-float v0))
      (unwind-protect
          (setf (aref v 0) (+ v0 1.0d0))  ; v0 not mutated
        (format t "cleanup~%"))))
  (values))


This should no boxing because v0 isn't mutated, so no value-cell is needed - the closure just
captures the immutable value.
-----------------------------


Kind Regards,
Adrian W. Pasieka