Re: Unboxed structures (yes, again, only this time for real)

On 16 Jan 2026, at 12:55, Yuri Davidovsky <[email protected]> wrote:
> 
> 
> Well, just to clear it out straight away, that does not say that data cannot be inlined within a struct body and is always boxed, does it?

For LW, objects which are never boxed (so immediate objects) are of course not boxed in structures.  And objects which are boxed (non-immediate objects) *are* boxed in structures.  There are other objects -- specialized arrays -- which can hold unboxed versions of (a very restricted number of types of) objects.  It would be possible to have this be true for some normally-boxed objects in structure fields.  LW does not do this.  SBCL can.

The restriction on types is because this can only be done for types for which object identity is weakened in the language.  Those types are numbers and characters in CL.

So for instance it must be the case that, given

(defun f (o)
  (let ((a (make-array 1))
        (b (make-array 1)))
    (setf (aref a 0) o
          (aref b 0) o)
    (values
     (eq o o)
     (eq o (aref a 0))
     (eq o (aref b 0))
     (eq (aref a 0) (aref b 0)))))

Then (f (cons 1 1)) returns t, t, t, t.  And that remains true even if a and b are some specialised array: they must store a pointer, not the cons itself.

But, given

(defun g (o)
  (let ((a (make-array 1 :element-type 'double-float))
        (b (make-array 1 :element-type 'double-float)))
    (setf (aref a 0) o
          (aref b 0) o)
    (values
     (eq o o)
     (eq o (aref a 0))
     (eq o (aref b 0))
     (eq (aref a 0) (aref b 0)))))

Then (g 1.0d0) may return any possible combination of four ts and nils (and in LW returns t nil nil nil, at least when I tried it).
> 
> Additionally, what does ’none of these objects will be boxed’ mean exactly? Does it mean the bit tags will be removed? If so, why can they be removed for single floats, but not for doubles? The default pointer size of 8 bytes allows that.
> 
> Or are they still tagged, but the values not located on the heap (except doubles, which are located on the heap)?

They are immediate objects, they're never boxed.  The word corresponding to them will have a bunch of tag bits at the bottom, perhaps some space, and then the value itself in the upper bits (so in the upper half-word for single-floats on a 64-bit machine).  Boxed objects have a pointer with a number of tag bits (likely 4) at the bottom and the pointer in the rest of the bits.  The object in memory then usually has some header saying what it is followed by the value.  Some special types don't have a header word (conses).

find-object-size is how you can poke at all this in LW:

(find-object-size 1.0) is 0 in 64-bit LW.  (find-object-size 1.0d0) is 16 (header and value), (find-object-size (cons nil nil)) is also 16 (no header for conses, just two pointers).

> 
> Lisp is notoriously very loose on types and that alone does not say much to me personally without seeing some ground truth on the byte level.

It's not very loose on types: no GCd language can get confused about types if it doesn't want to explode every once in a while.  it is dynamically typed, and for instance the type option to structure fields may not be enforced (or change the representation), and the same goes for type declarations in functions &c.

> However it could be correct that the particle array was indeed an array of pointers. The reason why they were performing as fast as a contiguous array of custom structs could be because the structs themselves were [...]

It is correct, yes.  The allocator will typically allocate sequentially I think:

(defun c (n)
  (let* ((foos (loop repeat n
                     collect (make-foo)))
         (addrs (mapcar #'sys:object-address foos)))
    (values (mapcar #'- (rest addrs) addrs)
            (find-object-size (first foos)))))

Then probably (c n) will return a list mostly of some value (32) and that value.



_______________________________________________
Lisp Hug - the mailing list for LispWorks users
[email protected]
http://www.lispworks.com/support/lisp-hug.html