Re: compiling crashes, compiling and loading slowness

Andreas Franke via Sbcl-help <[email protected]> Tue, 14 Apr 2026 21:20:08 +0000
Newsgroups	gmane.lisp.steel-bank.general
Message-ID	<trinity-7848b6df-178c-44fd-a44b-9be879180993-1776201608374@trinity-msg-rest-gmx-gmx-live-5cf7d7879b-tpsdp>
--refeik-bca044ab-575f-4258-bdf9-c3916c2b8015
Content-Type: text/html; charset=UTF-8

<html><body><div style="font-family: 'verdana'; font-size: 12px; color: #000;">Maybe the attachments could be worth trying?</div>
<div style="font-family: 'verdana'; font-size: 12px; color: #000;">The first one appears to fix the crash for me;</div>
<div style="font-family: 'verdana'; font-size: 12px; color: #000;">the second one addresses the same issue on a different hashset.</div>
<div style="font-family: 'verdana'; font-size: 12px; color: #000;"><br>Disclaimer: NO FITNESS FOR ANY PURPOSE IMPLIED.<br>PURE AI SLOP. MAY BE CONFIDENTLY WRONG IN UNEXPECTED<br>WAYS. MAY CRASH YOUR HARD DRIVE. MAY EAT YOU ALIVE.<br>YOU ARE SOLELY RESPONSIBLE FOR ANY USE. ENJOY.</div></body></html>
--refeik-bca044ab-575f-4258-bdf9-c3916c2b8015
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0001-Fix-hashset-degeneration-in-make-fun-name-hashset.patch
X-UI-Content-Hash: 525f34504525b2a0f6671876f55c9d2e88ac2976ad7f31d9862a9ef16e191ee6
Content-Transfer-Encoding: quoted-printable

=46rom 4e70b42af86789084bb7378caa0156ebbff8ae2c Mon Sep 17 00:00:00 2001
From: cc <cc@sbcl>
Date: Tue, 14 Apr 2026 21:05:35 +0000
Subject: [PATCH 1/2] Fix hashset degeneration in make-fun-name-hashset

The CL spec requires that same-named symbols in different packages
produce the same SXHASH.  With 1000+ packages each defining functions
like DESTROY!-0 or ENABLE-STATISTICS-0, the compiler's fun-name-hashset
(used for duplicate name detection during compilation and FASL loading)
degenerates -- PSL grows linearly and overflows the u8 counter at 256.

Replace SXHASH with a hash function that mixes symbol-name-hash with
a per-package identity hash.  Handle compound names like (SETF FOO)
by hashing elements individually.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
=2D--
 src/compiler/early-c.lisp | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)

diff --git a/src/compiler/early-c.lisp b/src/compiler/early-c.lisp
index a242ad9fe..c62d6d2d4 100644
=2D-- a/src/compiler/early-c.lisp
+++ b/src/compiler/early-c.lisp
@@ -203,14 +203,31 @@ possible.")
         ;; And who knows what the host considers "simple".
         #-sb-xc-host (not simple-array)))
=20
+;;; Can't use SXHASH as the hash function: CL requires that same-named
+;;; symbols in different packages produce the same SXHASH, so e.g.
+;;; 1000 packages each defining DESTROY!-0 would all collide identically,
+;;; making the Robin Hood hashset degenerate (PSL grows linearly with N).
+#-sb-xc-host
+(defun fun-name-hash (name)
+  (flet ((symbol-pkg-hash (s)
+           (logand (mix (symbol-name-hash s)
+                        (let ((pkg (symbol-package s)))
+                          (if pkg (sxhash pkg) 0)))
+                   most-positive-fixnum)))
+    (if (symbolp name)
+        (symbol-pkg-hash name)
+        ;; compound names like (SETF FOO) -- hash elements individually
+        ;; since sxhash of same-named symbols in different packages colli=
des
+        (let ((h 0))
+          (declare (fixnum h))
+          (dolist (elt name (logand h most-positive-fixnum))
+            (setq h (mix (if (symbolp elt) (symbol-pkg-hash elt) (sxhash =
elt))
+                         h)))))))
 (defun make-fun-name-hashset ()
   (make-hashset 32
                 (lambda (a b) (or (eq a b) (and (consp a) (consp b) (equa=
l a b))))
-                ;; We don't emulate sb-xc:sxhash thoroughly enough to has=
h compound names
-                ;; (lists are rejected) but it doesn't actually matter wh=
at the hash is
-                ;; for duplicate name detection.
                 #+sb-xc-host #'cl:sxhash
-                #-sb-xc-host #'sxhash))
+                #-sb-xc-host #'fun-name-hash))
=20
 (defstruct (compilation (:constructor make-compilation
                                       (&optional msan-unpoison
=2D-=20
2.43.0


--refeik-bca044ab-575f-4258-bdf9-c3916c2b8015
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0002-Fix-hashset-degeneration-in-slot-name-lists.patch
X-UI-Content-Hash: 196c29e61a60f0cdcae15ff1daa29b6c807ee366bc9482715e0c090c3ee0379
Content-Transfer-Encoding: quoted-printable

=46rom cea49c2b7d4084e9ed43481bea8ad284177d062b Mon Sep 17 00:00:00 2001
From: cc <cc@sbcl>
Date: Tue, 14 Apr 2026 21:14:16 +0000
Subject: [PATCH 2/2] Fix hashset degeneration in *slot-name-lists*

Same root cause as the make-fun-name-hashset fix.  The *slot-name-lists*
hashset interns PV-table slot-name-lists during FASL loading.  With
1000+ classes across packages each having slots named e.g. VALUE, the
single-element lists (PKG-N::VALUE) all hash identically, producing
PSL overflow at N=3D256.

Replace SXHASH with a hash function that mixes symbol-name-hash with
a per-package identity hash.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
=2D--
 src/pcl/vector.lisp | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/src/pcl/vector.lisp b/src/pcl/vector.lisp
index e88ae33b7..184b8a2f9 100644
=2D-- a/src/pcl/vector.lisp
+++ b/src/pcl/vector.lisp
@@ -50,7 +50,19 @@
=20
 ;;; Used for interning parts of SLOT-NAME-LISTS, as part of
 ;;; PV-TABLE interning -- just to save space.
-(define-load-time-global *slot-name-lists* (make-hashset 64 #'list-elts-e=
q #'sxhash))
+;;; Can't use SXHASH as the hash function: CL requires that same-named sy=
mbols
+;;; in different packages produce the same SXHASH, so e.g. 1000 wrapper c=
lasses
+;;; across packages each having a slot named VALUE would all collide iden=
tically,
+;;; making the Robin Hood hashset degenerate (PSL grows linearly with N).
+;;; Mix the symbol-name hash with a per-package identity hash instead.
+(defun slot-name-list-hash (list)
+  (let ((h 0))
+    (declare (fixnum h))
+    (dolist (sym list (logand h most-positive-fixnum))
+      (setq h (mix (let ((pkg (symbol-package sym)))
+                     (if pkg (sxhash pkg) 0))
+                   (mix (symbol-name-hash sym) h))))))
+(define-load-time-global *slot-name-lists* (make-hashset 64 #'list-elts-e=
q #'slot-name-list-hash))
=20
 ;;; Used for interning PV-TABLES, keyed by the SLOT-NAME-LISTS
 ;;; used.
=2D-=20
2.43.0


--refeik-bca044ab-575f-4258-bdf9-c3916c2b8015
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--refeik-bca044ab-575f-4258-bdf9-c3916c2b8015
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Sbcl-help mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sbcl-help

--refeik-bca044ab-575f-4258-bdf9-c3916c2b8015--