Re: The myth of fingerprints, er, no, garbage collectors
"Yuri Davidovsky (as work at disclosure dot ie)" <[email protected]> Thu, 25 Jun 2026 22:42:08 +0200
| Newsgroups | gmane.lisp.lispworks.general |
|---|---|
| Message-ID | <[email protected]> |
--Apple-Mail=_B13B509A-4277-47F4-ADA3-D94A91709456 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On 25 Jun 2026, at 22:18, Tim Bradshaw (as tfb at cley dot com) = <[email protected]> wrote: >=20 > That's why I wrote my (unreleased, may be abandoned) SOA thing. My custom struct thing is live and kicking in the meantime. Working now = on an upgrade that allows it to lay data out in a simd friendly format = while being able to access structs in a sequential manner in lisp = seamlessly. ;;; test simd set ;;; 9 structs are populated across 3 4-element frames=20 ;;; with three slots empty in last frame ;;; id: | 0 1 2 3| 4 5 6 7| 8 0 0 0| ;;; last-col-id:| 10 11 12 13| 14 15 16 17| 18 0 0 0| ;;; x: |0.0 0.1 0.2 0.3|0.4 0.5 0.6 0.7|0.8 0 0 0| ;;; y: |1.0 1.1 1.2 1.3|2.4 2.5 2.6 2.7|3.8 0 0 0| ;;; here we write the struct field values into the test struct set = vector ;;; sys:typed-aref is used to store various fundamental types in a = single ;;; byte array that can be easily outsourced to external libraries (defmacro put-bytes (list type offset &optional (by 4)) `(progn ,@(collect nil (do-list (el list :count-var i :by by) (add `(put-byte set (+ ,offset ,i) ,el ,type)))))) (defparameter *simd-set-test* (let ((set (make-byte-array (* 3 4 16)))) ; 3 frames 4x16 structs each ;;; populating frame 1 (put-bytes ( 0 1 2 3) :uint32 0) ; id simd-offset = =3D 0 (put-bytes ( 10 11 12 13) :uint32 16) ; last-col-id simd-offset = =3D 16 (put-bytes (0.0 0.1 0.2 0.3) :sf 32) ; x simd-offset =3D 32 (put-bytes (1.0 1.1 1.2 1.3) :sf 48) ; y simd-offset =3D 48 ;;; populating frame 2 (put-bytes ( 4 5 6 7) :uint32 64) ; id simd-offset = =3D 0 (put-bytes ( 20 21 22 23) :uint32 80) ; last-col-id simd-offset = =3D 16 (put-bytes (0.4 0.5 0.6 0.7) :sf 96) ; x simd-offset =3D 32 (put-bytes (2.4 2.5 2.6 2.7) :sf 112) ; y simd-offset =3D 48 ;;; populating frame 3 (only first slot is used) (put-bytes ( 8 0 0 0) :uint32 128) ; id simd-offset = =3D 0 (put-bytes ( 30 0 0 0) :uint32 144) ; last-col-id simd-offset = =3D 16 (put-bytes (0.8 0.0 0.0 0.0) :sf 160) ; x simd-offset =3D 32 (put-bytes (3.8 0.0 0.0 0.0) :sf 176) ; y simd-offset =3D 48 set)) This is actually a vector that groups fields from adjacent struct in = frames of arbitrary size to accommodate any SIMD lanes number on the = host machine. This allows one easily examine and operate on individual = structs in a lisp runtime, and offload to a vector library bulk = processing of the data when needed. I have had much success with it, the = performance is double that of the vanilla structs and the memory usage = is about 8-10 times smaller, depending on the number of the fields in = the struct. While it certainly can be even faster if written in C, but it is fast = enough to be considered to be used in production for a lot of = applications that may have to do some number crunching. --Apple-Mail=_B13B509A-4277-47F4-ADA3-D94A91709456 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html aria-label=3D"message body"><head><meta http-equiv=3D"content-type" = content=3D"text/html; charset=3Dus-ascii"></head><body = style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: space; = line-break: after-white-space;"><br = id=3D"lineBreakAtBeginningOfMessage"><br><blockquote type=3D"cite">On 25 = Jun 2026, at 22:18, Tim Bradshaw (as tfb at cley dot com) = <[email protected]> wrote:<br><br = class=3D"Apple-interchange-newline">That's why I wrote my (unreleased, = may be abandoned) SOA thing.</blockquote><br>My custom struct thing is = live and kicking in the meantime. Working now on an upgrade that allows = it to lay data out in a simd friendly format while being able to access = structs in a sequential manner in lisp = seamlessly.<div><br><div><br></div><div><span style=3D"font-family: = "Courier New";"><b><div>;;; test simd set</div><div>;;; 9 = structs are populated across 3 4-element frames </div><div>;;; with = three slots empty in last frame</div><div>;;; id: = | 0 1 2 3| 4 5 6 = 7| 8 0 0 0|</div><div>;;; last-col-id:| = 10 11 12 13| 14 15 16 17| 18 0 = 0 0|</div><div>;;; x: |0.0 = 0.1 0.2 0.3|0.4 0.5 0.6 0.7|0.8 0 0 0|</div><div>;;; = y: |1.0 1.1 1.2 1.3|2.4 2.5 2.6 = 2.7|3.8 0 0 0|</div><div><br></div><div>;;; here we = write the struct field values into the test struct set = vector</div><div>;;; sys:typed-aref is used to store various fundamental = types in a single</div><div>;;; byte array that can be easily outsourced = to external libraries</div><div><div>(defmacro put-bytes (list type = offset &optional (by 4))</div><div> `(progn ,@(collect = nil</div><div> (do-list = (el list :count-var i :by by)</div><div> = (add `(put-byte set (+ ,offset ,i) ,el = ,type))))))</div><div><br></div><div>(defparameter = *simd-set-test*</div><div> (let ((set (make-byte-array (* 3 4 = 16)))) ; 3 frames 4x16 structs each</div><div> ;;; = populating frame 1</div><div> (put-bytes ( 0 1 = 2 3) :uint32 0) ; id = simd-offset =3D 0</div><div> (put-bytes ( 10 = 11 12 13) :uint32 16) ; last-col-id simd-offset = =3D 16</div><div> (put-bytes (0.0 0.1 0.2 0.3) :sf = 32) ; x simd-offset =3D 32</div><div> = (put-bytes (1.0 1.1 1.2 1.3) :sf 48) ; y simd-offset = =3D 48</div><div> ;;; populating frame 2</div><div> = (put-bytes ( 4 5 6 7) :uint32 = 64) ; id simd-offset =3D = 0</div><div> (put-bytes ( 20 21 22 23) = :uint32 80) ; last-col-id simd-offset =3D = 16</div><div> (put-bytes (0.4 0.5 0.6 0.7) :sf = 96) ; x simd-offset =3D 32</div><div> = (put-bytes (2.4 2.5 2.6 2.7) :sf 112) ; y = simd-offset =3D 48</div><div> ;;; populating frame 3 (only = first slot is used)</div><div> (put-bytes ( 8 = 0 0 0) :uint32 128) ; id = simd-offset =3D 0</div><div> (put-bytes ( 30 0 = 0 0) :uint32 144) ; last-col-id simd-offset =3D = 16</div><div> (put-bytes (0.8 0.0 0.0 0.0) :sf = 160) ; x simd-offset =3D 32</div><div> = (put-bytes (3.8 0.0 0.0 0.0) :sf 176) ; y = simd-offset =3D 48</div><div> = set))</div></div></b></span></div><div><br></div><div><br></div><div>This = is actually a vector that groups fields from adjacent struct in frames = of arbitrary size to accommodate any SIMD lanes number on the host = machine. This allows one easily examine and operate on individual = structs in a lisp runtime, and offload to a vector library bulk = processing of the data when needed. I have had much success with it, the = performance is double that of the vanilla structs and the memory usage = is about 8-10 times smaller, depending on the number of the fields in = the struct.</div><div><br></div><div>While it certainly can be even = faster if written in C, but it is fast enough to be considered to be = used in production for a lot of applications that may have to do some = number = crunching.</div><div><br></div><div><br></div><div><br></div></div></body>= </html>= --Apple-Mail=_B13B509A-4277-47F4-ADA3-D94A91709456-- _______________________________________________ Lisp Hug - the mailing list for LispWorks users [email protected] http://www.lispworks.com/support/lisp-hug.html