fast serialization of huge bit vectors?

Jens Thiele <[email protected]> Tue, 28 Apr 2026 15:11:35 +0200
Newsgroups gmane.lisp.scheme.gauche
Message-ID <[email protected]>
Hi, just some minor thoughts:

I just found gauche.bitvector and SRFI-178. Until now I used a small
hack based on u8vectors for huge bit vectors. Using u8vectors I simply
used write-uvector and port->uvector for serialization (not sure whether
using port->uvector is really clever, maybe I should have added a size
header ...)

Now I wanted to get rid of that hack and switch to gauche.bitvector
and/or SRFI-178, but I didn't find a good way to read/write a bitvector
as binary data. Likely this isn't by accident but to prevent
byte-order/word size/padding ugliness? Or did I miss something obvious?

jens

PS: playing around:
gosh> (use gauche.uvector)
gosh> (use gauche.bitvector)
gosh> (use gauche.generator)
;; for comparision test with u8vector
gosh> (define v (make-u8vector (ash 1 29) 0))
v
gosh> (time (with-output-to-file "/tmp/v" (cut write-uvector v)))
;(time (with-output-to-file "/tmp/v" (cut write-uvector v)))
; real   2.718
; user   2.490
; sys    0.220
gosh> (define bv (make-bitvector (ash 1 29) 0))
bv
;; first very slow try to dump bitvector to file (no size/padding info stored)
gosh> (time (with-output-to-file "/tmp/v" (lambda() (generator-for-each (compose write-byte bitvector->integer (cut apply bitvector <>)) (ggroup (bitvector->int-generator bv) 8)))))
;(time (with-output-to-file "/tmp/v" (lambda () (generator-for-each (com ...
; real 167.931
; user 180.650
; sys    0.370