Re: gauche.bitvector bug with somewhat bigger vectors (2^31)
Shiro Kawai <[email protected]> Thu, 7 May 2026 23:49:37 -1000
| Newsgroups | gmane.lisp.scheme.gauche |
|---|---|
| Message-ID | <CALN0JNEng96d5otzhZxY8BVjf4xxggfE-7KLmuyrrQvy1QWsTQ@mail.gmail.com> |
--===============2114069676305915196== Content-Type: multipart/alternative; boundary="0000000000004765aa06514b512d" --0000000000004765aa06514b512d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks for the catching Scm_MakeBits. My reasoning to switch to ScmSmallInt is that, if we expose those indexes to Scheme and allow values beyond fixnum range, unboxing would be cumbersome. ScmBits isn't directly exposed to Scheme so technically it can be used internally with the size beyond fixnum range, but I think it's unlikely on 32bit systems. You brought up an important point, though. 64bit Windows/MinGW is LLP64, so its long is still 32bits; we need long long to get 64bits. So their fixnum is 30bits. I think I originally kept it so that we can catch issues caused by different fixnum sizes. However, now that 64bit systems are norm, we may want to switch to use 62bit fixnum on Windows as well. For the time being, I keep ScmBits with ScmSmallInt for the simplicity. --shiro On Thu, May 7, 2026 at 8:44=E2=80=AFPM Jens Thiele <[email protected]> wrote: > Shiro Kawai <[email protected]> writes: > > > It may be a simple overlook, or pretty old before we start using > > ScmSmallInt. Anyway, I updated the code so that bit indexes are all > > ScmSmallInt. This may require recompiling extensions if they use bits.= h. > > I wasn't sure about that one. Maybe you wanted to keep int on 32-bit > platforms? But everywhere else it already was ScmSmallInt and > make-bitvector used fixnum since at least 2022. > =3D> there already was the limit of 2^(32-3)-1 bits? > > (the ufixnum is quite new?) > > But I think you missed Scm_MakeBits and bitvector-length. After this > additional change: > > =3D=3D > > diff --git a/src/bits.c b/src/bits.c > index 6d362a208..962bc21d8 100644 > --- a/src/bits.c > +++ b/src/bits.c > @@ -40,7 +40,7 @@ > * Construct, copy, fill > */ > > -ScmBits *Scm_MakeBits(int numbits) > +ScmBits *Scm_MakeBits(ScmSmallInt numbits) > { > size_t nw =3D SCM_BITS_NUM_WORDS(numbits); > ScmBits *bits =3D SCM_NEW_ATOMIC_ARRAY(ScmBits, nw); > diff --git a/src/gauche/bits.h b/src/gauche/bits.h > index 4e18d0ba8..babc3381b 100644 > --- a/src/gauche/bits.h > +++ b/src/gauche/bits.h > @@ -52,7 +52,7 @@ > typedef u_long ScmBits; > > /* Allocates and returns a bitmap that can hold NUMBITS. Zero-cleared. = */ > -SCM_EXTERN ScmBits *Scm_MakeBits(int numbits); > +SCM_EXTERN ScmBits *Scm_MakeBits(ScmSmallInt numbits); > > #define SCM_BITS_NUM_WORDS(size) \ > (((size)+SCM_WORD_BITS-1)/SCM_WORD_BITS) > diff --git a/src/libvec.scm b/src/libvec.scm > index feb700810..6f9bca3b8 100644 > --- a/src/libvec.scm > +++ b/src/libvec.scm > @@ -668,7 +668,7 @@ > ;;; > > (define-cproc bitvector? (obj) ::<boolean> SCM_BITVECTORP) ;SRFI-178 > -(define-cproc bitvector-length (v::<bitvector>) ::<int> ;SRFI-178 > +(define-cproc bitvector-length (v::<bitvector>) ::<fixnum> ;SRFI-178 > SCM_BITVECTOR_SIZE) > > (define-cproc bitvector-any-value? (v::<bitvector> bit > > =3D=3D > > it seems to work: > gosh$ (sid-amd64-sbuild)karme@amalthea:/tmp/Gauche$ src/gosh -ftest -V > Gauche scheme shell, version 0.9.16_pre2 [utf-8,pthreads], > x86_64-pc-linux-gnu > (version "0.9.16_pre2") > (command "gosh") > (scheme.id gauche) > (languages scheme r5rs r7rs) > (encodings utf-8) > (website "https://practical-scheme.net/gauche") > (build.platform "x86_64-pc-linux-gnu") > (build.configure) > (build.gosh-version "0.9.16_pre2") > (scheme.path "/tmp/Gauche/src/../lib" "/tmp/Gauche/src/../libsrc" > "/tmp/Gauche/src/../src" "/usr/local/share/gauche-0.98/site/lib" > "/usr/local/share/gauche-0.98/0.9.16_pre2/lib") > (threads pthreads) > (gauche.net.tls) > gosh$ (use gauche.bitvector) > gosh$ (define bv (make-bitvector (ash 1 31))) > bv > gosh$ (bitvector-length bv) > 2147483648 > gosh$ (~ bv (- 2147483648 1)) > 0 > gosh$ (set! (~ bv (- 2147483648 1)) 1) > #<undef> > gosh$ (~ bv (- 2147483648 1)) > 1 > > after all those years, I still always get confused by the C integer > types/sizes. (especially that long on 64-bit windows is 32-bit and > 64-bit everywhere else? ...) But if I understand correctly on 32-bit > systems and 64-bit windows the size limit for bitvectors is 2^(32-3)-1 > and 2^(64-3)-1 everywhere else? > > jens > > PS: > I just saw that C23 introduced support for N-bit integers: > https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2763.pdf > but using that for ScmSmallInt likely would cause more trouble, than it > would help? > > > _______________________________________________ > Gauche-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/gauche-devel > --0000000000004765aa06514b512d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-size:large">Tha= nks for the catching Scm_MakeBits.<br>My reasoning to switch to ScmSmallInt= is that, if we expose those indexes to Scheme and allow values beyond fixn= um range, unboxing would be cumbersome.=C2=A0 =C2=A0ScmBits isn't direc= tly exposed to Scheme so technically it can be used internally with the siz= e beyond fixnum range, but I think it's unlikely on 32bit systems.<br><= br>You brought up an important point, though.=C2=A0 64bit Windows/MinGW is = LLP64, so its long is still 32bits; we need long long to get 64bits.=C2=A0 = So their fixnum is 30bits.=C2=A0 =C2=A0I think I originally kept it so that= we can catch issues caused by different fixnum sizes.=C2=A0 However, now t= hat 64bit systems=C2=A0are norm, we may want to switch to use 62bit fixnum = on Windows as well.</div><div class=3D"gmail_default" style=3D"font-size:la= rge"><br></div><div class=3D"gmail_default" style=3D"font-size:large">For t= he time being, I keep ScmBits with ScmSmallInt for the simplicity.</div><di= v class=3D"gmail_default" style=3D"font-size:large"><br></div><div class=3D= "gmail_default" style=3D"font-size:large">--shiro</div><div class=3D"gmail_= default" style=3D"font-size:large"><br><br></div></div><br><div class=3D"gm= ail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On T= hu, May 7, 2026 at 8:44=E2=80=AFPM Jens Thiele <<a href=3D"mailto:karme@= karme.de">[email protected]</a>> wrote:<br></div><blockquote class=3D"gmail= _quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204= ,204);padding-left:1ex">Shiro Kawai <<a href=3D"mailto:shiro.kawai@gmail= .com" target=3D"_blank">[email protected]</a>> writes:<br> <br> > It may be a simple overlook, or pretty old before we start using<br> > ScmSmallInt.=C2=A0 Anyway, I updated the code so that bit indexes are = all<br> > ScmSmallInt.=C2=A0 This may require recompiling extensions if they use= bits.h.<br> <br> I wasn't sure about that one. Maybe you wanted to keep int on 32-bit<br= > platforms? But everywhere else it already was ScmSmallInt and<br> make-bitvector used fixnum since at least 2022.<br> =3D> there already was the limit of 2^(32-3)-1 bits?<br> <br> (the ufixnum is quite new?)<br> <br> But I think you missed Scm_MakeBits and bitvector-length. After this<br> additional change:<br> <br> =3D=3D<br> <br> diff --git a/src/bits.c b/src/bits.c<br> index 6d362a208..962bc21d8 100644<br> --- a/src/bits.c<br> +++ b/src/bits.c<br> @@ -40,7 +40,7 @@<br> =C2=A0 * Construct, copy, fill<br> =C2=A0 */<br> <br> -ScmBits *Scm_MakeBits(int numbits)<br> +ScmBits *Scm_MakeBits(ScmSmallInt numbits)<br> =C2=A0{<br> =C2=A0 =C2=A0 =C2=A0size_t nw =3D SCM_BITS_NUM_WORDS(numbits);<br> =C2=A0 =C2=A0 =C2=A0ScmBits *bits =3D SCM_NEW_ATOMIC_ARRAY(ScmBits, nw);<br= > diff --git a/src/gauche/bits.h b/src/gauche/bits.h<br> index 4e18d0ba8..babc3381b 100644<br> --- a/src/gauche/bits.h<br> +++ b/src/gauche/bits.h<br> @@ -52,7 +52,7 @@<br> =C2=A0typedef u_long ScmBits;<br> <br> =C2=A0/* Allocates and returns a bitmap that can hold NUMBITS.=C2=A0 Zero-c= leared. */<br> -SCM_EXTERN ScmBits *Scm_MakeBits(int numbits);<br> +SCM_EXTERN ScmBits *Scm_MakeBits(ScmSmallInt numbits);<br> <br> =C2=A0#define SCM_BITS_NUM_WORDS(size) \<br> =C2=A0 =C2=A0 =C2=A0(((size)+SCM_WORD_BITS-1)/SCM_WORD_BITS)<br> diff --git a/src/libvec.scm b/src/libvec.scm<br> index feb700810..6f9bca3b8 100644<br> --- a/src/libvec.scm<br> +++ b/src/libvec.scm<br> @@ -668,7 +668,7 @@<br> =C2=A0;;;<br> <br> =C2=A0(define-cproc bitvector? (obj) ::<boolean> SCM_BITVECTORP) ;SRF= I-178<br> -(define-cproc bitvector-length (v::<bitvector>) ::<int>=C2=A0 = =C2=A0 ;SRFI-178<br> +(define-cproc bitvector-length (v::<bitvector>) ::<fixnum>=C2= =A0 =C2=A0 ;SRFI-178<br> =C2=A0 =C2=A0SCM_BITVECTOR_SIZE)<br> <br> =C2=A0(define-cproc bitvector-any-value? (v::<bitvector> bit<br> <br> =3D=3D<br> <br> it seems to work:<br> gosh$ (sid-amd64-sbuild)karme@amalthea:/tmp/Gauche$ src/gosh -ftest -V<br> Gauche scheme shell, version 0.9.16_pre2 [utf-8,pthreads], x86_64-pc-linux-= gnu<br> (version "0.9.16_pre2")<br> (command "gosh")<br> (<a href=3D"http://scheme.id" rel=3D"noreferrer" target=3D"_blank">scheme.i= d</a> gauche)<br> (languages scheme r5rs r7rs)<br> (encodings utf-8)<br> (website "<a href=3D"https://practical-scheme.net/gauche" rel=3D"noref= errer" target=3D"_blank">https://practical-scheme.net/gauche</a>")<br> (build.platform "x86_64-pc-linux-gnu")<br> (build.configure)<br> (build.gosh-version "0.9.16_pre2")<br> (scheme.path "/tmp/Gauche/src/../lib" "/tmp/Gauche/src/../li= bsrc" "/tmp/Gauche/src/../src" "/usr/local/share/gauche= -0.98/site/lib" "/usr/local/share/gauche-0.98/0.9.16_pre2/lib&quo= t;)<br> (threads pthreads)<br> (gauche.net.tls)<br> gosh$ (use gauche.bitvector)<br> gosh$ (define bv (make-bitvector (ash 1 31)))<br> bv<br> gosh$ (bitvector-length bv)<br> 2147483648<br> gosh$ (~ bv (- 2147483648 1))<br> 0<br> gosh$ (set! (~ bv (- 2147483648 1)) 1)<br> #<undef><br> gosh$ (~ bv (- 2147483648 1))<br> 1<br> <br> after all those years, I still always get confused by the C integer<br> types/sizes. (especially that long on 64-bit windows is 32-bit and<br> 64-bit everywhere else? ...) But if I understand correctly on 32-bit<br> systems and 64-bit windows the size limit for bitvectors is 2^(32-3)-1<br> and 2^(64-3)-1 everywhere else?<br> <br> jens<br> <br> PS:<br> I just saw that C23 introduced support for N-bit integers:<br> <a href=3D"https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2763.pdf" rel= =3D"noreferrer" target=3D"_blank">https://www.open-std.org/jtc1/sc22/wg14/w= ww/docs/n2763.pdf</a><br> but using that for ScmSmallInt likely would cause more trouble, than it<br> would help?<br> <br> <br> _______________________________________________<br> Gauche-devel mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">Gau= [email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/gauche-devel" rel= =3D"noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listi= nfo/gauche-devel</a><br> </blockquote></div> --0000000000004765aa06514b512d-- --===============2114069676305915196== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============2114069676305915196== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Gauche-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/gauche-devel --===============2114069676305915196==--