Re: Read-sequence is slow?

Raymond Toy <[email protected]> Tue, 14 Apr 2026 16:59:22 -0700
Newsgroups gmane.lisp.gcl.devel
Message-ID <[email protected]> 
This is a multi-part message in MIME format.
--------------8i0Yt474Ff3iXEkiX70yQO6O
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit


On 4/13/26 10:40 AM, Camm Maguire wrote:
> Greetings, and thank you so much for bringing this to my attention!  I
> have committed a fix to branch 2_7_2pre, which will be in the
> forthcoming debian package being built now.
Thanks for the fix.  My simple slurp-bytes test with maxima.info-1 
(1030671 bytes) goes from 19.1 sec with gcl 2.7.1 to 0.090 sec with this 
fix.  With `si:fread`, it's 0.01 sec with the latest gcl.  So there is 
some room for improvement.  But going from 19.1 to 0.09 is a huge 
improvement and fast enough for me now.
>
> Right now read-sequence loops over read-byte when appropriate, so could
> be accelerated further with fread, but I doubt you will be able to see
> much difference.  If using fread, we will have to duplicate the logic in
> read-byte concerning the stream-element-type, but that is just ugly and
> could be done.  One thing I've always wanted in GCL is an interface to
> my favorite Linux file I/O facility, mmap.
>
> Take care,
>
> Raymond Toy<[email protected]> writes:
>
>> On 4/8/26 9:21 AM, Raymond Toy wrote:
>>
>>   I have this routine:
>>
>>
>> (defun slurp-bytes (filename)
>>    "Return the entire contents of FILENAME as a (unsigned-byte 8) vector.
>>     Results are cached in *bytes-cache* so each file is read only once."
>>    (or (gethash filename *bytes-cache*)
>>        (setf (gethash filename *bytes-cache*)
>>              (with-open-file (s filename :direction :input
>>                                          :element-type '(unsigned-byte 8))
>>                (let* ((len (file-length s))
>>                       (buf (make-array len :element-type '(unsigned-byte 8))))
>>                  (read-sequence buf s)
>>                  buf)))))
>>
>>   When reading maxima.info-1 (1030671 bytes), it takes a very long time (several seconds). Is there anyway to speed this up?
>>   I would have expected reading a bunch of bytes from a file via read-sequence would be very fast.
>>
>> Some googling pointed me to si:fread. It makes slurp-bytes really fast taking tiny fractions of a sec to read i a 1 MB file.
>> Having read-sequence use si:fread for element types of character or (unsigned-byte 8) would be a really good thing.
>>
>> &#8203;
>>
--------------8i0Yt474Ff3iXEkiX70yQO6O
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 4/13/26 10:40 AM, Camm Maguire
      wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:[email protected]">
      <pre wrap="" class="moz-quote-pre">Greetings, and thank you so much for bringing this to my attention!  I
have committed a fix to branch 2_7_2pre, which will be in the
forthcoming debian package being built now.</pre>
    </blockquote>
    Thanks for the fix.  My simple slurp-bytes test with maxima.info-1
    (1030671 bytes) goes from 19.1 sec with gcl 2.7.1 to 0.090 sec with
    this fix.  With `si:fread`, it's 0.01 sec with the latest gcl.  So
    there is some room for improvement.  But going from 19.1 to 0.09 is
    a huge improvement and fast enough for me now.
    <blockquote type="cite" cite="mid:[email protected]">
      <pre wrap="" class="moz-quote-pre">

Right now read-sequence loops over read-byte when appropriate, so could
be accelerated further with fread, but I doubt you will be able to see
much difference.  If using fread, we will have to duplicate the logic in
read-byte concerning the stream-element-type, but that is just ugly and
could be done.  One thing I've always wanted in GCL is an interface to
my favorite Linux file I/O facility, mmap.

Take care,

Raymond Toy <a class="moz-txt-link-rfc2396E" href="mailto:[email protected]">&lt;[email protected]&gt;</a> writes:

</pre>
      <blockquote type="cite">
        <pre wrap="" class="moz-quote-pre">On 4/8/26 9:21 AM, Raymond Toy wrote:

 I have this routine:


(defun slurp-bytes (filename)
  "Return the entire contents of FILENAME as a (unsigned-byte 8) vector.
   Results are cached in *bytes-cache* so each file is read only once."
  (or (gethash filename *bytes-cache*)
      (setf (gethash filename *bytes-cache*)
            (with-open-file (s filename :direction :input
                                        :element-type '(unsigned-byte 8))
              (let* ((len (file-length s))
                     (buf (make-array len :element-type '(unsigned-byte 8))))
                (read-sequence buf s)
                buf)))))

 When reading maxima.info-1 (1030671 bytes), it takes a very long time (several seconds). Is there anyway to speed this up?
 I would have expected reading a bunch of bytes from a file via read-sequence would be very fast.

Some googling pointed me to si:fread. It makes slurp-bytes really fast taking tiny fractions of a sec to read i a 1 MB file.
Having read-sequence use si:fread for element types of character or (unsigned-byte 8) would be a really good thing.

&amp;#8203;

</pre>
      </blockquote>
      <pre wrap="" class="moz-quote-pre">
</pre>
    </blockquote>
    <ul class="dropdown-menu textcomplete-dropdown"
      style="display: none; position: absolute; z-index: 1000;"
      contenteditable="false" popover="auto">
    </ul>
  </body>
</html>

--------------8i0Yt474Ff3iXEkiX70yQO6O--