byte order marker puzzle

Stephen Lewis <[email protected]> Tue, 5 Dec 2023 17:23:41 -0800
Newsgroups gmane.lisp.scheme.gauche
Message-ID <[email protected]>
Shiro,
I am having trouble with a text file that has a BOM
(byte order mark) at the beginning. 'read-line'
reads a string which appears to contain a number
'print' can print the number, but 'string->number'
returns '#f'. It seems as if the BOM is somehow
visible to 'string->number'.
Here is a short test program and text file that
shows the problem. Is this expected behaviour of
'read-line' and/or 'string->number'? Many programs
treat the BOM like a magic number and read the
file after the BOM. Am I doing something wrong?
Stephen Lewis

====================================================
;;;
;;; Odd behaviour of 'string->number' if BOM is present
;;;
;;; Stephen Lewis 3-dec 2023
;;;

(define (usage program-name)
  (format (current-error-port)
          "Usage: gosh ~a ./bom.txt\n" program-name)
  (exit 2))

(define (do-test args)
  (let1 fn (cadr args)
        (print "Input file \"" (string-scan-right fn "/" 'after) "\"")
        (call-with-input-file fn get-numbers)
        ))

(define (get-numbers port)
(until (read-line port) eof-object? => line
    (print "Line contains \"" line "\"")
    (print "Line is a string? " (string? line))
    (print "Parses as number " (string->number line))))

(define (main args)
(if (eq? (length args) 2)
  (do-test args)
  (usage (car args)))
0)
====================================================

This generates the test input file:
echo "77u/MQ0KMg0KMw0K" | base64 -d > bom.txt

Gauche version:
gosh -V
Gauche scheme shell, version 0.9.12 [utf-8,pthreads], x86_64-pc-linux-gnu
(version "0.9.12")
(command "gosh")
(scheme.id gauche)
(languages scheme r5rs r7rs)
(encodings utf-8)
(website "https://practical-scheme.net/gauche")
(build.platform "x86_64-pc-linux-gnu")
(build.configure "--prefix=/usr" "--build=x86_64-pc-linux-gnu" "--host=x86_64-pc-linux-gnu" "--mandir=/usr/share/man" "--infodir=/usr/share/info" "--datadir=/usr/share" "--sysconfdir=/etc" "--localstatedir=/var/lib" "--datarootdir=/usr/share" "--docdir=/usr/share/doc/gauche-0.9.12" "--htmldir=/usr/share/doc/gauche-0.9.12/html" "--libdir=/usr/lib64" "--enable-ipv6" "--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt" "--with-slib=/usr/share/slib" "--with-tls=mbedtls" "build_alias=x86_64-pc-linux-gnu" "host_alias=x86_64-pc-linux-gnu" "CFLAGS=-O2 -pipe -mtune=native -march=native" "LDFLAGS=-Wl,-O1 -Wl,--as-needed" "--enable-multibyte=utf-8")
(scheme.path "/usr/share/gauche-0.98/site/lib" "/usr/share/gauche-0.98/0.9.12/lib")
(gauche.threads pthreads)
(gauche.net.tls mbedtls)