character set encoding of scheme source files
Jonathan Rees <[email protected]> Fri, 26 Oct 2012 17:01:04 -0400
| Newsgroups | gmane.lisp.scheme.scheme48 |
|---|---|
| Message-ID | <[email protected]> |
Mike: I noticed the presence of an apparent u-umlaut in your recent Scheme 48 sources. This raises an interesting question for me: How are various parts of the ecosystem supposed to know what the character set encoding of scheme files is? The question hasn't come up much previously because Scheme is so retro (or provincial) that most source files so far are encoded in 7-bit ASCII, which is a subset shared among UTF-8, Latin-1, etc., so until now it just hasn't mattered. This is a practical question, I don't mean to be pedantic. My emacs displayed the source in what I think was the intended way, but it feels like an accident that it did so. What if it had treated the source file as UTF-8 or Latin-5? What if it had paid attention to the Unix locale, and the locale specified an encoding different from what was intended? Similarly, there's the question of how the encoding is (or should be) determined by Scheme. I haven't looked at the Unicode support in Scheme 48 yet, but I'm guessing that the problem of deciding on the encoding is pushed off to the user/programmer. Maybe that's OK, but the problem remains, how is the user/programmer supposed to decide what the encoding is, in a 'best practice' sense? XML and N3 have answers to these questions: XML says (IIUIC) the encoding can be determined by reading the first few bytes of the file assuming 7-bit ASCII; having decoded that you can decode the rest (because the initial text tells you the encoding). N3 just says that the source file is in UTF-8, period. HTTP also has an answer: If content comes in an HTTP message the answer is given by the HTTP headers and if necessary the media type documentation (the story differs between text/ and application/, etc.). So what does Scheme (or R6RS Scheme or R7RS Scheme or Scheme-48 or folklore) say about this? Just wondering, too lazy to dig around to find the answers, and hoping you'll brief me so I don't have to dig. Pointers to documentation would be a fine answer to my question. Thanks Jonathan