Note for picfolio stylesheet authors--character encoding
sd <[email protected]> Sat, 15 May 2004 07:52:56 -0600
| Newsgroups | gmane.comp.web.picfolio.user |
|---|---|
| Message-ID | <[email protected]> |
Hello all--
For the sake of other picfolio stylesheet authors I thought I would
share a note about a particular issue involving character encoding.
A user of the chocolate_cream stylesheets I created sent me an email
describing an issue where certain strange characters were appearing
in his image gallery (such as many capital A's with circumflexes
above them).
The problem, I eventually discovered, was caused by the pages of his
gallery being displayed using the ISO-8859-1 character encoding
rather than the UTF-8 character encoding that my stylesheets specify.
The reason for that is that the webserver (Apache) that served the
HTML pages of his gallery was configured to specify, in the HTTP
headers that it sent with each file, a character encoding of
ISO-8859-1, which overrode the character encoding specified in the
HTML pages themselves (UTF-8).
Both the issue and the solution I found are described in an email I
wrote to this user of the chocolate_cream stylesheets, which is
quoted directly below, followed by a note on how I altered my XSLT
stylesheets to (hopefully) resolve the problem.
--- begin quote of email to chocolate_cream picfolio style user ---
Hello [name omitted]--
This is Steve D, author of the XSLT stylesheets for picfolio called
chocolate_cream. You had mentioned, a week or two ago, an issue with
the stylesheets in which certain characters were not being displayed
correctly in a picfolio gallery. It turned out that the reason for
this is that the stylesheets specified a character encoding of UTF-8
(one of the Unicode encodings that includes ASCII and
"western-Latin"), and the web server that was serving your gallery
was overriding the documents' own character coding with a default
encoding of ISO-8859-1 ("Western Latin," or "Latin 1")
This caused a non-breaking space, for example, to appear as a capital
A with a circumflex above it, if I remember correctly.
Your workaround for this problem was to change every instance of
"UTF-8" in the XSLT stylesheets to "ISO-8859-1" and then to run
picfolio again so that the resulting gallery pages would be written
using the same encoding that your web server specified. This was a
good workaround.
However, when I approached the problem, I wanted to keep UTF-8
encoding for maximum flexibility, forward AND backward compatibility,
and so that people who speak non-English, non Romance languages can
use the stylesheets hopefully with only minimal problems.
I found a solution that I believe will achieve both the aims expressed
in the paragraph above, as well as the problem that happens (as it
did to you) when a web server overrides the document-specified
encoding and serves the document as ASCII or Western-Latin
(ISO-8859-1).
The solution was to cause the XSLT stylesheets to produce certain
characters (the problematic ones) as character references (such as
 ) instead of directly encoding them in UTF-8 within the final
HTML documents that picfolio creates. The reason this works is that
the character that occupies that particular numerical position
(hexadecimal A0 or decimal 160) is exactly the same whether the
encoding is UTF-8 or ISO-8859-1 or ASCII. The Unicode consortium took
great pains to ensure that the numerical-index (the position of a
character in a character set) of UTF-8 kept both the ASCII (7-bit
English) and ISO-8859-1 (Western-Latin-1) characters in their same
exact locations within the UTF-8 character set or table, which
contains many more characters than both ASCII and ISO-8859-1.
I have updated the stylesheets, available at:
http://www.xscd.com/pub/picfolio-styles/
They now address and hopefully remedy the problem you had. Thank you
very much for bringing the issue to my attention.
--- end of quote of email to chocolate_cream picfolio style user ---
NOTE for picfolio stylesheet authors
------------------------------------
To address this problem, in my picfolio stylesheets I enclosed
non-ASCII characters (such as a "bullet" or non-breaking space) in an
<xsl:text></xsl:text> tag or instruction with the attribute
disable-output-escaping set to "yes." I caused the non-ASCII
character to be written to the final output pages as a character
reference (such as  ) rather than being directly written as the
character itself, in whatever encoding was specified by the
stylesheets (in this case, UTF-8).
To cause a non-breaking space to apear as a character-reference in the
HTML or XHTML pages, I wrote it like this in my XSLT stylesheet:
<xsl:text disable-output-escaping="yes">&#xA0;</xsl:text>
The 'disable-output-escaping="yes"' causes the "&" to be output as
simply an ampersand (&), and the following text (#xA0;) is output
exactly as it is written in the XSLT stylesheet, causing the
"&#xA0;" to appear as " " (without the quotes, of course) in
the final document--a character reference that specifies the
numerical index in ASCII and ISO-8859-1 and UTF-8 for a non-breaking
space, which occupies that position in all of those
character-encodings.
Problem 1 --
" " specified within the <xsl:text> instruction produces an error
in the XSLT processor because of the literal ampersand in it.
Problem 2 --
"&#xA0;" specified within the <xsl:text> instruction, without
'disable-output-escaping="yes",' produces that exact same character
string (&#xA0;) in the final HTML or XHTML document because
first, the XSLT processor reads the "&" as a literal ampersand
(&), then "output-escapes" it into a reference again (&) when it
includes it in the final (X)HTML document. And of course, when a web
browser sees the character string "&#xA0;" in a document, it
displays it as the characters " " (substituting an ampersand for
the reference &) instead of substituting the character that such
a character sequence references (in this case, a non-breaking space).
SO, to get a web browser to produce a literal non-breaking space, the
(X)HTML document must contain the character sequence   (the
actual character reference for a non-breaking space) rather than
&#xA0; by using the
<xsl:text disable-output-escaping="yes">&#xA0;</xsl:text>
method described above. The same goes for other "strange" characters
that one wishes to include in the final (X)HTML output documents.
This is very confusing to try to explain, but I hope that this note
has been sufficient.
Best wishes everyone,
Steve D
New Mexico, US
--
----------------------------------------------------------------
The ships hung in the sky in much the same way that bricks don't.
-Douglas Adams
----------------------------------------------------------------