Re: feature request: utf encoding
Jens Seidel <[email protected]>
| Newsgroups | gmane.text.doxygen.devel |
|---|---|
| Message-ID | <20061124231218.GB15079@pluto> |
Hi Peter, On Fri, Nov 24, 2006 at 06:43:30PM +0100, Peter Münster wrote: > since the encoding of my source files is utf-8 since some time, would it be > possible to add utf support to doxygen? of course. It should be not very difficult. > For HTML, this means > <meta http-equiv="Content-Type" content="text/html;charset=utf-8"> > instead of > <meta http-equiv="Content-Type" content="text/html;charset=iso-8859-1"> I tried it once for German documents but did not complete it. Nevertheless the attached patch give should you sufficient ideas what needs to be changed. (translator_de.h needs to be recoded to UTF-8 as well, this is missing in the patch to keep it small.) HTML is trivial, but do not forget other output formats. LaTeX could cause trouble. I noticed failures for a special Doxygen configuration. I think it was the PDF output created via LaTeX which failed because of a missing escaping of "_" but I forgot about it. After manual fixing two or three escaping errors I was able to process a large document containing German and Russian texts. Manual pages should also not depend on the encoding (the manual page encoding is hardcoded into man. It may be necessary to recode these from UTF-8 via an external script on systems which expect classical encodings). I don't know the RTF format. Using UTF-8 per default would also allow to drop the stupid Use Windows encoding option in the config file. It's also still possible to use INPUT_FILTER = "iconv -f iso-8859-1 -t UTF-8 -c" for latin1 encoded source to keep these workings. Jens ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Doxygen-develop mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/doxygen-develop
doxygen.utf-8.diff
(text/plain, 3.8 KB)
Index: doc/doxygen.sty
===================================================================
RCS file: /cvsroot/doxygen/doc/doxygen.sty,v
retrieving revision 1.20
diff -u -r1.20 doxygen.sty
--- doc/doxygen.sty 7 May 2006 18:20:10 -0000 1.20
+++ doc/doxygen.sty 24 Nov 2006 23:00:28 -0000
@@ -30,6 +30,7 @@
\newenvironment{Code}
{\footnotesize}
{\normalsize}
+% FIXME: argument can still contain "_" outside math mode!
\newcommand{\doxyref}[3]{\textbf{#1} (\textnormal{#2}\,\pageref{#3})}
\newenvironment{DocInclude}
{\footnotesize}
Index: src/latexdocvisitor.cpp
===================================================================
RCS file: /cvsroot/doxygen/src/latexdocvisitor.cpp,v
retrieving revision 1.41
diff -u -r1.41 latexdocvisitor.cpp
--- src/latexdocvisitor.cpp 29 Oct 2006 18:39:27 -0000 1.41
+++ src/latexdocvisitor.cpp 24 Nov 2006 23:00:32 -0000
@@ -1042,6 +1042,7 @@
}
else if (ref.isEmpty()) // internal non-PDF link
{
+ // Doesn't work yet, since _ needs to be escaped
m_t << "\\doxyref{";
}
else // external link
Index: src/translator_de.h
===================================================================
RCS file: /cvsroot/doxygen/src/translator_de.h,v
retrieving revision 1.66
diff -u -r1.66 translator_de.h
--- src/translator_de.h 16 Jul 2006 20:10:06 -0000 1.66
+++ src/translator_de.h 24 Nov 2006 23:00:33 -0000
@@ -111,6 +111,11 @@
// 2006/06/12 Jens Seidel ([email protected])
// - Updated for "new since 1.4.6" version
//
+// 2006/07/17 Jens Seidel ([email protected])
+// - Switch to UTF-8 instead of latin1 everywhere
+// It's possible to use INPUT_FILTER = "iconv -f iso-8859-1 -t UTF-8 -c"
+// for latin1 encoded source
+//
// Todo:
// - see FIXME
@@ -138,15 +143,15 @@
virtual QCString latexLanguageSupportCommand()
{
QCString result="\\usepackage{ngerman}\n";
- result+="\\usepackage[latin1]{inputenc}\n";
- //result+="%\\usepackage[latin1]{inputenc}% Kodierung (cp850,latin1,ansinew)\n";
+ //result+="\\usepackage[latin1]{inputenc}\n";
+ result+="\\usepackage[utf8]{inputenc}% Kodierung (utf8,cp850,latin1,ansinew)\n";
return result;
}
/*! return the language charset. This will be used for the HTML output */
virtual QCString idLanguageCharset()
{
- return "iso-8859-1";
+ return "UTF-8";
}
// --- Language translation methods -------------------
Index: src/util.cpp
===================================================================
RCS file: /cvsroot/doxygen/src/util.cpp,v
retrieving revision 1.191
diff -u -r1.191 util.cpp
--- src/util.cpp 29 Oct 2006 18:39:27 -0000 1.191
+++ src/util.cpp 24 Nov 2006 23:00:39 -0000
@@ -5640,6 +5640,7 @@
bool insideTabbing,bool insidePre,bool insideItem)
{
static bool isCzech = theTranslator->idLanguage()=="czech";
+ static bool isGerman = theTranslator->idLanguage()=="german";
static bool isJapanese = theTranslator->idLanguage()=="japanese" ||
theTranslator->idLanguage()=="japanese-en";
static bool isKorean = theTranslator->idLanguage()=="korean" ||
@@ -5672,7 +5673,7 @@
default:
{
// Some languages use wide characters
- if (c>=128 && (isJapanese || isKorean || isChinese))
+ if (c>=128 && (isJapanese || isKorean || isChinese /*|| isGerman ?? */))
{
t << (char)c;
if (*p)
@@ -5737,7 +5738,7 @@
default:
{
// Some languages use wide characters
- if (isJapanese || isKorean || isChinese)
+ if (isJapanese || isKorean || isChinese || isGerman)
{
if (c>=128)
{