Re: Charset converting

Rune Saetre wrote:

>Hi
>
>I just want to remind you that if there is no charset specified in the
>HTTP Content-Type header this means that the document is encoded using the
>iso-8859-1 charset.
>
>The RFC is explicitely states that if no charset is indicated in the
>Content-Type header the recipient should not try to guess the charset, but
>the charset is in fact iso-8859-1.
>
>Of course, "web applications" might not know what charset the webserver
>would announce in the Content-Type header, so it makes sense to let this
>be overridden in the document preamble. (I guess this is not strictly
>legal regarding to the HTTP RFC...)
>
>But once again, if no charset is indicated whatsoever, then the document
>should be interpreted according to the iso-8859-1 charset, and NOT utf-8
>(or whatever).
>
>Here in Norway we have a lot of servers not indicating any charset at all,
>but they usually do send out content encoded with the iso-8859-1 charset,
>and is thus RFC compliant.
>
>These sites did not to work with kannel earlier, so I sent a patch some
>time ago to fix this. It should now be in CVS.
>
>It wouldn't do much good for speed, but if conversion fails using the
>implied iso-8859-1 charset, one could perhaps try another conversion based
>on a "qualified guess" at the charset? In this way Kannel would be RFC
>compliant so far as the documents are, and then still be able to process
>non-RFC compliant documents with meaningfull content.
>
>  
>
Yes, you are right, but 90% popular wap resources is NON-RFC complaint :((

May be iso-8859-1 charset must be set in wml_compile (wml_compiler.c) ?
like:

    if (charset && octstr_len(charset)) {
        if(octstr_case_compare(charset, octstr_imm("UTF-8")) == -1) {
        debug("wml_compile", 0, "WML compiler: Transcoding from <%s> to 
UTF-8",
              octstr_get_cstr(charset));
        set_charset(wml_text, charset);
        }
    } else {
        /* If charset not detected assume ISO-8859-1 encoding */
        debug("wml_compile", 0, "WML compiler: Unknown encoding, 
assuming ISO-8859-1");
        debug("wml_compile", 0, "WML compiler: Transcoding from 
ISO-8859-1 to UTF-8");
        set_charset(wml_text, octstr_create("ISO-8859-1"));
    }


-- 
 
Best regards,
Yury Mikhienko.
IT ERP group head, ZAO "Mobikom-Kavkaz"
mob: +7 (928) 2010199
tel: +7 (863) 2704188