RE: How about converting Webpage to UTF-8 before tidying process ?

I agree that character set conversion should be done outside the Tidy
processor. This would both simplify the internal Tidy code and 

extend its usefulness. Since the Tidy code already uses input/output
streams and ReadChar()/WriteChar() functions, it's not really a 

big change.

According to the W3C, HTML and XML authors should "choose UTF-8 or
another Unicode encoding for all content" 

[http://www.w3.org/International/geo/html-tech/tech-character.html#ri200
30112.213746362] UTF8 gives true byte character encoding and 

the best performance, but adds a lot of overhead for non-European
languages. Unicode supports these languages better, but adds the 

pain of multibyte character encodings (except in 32 bit Windows, where
it's native and a bit less painful).

Suggestion:
1 use a generic character conversion layer between the outside and Tidy
2 use UTF8 and Unicode for all internal text (depending on platform,
options)

see: Authoring Techniques for XHTML & HTML Internationalization 

[http://www.w3.org/International/geo/html-tech/outline/html-authoring-ou
tline.html]
  _____  



RE: [Tidy-dev] How about converting Webpage to UTF-8 before tidying
process ?

Hi Ben,

If you are interested, in the firefox extension of Tidy
(http://users.skynet.be/mgueury/mozilla)
I do exactly this, but I use the firefox/mozilla library to do it.

It does about something like that:

firefox / original charset --> firefox utf8 conv --> tidy --> then it
shows the output webpage 
in unicode rendered by Firefox + the source code.

Then if the user wants, he can copy paste the result in his favorite
editor. Due that
he will do that on a OS using the charset he wants, he do not need to
think about
saving it in the right charset, it will be the default one.

If not, he needs to take care about choosing the right charset when
saving with his editor.

Marc

----------------------------------------
Hi, 

I am thinking that tidy can support more encodings if we make use of the
GNU
iconv lib. We use iconv to convert any encoding of webpage into UTF-8
first
and then do the tidying process. F! inally, we convert the webpage back
to its
original encoding. 

The flow would be: 

input webpage --> iconv --> tidy --> iconv --> output webpage


How do you think of this idea ? 

Best Regards
-Ben (Benfeng CHEN)




-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
Tidy-develop mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/tidy-develop





  _____  

Sent from NewsGator Online <http://www.newsgator.com> .