Re: mixed encoding and Simpleentiy

On Jun 27, 2005, at 9:15, Jean-Christophe Michel wrote:


> Bas Kamer wrote
>
>
>> Simpleentity translates the data on lading from the datasource   
>> encoding
>> to the output incoding. I believe this is wrong.
>>
>> Firstly if there is a translator for getting data, then there should
>> also be one for putting it back in the database. But my main point is
>> that it should not translate to and from the output encoding but  
>> to  the
>> encoding that that particular Node is using. (Which is currently
>> always  UTF-8, see #219).
>>
>> Let me illustrate.
>>
>> We need to serve iso-8859-1 encoded html pages.
>> There is a database in which the data is encoded in iso-8859-1.
>>
>> When binarycloud starts it sets internal encoding set to utf-8 and
>> treats all variables as such. All templates, ndf, translation, php,
>> etc... are utf-8. Even POST/GET values are translated to utf-8 (not
>> cookies though)
>> Binarycloud then sets output encoding to whatever is desired
>> (iso-8859-1) and will initiate an output buffering session with the
>> iconv_ob_handler parameter, all output send to the browser will first
>> be encoded.
>>
>> Because SimpleEntity translates the data from the datasource encoding
>> to the output encoding a Node potentially can receive iso-8859-1   
>> data.
>> If a Node would only save that data back into the database  there  
>> is no
>> trouble, but when the data travels to the browser it is  first
>> translated from utf-8 to iso-8859-1 andd then return via a POST
>> operation, the then presumed iso data is translated back into  
>> utf-8  and
>> then that is being save as utf-8 in a iso-8859-1 database.
>>
>> It seems to make more sense to translate it to utf-8 before
>> SimpleEntity returns and data to the Node. Additionally it should   
>> then
>> do the oposite translate from utf-8 to datasource encoding while   
>> saving
>> data.
>>
>>
>
> You're right; I already changed
> binarycloud/creole/drivers/mysql/MySQLResultSet.php and  to convert  
> all
> encodings whenever db encoding is not utf8. So converting in
> SimpleEntity too is useless.
>

Ah, no wonder I am getting unexpected results... Please have a look  
because not all data is translated before getting it out of The  
Result Set. Only data that is retrieved via getString is translated.

Also,

InMySQLConnection::connect you do;
    $this->dsn['output_encoding'] = $i18n->getOutputCharacterEncoding();

i think;
    $this->dsn['output_encoding'] = 'utf-8';

would be better. see the rest of my mail. what do you think of that?

creole should honor this;
getting : db encoding (iso-8859-1) => utf-8
putting : utf-8 => db_encoding (iso-8859-1)




> Concerning cookies, I didn't see wether we need to convert it to  
> utf8 or
> not. As I use it to store session id only, I'll probably not see the
> need of a more advanced conversion :-)
>
> ANother point: didn't you mention the need to specify an 'encoding'  
> for
> each node, and have each node convert its children's output from their
> encoding to its own ? (Though personnally I find it simpler to decide
> that all vortex nodes should output utf8, and eventually use iconv  
> once
> in htdocs/* file to convert to another specific encoding if necessary)
>

see: #219


_______________________________________________
dev mailing list
dev-PnctHDZWAvB/Cz2I37pSEPZ4XP/[email protected]
http://lists.binarycloud.com/mailman/listinfo/dev