I18n issues with 4.1.1 and 5.0

Alexander Kirillov <[email protected]>
Newsgroups gmane.comp.db.mysql.bugs
Message-ID <[email protected]>
Hi guys and Happy Easter to all of you,
I've tried to upgrade to either 4.1.1 or 5.0 recently and spent a week
trying to make my apps work with 4.1.1 or 5.0 and than downgrading back 
to 4.0.18.
That's a pity it nearly worked and seems so easy to fix and still 
rendered  my apps practically unusable.
Now back to details. I have a bunch of database, table and column names 
in cyrillic (koi8r).
Generally it's a bad practice to use non-ascii names but offers an easy 
localization of
some db-driven apps. The main problem is there's no specific encoding 
assosiated
with user, database, table and column values in system tables. They are 
now all utf8 internally
but still mysql_install_db declares all these fields BINARY. If client 
encoding is anything else but utf8
you won't get readable results doing SELECTs on system tables.
What is more the behaviour of various administrative commands which 
retrieve or set user, db ot table values
in system tables like GRANT/SHOW GRANTS, SHOW CREATE TABLE, SHOW BINLOG 
EVENTS and so on is inconsistent.
For example you use GRANT smth on a table with cyrillic name and it 
works and you can SELECT values
from mysql.db and get meaningful results IF(!) your client encoding is 
set to UTF8.
But SHOW GRANTS yields unreadable results no matter what your client 
encoding is.
All cyrillic names in SHOW GRANTS output are screwed up. I've tried 
different combinations
of client encodings and encodings explicitly set for user/db/table 
fields in system tables. No luck.
Same stands for master binary logs where a slave chokes with every query 
with non-ascii db/table name.
Viewing the logs (SHOW BINLOG EVENTS) one may notice the queries are OK 
and show up properly converted to encoding specified by the client. But 
all USE `db` statements inserted by the master are in God knows what 
encoding
and show up as meaningless rubbish.
I would suggest to explicitly declare user, db, table and column fields 
in system tables to be utf8  and
use normal character set conversion rules when fetching data for the 
client (it seems to be working well for userland tables).
A slave can always request the data to be in utf8 format and this seems 
to be compatible
with new MySQL layout where each column may use different encoding.
If the requested data can't be converted to the encoding specified by 
the client MySQL could either fetch a binary value
or produce a fatal runtime error which is probably a better choice.
Thanks for your help and looking forward for fully working i18n support 
in MySQL.
Cheers,
Sasha

PS
I didn't mention minor issues like error messages for languages other 
than english are not translated to client encoding,
Also MySQL server treats all file names as utf8 (LOAD DATA, SELECT * 
INTO OUTFILE) even if
local file sytem uses different encoding. But that's a real pain in the 
tail if you are trying to rebuild
your database from pre 4.1 dumps.


-- 
MySQL Bugs Mailing List
For list archives: http://lists.mysql.com/bugs
To unsubscribe:    http://lists.mysql.com/[email protected]
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.