Re: mysqlIndexerTest.test_unicode fails for me

Hi Joseph:

In message <[email protected]>,
Joseph Myers writes:
>On Thu, 30 Apr 2020, Ralf Schlatterbeck wrote:
>> In roundup/backends/back_mysql.py we had a condition on python version
>> >2 for supplying the charset (utf-8) argument when creating a mysql
>> connection. I've changed this to an unconditional initialization. Works
>
>That condition was so that a database created with older Roundup, with 
>Python 2, continues to work properly with newer Roundup, with Python 2.  
>(I couldn't get such a database to work with Python 3, hence the 
>documentation of needing to export and import when moving to Python 3 with 
>MySQL.)
>
>The key thing to test is: create a database with older Roundup (e.g. 1.6) 
>with Python 2 and MySQL (on a default MySQL installation, so with the 
>latin1 default).  Create items in the database with string fields stored 
>in the database (issue title, etc.) that contain Unicode characters that 
>cannot be represented in latin1 (this works fine - the individual bytes of 
>the UTF-8 representation are stored as if they were latin1 characters).  
>Then try using that database with Roundup with your change, still with 
>Python 2, and make sure that the Unicode characters previously put in 
>those string fields still appear correctly (my expectation is that they 
>won't, because MySQL will try to translate from latin1 to UTF-8).

This makes sense and it probably should remain latin1 for backwards
compatibility. Maybe it's something that could be configured using
config.ini [rdbms] key. But given the need to dump/restore when moving
to python3 maybe it doesn't need to be configurable unless it needs to
match the server's config.

However Joseph I am confused by Ralf's issue. The test should create a
new database and database connection. So latin1 should work for
python2 and does work in the CI platforms. Yet it doesn't.

Could my.cnf be overriding the connection encoding server side?
Forcing utf8 on the server side while the client uses
latin1. Everything would work as long as the representations in latin1
and ut8 were the same right?

So you could have a mysql instance using utf8 even under python2. It
just depends on how the mysql server is configured.

Thoughts on my musings?

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.