DBD::ODBC 1.46_1 released - You REALLY need to test this release

[email protected] ("Martin J. Evans") Sat, 16 Nov 2013 10:33:35 +0000
Newsgroups perl.dbi.dev,perl.dbi.users,perl.dbi.announce
Organization at home
Message-ID <[email protected]>
I've just uploaded DBD::ODBC 1.46_1 to the CPAN. In the process of 
writing 
http://www.easysoft.com/developer/languages/perl/sql-server-unicode.html 
and https://github.com/mjegh/dbd_odbc_sql_server_unicode I discovered a 
serious bug in the way DBD::ODBC can attempt to insert unicode 
characters into char/varchar/longvarchar columns. This experimental 
release fixes that issue but it does mean this release contains a 
significant change in behaviour.

The issue ONLY applied to unicode builds of DBD::ODBC (the default on 
Windows) and enabled in Linux/Unix via the -u switch to Makefile.PL.

The problem was that when inserting parameter data into 
char/varchar/longvarchar columns DBD::ODBC ignored what your parameter 
actually looked like and simply bound the parameter as the type 
described by the database (SQL_CHAR). This meant that if you bound data 
was unicode, the separate octets of the perl UTF-8 encoded data would be 
inserted instead of the unicode characters. A simple example illustrates 
this easiest:

Say you had a unicode euro in a perl scalar. This is U+20AC and is 
encoded in UTF-8 as 0xe2,0x82,0xc2. If you inserted into a 
char/varchar/longvarchar the database would receive it as 3 separate 
chrs instead of 1 i.e., select len(mycol) from mytable would return 3 
instead of 1.

There are a few situations when this did not apply 1) if you overrode 
the bind type with SQL_WVARCHAR 2) if your ODBC driver did not support 
SQLDescribeParam or you told DBD::ODBC not to use it.

A new test (45_unicode_varchar.t which has high verbosity set right now) 
has been added to the test suite. Unfortunately, this test only runs to 
MS SQL Server right now. If this test does not pass for you please 
report it and the output to me as soon as possible.

You are strongly advised to test this release with your development 
environment as I've not implemented a deprecation policy for this change 
as yet. I'm hoping to release a full version as 1.46_1 is, BUT if it is 
reported to me that this will cause too many people problems I'll 
reconsider.

I full description of the issues and other problems can be found at 
http://www.easysoft.com/developer/languages/perl/sql-server-unicode.html 
and https://github.com/mjegh/dbd_odbc_sql_server_unicode.

Here is a full list of the changes since 1.45:

   [CHANGE IN BEHAVIOUR]

   As warned in release 1.45, the binding of unicode parameters to
   char/varchar columns has changed significantly - see warning at
   start of DBD::ODBC pod. If you don't attempt to insert unicode into
   char/varchar columns or if you only inserted unicode into
   nchar/nvarchar columns you should see no difference.

   Previously if DBD::ODBC received an error or (SQL_SUCCESS_WITH_INFO)
   from an ODBC API call and then the driver refused to return the
   error state/text DBD::ODBC would issue its own error saying "Unable
   to fetch information about the error" and state IM008. That state
   was wrong and has been changed to HY000.

   [BUG FIXES]

   Some drivers cannot support catalogs and/or schema names in
   SQLTables.  Recent changes set the schema/catalog name to the empty
   string (good reasons below) which causes "optional feature not
   implemented" from MS Access (which does not support schemas - even
   for a simply ping (which uses SQLTables)). Now we call
   SQLCATALOG_NAME and SQLSCHEMA_USAGE on connect to ascertain support
   which modifies SQLTables call.

   [MISCELLANEOUS]

   Added test 45_unicode_varchar.t for MS SQL Server only so far.

Martin
-- 
Martin J. Evans
Wetherby, UK