Re: Add Unicode Support to the DBI

On Tue, 08 Nov 2011 21:12:13 +0000, "Martin J. Evans"
<[email protected]> wrote:

> I've just checked in unicode_test.pl to DBI's subversion trunk in /ex dir.
> 
> It won't run right now without changing the do_connect sub as you have 
> to specify how to connect to the DB.
> Also, there is a DBD specific section at the start where you might have 
> to add a DBD it does not know about (anything other than DBD::ODBC, 
> DBD::Oracle, DBD::SQLite, DBD::CSV, DBD::mysql) if it needs something 
> other than the defaults e.g., the name of the length function in SQL, 
> the column type for unicode columns and binary columns, the setting to 
> enable UTF8/Unicode support. It could be a bit of a pain if your DBD 
> does not support type_info_all but I'm around on irc and in this list if 
> anyone wants any help making it work.
> 
> It needs rather a lot of tidying up so I'm not putting it forward as 
> code-of-the-year but it is a start.
> 
> BTW, you'll need Test::More::UTF8 and perhaps a couple of other non core 
> modules to run it.
> 
> Martin

I'll have some deeper look at both Unify and CSV ...
Attached is a revised version of the script (first argument is the
driver to test, some need more work)

$ perl /tmp/unicode_test.pl Unify
# Driver DBD::Unify-0.78
# Using DBMS_NAME 'Unify DataServer'
# Using DBMS_VER undef
# Using DRIVER_NAME '/pro/asql/v83I/lib/perl/5.10.1/DBD/Unify.pm'
# Using DRIVER_VER '00.78.0000'
# SQL_IDENTIFIER_CASE 3
# LANGDIR = dutch
print() on closed filehandle $fh at /tmp/unicode_test.pl line 438.
Use of uninitialized value in concatenation (.) or string at /tmp/unicode_test.pl line 421.
Use of uninitialized value in concatenation (.) or string at /tmp/unicode_test.pl line 421.
# Found type  (HUGE AMOUNT) size=
Use of uninitialized value in string eq at /tmp/unicode_test.pl line 422.
Use of uninitialized value in concatenation (.) or string at /tmp/unicode_test.pl line 421.
Use of uninitialized value in concatenation (.) or string at /tmp/unicode_test.pl line 421.
# Found type  (AMOUNT) size=
Use of uninitialized value in string eq at /tmp/unicode_test.pl line 422.
Use of uninitialized value in concatenation (.) or string at /tmp/unicode_test.pl line 421.
Use of uninitialized value in concatenation (.) or string at /tmp/unicode_test.pl line 421.
# Found type  (BINARY) size=
Use of uninitialized value in string eq at /tmp/unicode_test.pl line 422.
Use of uninitialized value in concatenation (.) or string at /tmp/unicode_test.pl line 421.
:
:
# Found type  (TIME) size=
Use of uninitialized value in string eq at /tmp/unicode_test.pl line 422.
DBD::Unify::db prepare failed: Syntax error in SQL dynamic statement. [for Statement "create table fredĀ ( a int)"] at /tmp/unicode_test.pl line 214.
not ok 1 - unicode table name supported
#   Failed test 'unicode table name supported'
#   at /tmp/unicode_test.pl line 216.
# died: DBD::Unify::db prepare failed: Syntax error in SQL dynamic statement. [for Statement "create table fredÄ ( a int)"] at /tmp/unicode_test.pl line 214.
ok 2 # skip Failed to create unicode table name
ok 3 # skip Failed to create unicode table name
DBD::Unify::db prepare failed: Syntax error in SQL dynamic statement. [for Statement "create table fred ( daveĀ int)"] at /tmp/unicode_test.pl line 214.
not ok 4 - unicode column name supported


$ perl /tmp/unicode_test.pl
# Driver DBD::SQLite-1.33
# Using DBMS_NAME 'SQLite'
# Using DBMS_VER '3.7.6.3'
# Using DRIVER_NAME undef
# Using DRIVER_VER undef
# SQL_IDENTIFIER_CASE undef
# LANGDIR = dutch
print() on closed filehandle $fh at /tmp/unicode_test.pl line 438.
ok 1 - unicode table name supported
ok 2 - unicode table found in unqualified table_info
ok 3 - unicode table found by qualified table_info
ok 4 - unicode column name supported
ok 5 - unicode column found in unqualified column_info
ok 6 - unicode column found by qualified column_info
ok 7 - table for unicode data
ok 8 - insert unicode data into table
ok 9 - unicode data out = unicode data in, no where
ok 10 - length of output data the same
ok 11 - db length of unicode data correct
ok 12 - select unicode data via parameterised where
ok 13 - select unicode data via inline where
ok 14 - table for unicode data
ok 15 - insert unicode data and blob into table
ok 16 - unicode data out = unicode data in, no where with blob
ok 17 - utf8 flag not set on blob data
Use of uninitialized value in string eq at /tmp/unicode_test.pl line 373.
ok 18 - retrieved blob = inserted blob
ok 19 - test table for unicode parameter markers
DBD::SQLite::st bind_param failed: Unknown named parameter: fred⬠[for Statement "insert into fred (a) values (:fred€)"] at /tmp/unicode_test.pl line 390.
not ok 20 - bind parameter with unicode parameter marker
#   Failed test 'bind parameter with unicode parameter marker'
#   at /tmp/unicode_test.pl line 392.
# died: DBD::SQLite::st bind_param failed: Unknown named parameter: fred⬠[for Statement "insert into fred (a) values (:fredâ¬)"] at /tmp/unicode_test.pl line 390.
1..20
# Looks like you failed 1 test of 20.


-- 
H.Merijn Brand  http://tux.nl      Perl Monger  http://amsterdam.pm.org/
using 5.00307 through 5.14 and porting perl5.15.x on HP-UX 10.20, 11.00,
11.11, 11.23 and 11.31, OpenSuSE 10.1, 11.0 .. 11.4 and AIX 5.2 and 5.3.
http://mirrors.develooper.com/hpux/           http://www.test-smoke.org/
http://qa.perl.org      http://www.goldmark.org/jeff/stupid-disclaimers/