clisp + unicode + postgresql-socket

"Dmitry Petrov" <[email protected]> Tue, 7 Nov 2006 15:32:49 +0300
Newsgroups gmane.lisp.clsql.devel
Message-ID <[email protected]>
------=_Part_7494_21522668.1162902769920
Content-Type: multipart/alternative; 
	boundary="----=_Part_7495_14713213.1162902769920"

------=_Part_7495_14713213.1162902769920
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi!
I've just tried to use clsql with clisp and PostgreSQL database containing
non-ASCII strings and failed to obtain query results with message "connot
convert character #u... to KOI8-R".
The reason is that clisp supports unicode and it's wrong to perform
(unsigned-byte 8) -> standard-char conversion with char-code. I've made
small path which behaves like for sbcl: it reads byte sequence and then
converts it to string using charset specified in
postgresql-socket:*pg-client-encoding*.
I think such conversion is necessary because postgres databases may use
different encodings, so some changes should be made to handle postgres with
unicode clisp correctly.
Patch is far from perfect (especially in the way of specifying encoding in
global variable) but it works.
I hope some corrections will be made in future clsql versions.

Regards,
Dmitry

------=_Part_7495_14713213.1162902769920
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi!<br>I've just tried to use clsql with clisp and PostgreSQL database containing non-ASCII strings and failed to obtain query results with message &quot;connot convert character #u... to KOI8-R&quot;.<br>The reason is that clisp supports unicode and it's wrong to perform (unsigned-byte 8) -&gt; standard-char conversion with char-code. I've made small path which behaves like for sbcl: it reads byte sequence and then converts it to string using charset specified in postgresql-socket:*pg-client-encoding*.
<br>I think such conversion is necessary because postgres databases may use different encodings, so some changes should be made to handle postgres with unicode clisp correctly.<br>Patch is far from perfect (especially in the way of specifying encoding in global variable) but it works.
<br>I hope some corrections will be made in future clsql versions.<br><br>Regards,<br>Dmitry<br>

------=_Part_7495_14713213.1162902769920--

------=_Part_7494_21522668.1162902769920
Content-Type: text/x-patch; name=clsql-postgres+clisp-charset.diff; 
	charset=ANSI_X3.4-1968
Content-Transfer-Encoding: base64
X-Attachment-Id: f_eu8aenz1
Content-Disposition: attachment; filename="clsql-postgres+clisp-charset.diff"

ZGlmZiAtcnUgYS9kYi1wb3N0Z3Jlc3FsLXNvY2tldC9wb3N0Z3Jlc3FsLXNvY2tldC1hcGkubGlz
cCBiL2RiLXBvc3RncmVzcWwtc29ja2V0L3Bvc3RncmVzcWwtc29ja2V0LWFwaS5saXNwCi0tLSBh
L2RiLXBvc3RncmVzcWwtc29ja2V0L3Bvc3RncmVzcWwtc29ja2V0LWFwaS5saXNwCTIwMDYtMTAt
MDIgMTM6NTc6NTYuMDAwMDAwMDAwICswNDAwCisrKyBiL2RiLXBvc3RncmVzcWwtc29ja2V0L3Bv
c3RncmVzcWwtc29ja2V0LWFwaS5saXNwCTIwMDYtMTEtMDcgMTU6MTk6MjkuMDAwMDAwMDAwICsw
MzAwCkBAIC0zNiw2ICszNiw3IEBACiAoZGVmbWV0aG9kIGNsc3FsLXN5czpkYXRhYmFzZS10eXBl
LWxvYWQtZm9yZWlnbiAoKGRhdGFiYXNlLXR5cGUgKGVxbCA6cG9zdGdyZXNxbC1zb2NrZXQpKSkK
ICAgdCkKIAorKGRlZnZhciBwb3N0Z3Jlc3FsLXNvY2tldDoqcGctY2xpZW50LWVuY29kaW5nKiAi
S09JOC1SIikKIAogOzs7IE1lc3NhZ2UgSS9PIHN0dWZmCiAKQEAgLTEwNywxMSArMTA4LDE1IEBA
CiAoZGVmdW4gc2VuZC1zb2NrZXQtdmFsdWUtc3RyaW5nIChzb2NrZXQgdmFsdWUpCiAgIChkZWNs
YXJlICh0eXBlIHN0cmVhbSBzb2NrZXQpCiAJICAgKHR5cGUgc3RyaW5nIHZhbHVlKSkKLSAgIy1z
Yi11bmljb2RlCisgICMtKG9yIHNiLXVuaWNvZGUgdW5pY29kZSkKICAgKGxvb3AgZm9yIGNoYXIg
YWNyb3NzIHZhbHVlCiAgICAgICAgIGZvciBjb2RlID0gKGNoYXItY29kZSBjaGFyKQogICAgICAg
ICBkbyAod3JpdGUtYnl0ZSBjb2RlIHNvY2tldCkKICAgICAgICAgZmluYWxseSAod3JpdGUtYnl0
ZSAwIHNvY2tldCkpCisgICMrKGFuZCBjbGlzcCB1bmljb2RlKQorICAocHJvZ24KKyAgICAod3Jp
dGUtc2VxdWVuY2UgKGV4dDpjb252ZXJ0LXN0cmluZy10by1ieXRlcyB2YWx1ZSBwb3N0Z3Jlc3Fs
LXNvY2tldDoqcGctY2xpZW50LWVuY29kaW5nKikgc29ja2V0KQorICAgICh3cml0ZS1ieXRlIDAg
c29ja2V0KSkKICAgIytzYi11bmljb2RlCiAgICh3cml0ZS1zZXF1ZW5jZSAoc2ItZXh0OnN0cmlu
Zy10by1vY3RldHMgdmFsdWUgOm51bGwtdGVybWluYXRlIHQpIHNvY2tldCkKICAgbmlsKQpAQCAt
MTU0LDExICsxNTksMjIgQEAKIAogKGRlZnVuIHJlYWQtc29ja2V0LXZhbHVlLXN0cmluZyAoc29j
a2V0KQogICAoZGVjbGFyZSAodHlwZSBzdHJlYW0gc29ja2V0KSkKLSAgIy1zYi11bmljb2RlCisg
ICMtKG9yIHNiLXVuaWNvZGUgdW5pY29kZSkKICAgKHdpdGgtb3V0cHV0LXRvLXN0cmluZyAob3V0
KQogICAgIChsb29wIGZvciBjb2RlID0gKHJlYWQtYnl0ZSBzb2NrZXQpCiAgICAgICAgICAgdW50
aWwgKHplcm9wIGNvZGUpCiAgICAgICAgICAgZG8gKHdyaXRlLWNoYXIgKGNvZGUtY2hhciBjb2Rl
KSBvdXQpKSkKKyAgIysoYW5kIGNsaXNwIHVuaWNvZGUpCisgIChsZXQgKChieXRlcyAobWFrZS1h
cnJheSA2NAorICAgICAgICAgICAgICAgICAgICAgICAgICAgOmVsZW1lbnQtdHlwZSAnKHVuc2ln
bmVkLWJ5dGUgOCkKKyAgICAgICAgICAgICAgICAgICAgICAgICAgIDphZGp1c3RhYmxlIHQKKyAg
ICAgICAgICAgICAgICAgICAgICAgICAgIDpmaWxsLXBvaW50ZXIgMCkpKQorICAgIChsb29wIGZv
ciBjb2RlID0gKHJlYWQtYnl0ZSBzb2NrZXQpCisgICAgICAgICAgdW50aWwgKHplcm9wIGNvZGUp
CisgICAgICAgICAgZG8gKHZlY3Rvci1wdXNoLWV4dGVuZCBjb2RlIGJ5dGVzKSkKKyAgICAocHJp
bnQgIkRvbmUiKQorICAgIChwcmludCBieXRlcykKKyAgICAoZXh0OmNvbnZlcnQtc3RyaW5nLWZy
b20tYnl0ZXMgYnl0ZXMgcG9zdGdyZXNxbC1zb2NrZXQ6KnBnLWNsaWVudC1lbmNvZGluZyopKQog
ICAjK3NiLXVuaWNvZGUKICAgKGxldCAoKGJ5dGVzIChtYWtlLWFycmF5IDY0CiAgICAgICAgICAg
ICAgICAgICAgICAgICAgICA6ZWxlbWVudC10eXBlICcodW5zaWduZWQtYnl0ZSA4KQpAQCAtMjE3
LDExICsyMzMsMTYgQEAKICAgIktNUiAtLSBBZGRlZCB0byBzdXBwb3J0IHJlYWRpbmcgZnJvbSBi
aW5hcnkgc3RyZWFtIGludG8gYSBzdHJpbmciCiAgIChkZWNsYXJlIChzdHJlYW0gc3RyZWFtKQog
CSAgIChvcHRpbWl6ZSAoc3BlZWQgMykgKHNhZmV0eSAwKSkpCi0gICMtc2ItdW5pY29kZQorICAj
LShvciBzYi11bmljb2RlIHVuaWNvZGUpCiAgIChsZXQgKChyZXN1bHQgKG1ha2Utc3RyaW5nIGxl
bmd0aCkpKQogICAgIChkb3RpbWVzIChpIGxlbmd0aCByZXN1bHQpCiAgICAgICAoZGVjbGFyZSAo
Zml4bnVtIGkpKQogICAgICAgKHNldGYgKGNoYXIgcmVzdWx0IGkpIChjb2RlLWNoYXIgKHJlYWQt
Ynl0ZSBzdHJlYW0pKSkpKQorICAjKyhhbmQgY2xpc3AgdW5pY29kZSkKKyAgKGxldCAoKGJ5dGVz
IChtYWtlLWFycmF5IGxlbmd0aCA6ZWxlbWVudC10eXBlICcodW5zaWduZWQtYnl0ZSA4KSkpKQor
ICAgIChkZWNsYXJlICh0eXBlIChzaW1wbGUtYXJyYXkgKHVuc2lnbmVkLWJ5dGUgOCkgKCopKSBi
eXRlcykpCisgICAgKHJlYWQtc2VxdWVuY2UgYnl0ZXMgc3RyZWFtKQorICAgIChleHQ6Y29udmVy
dC1zdHJpbmctZnJvbS1ieXRlcyBieXRlcyBwb3N0Z3Jlc3FsLXNvY2tldDoqcGctY2xpZW50LWVu
Y29kaW5nKikpCiAgICMrc2ItdW5pY29kZQogICAobGV0ICgoYnl0ZXMgKG1ha2UtYXJyYXkgbGVu
Z3RoIDplbGVtZW50LXR5cGUgJyh1bnNpZ25lZC1ieXRlIDgpKSkpCiAgICAgKGRlY2xhcmUgKHR5
cGUgKHNpbXBsZS1hcnJheSAodW5zaWduZWQtYnl0ZSA4KSAoKikpIGJ5dGVzKSkKZGlmZiAtcnUg
YS9kYi1wb3N0Z3Jlc3FsLXNvY2tldC9wb3N0Z3Jlc3FsLXNvY2tldC1wYWNrYWdlLmxpc3AgYi9k
Yi1wb3N0Z3Jlc3FsLXNvY2tldC9wb3N0Z3Jlc3FsLXNvY2tldC1wYWNrYWdlLmxpc3AKLS0tIGEv
ZGItcG9zdGdyZXNxbC1zb2NrZXQvcG9zdGdyZXNxbC1zb2NrZXQtcGFja2FnZS5saXNwCTIwMDYt
MTEtMDcgMTU6MjE6MDIuMDAwMDAwMDAwICswMzAwCisrKyBiL2RiLXBvc3RncmVzcWwtc29ja2V0
L3Bvc3RncmVzcWwtc29ja2V0LXBhY2thZ2UubGlzcAkyMDA2LTExLTA3IDE1OjE4OjUzLjAwMDAw
MDAwMCArMDMwMApAQCAtMzAsNiArMzAsOCBAQAogCSAgICM6cGdzcWwtZnR5cGUjZmxvYXQ0CiAg
ICAJICAgIzpwZ3NxbC1mdHlwZSNmbG9hdDgKIAorCSAgICM6KnBnLWNsaWVudC1lbmNvZGluZyoK
KwogCSAgICM6K2NyeXB0LWxpYnJhcnkrCiAJICAgIzpwb3N0Z3Jlc3FsLWNvbmRpdGlvbgogCSAg
ICM6cG9zdGdyZXNxbC1jb25kaXRpb24tY29ubmVjdGlvbgo=
------=_Part_7494_21522668.1162902769920
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
CLSQL-Devel mailing list
[email protected]
http://lists.b9.com/mailman/listinfo/clsql-devel

------=_Part_7494_21522668.1162902769920--