clisp + unicode + postgresql-socket
"Dmitry Petrov" <[email protected]> Tue, 7 Nov 2006 15:32:49 +0300
| Newsgroups | gmane.lisp.clsql.devel |
|---|---|
| Message-ID | <[email protected]> |
------=_Part_7494_21522668.1162902769920 Content-Type: multipart/alternative; boundary="----=_Part_7495_14713213.1162902769920" ------=_Part_7495_14713213.1162902769920 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi! I've just tried to use clsql with clisp and PostgreSQL database containing non-ASCII strings and failed to obtain query results with message "connot convert character #u... to KOI8-R". The reason is that clisp supports unicode and it's wrong to perform (unsigned-byte 8) -> standard-char conversion with char-code. I've made small path which behaves like for sbcl: it reads byte sequence and then converts it to string using charset specified in postgresql-socket:*pg-client-encoding*. I think such conversion is necessary because postgres databases may use different encodings, so some changes should be made to handle postgres with unicode clisp correctly. Patch is far from perfect (especially in the way of specifying encoding in global variable) but it works. I hope some corrections will be made in future clsql versions. Regards, Dmitry ------=_Part_7495_14713213.1162902769920 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi!<br>I've just tried to use clsql with clisp and PostgreSQL database containing non-ASCII strings and failed to obtain query results with message "connot convert character #u... to KOI8-R".<br>The reason is that clisp supports unicode and it's wrong to perform (unsigned-byte 8) -> standard-char conversion with char-code. I've made small path which behaves like for sbcl: it reads byte sequence and then converts it to string using charset specified in postgresql-socket:*pg-client-encoding*. <br>I think such conversion is necessary because postgres databases may use different encodings, so some changes should be made to handle postgres with unicode clisp correctly.<br>Patch is far from perfect (especially in the way of specifying encoding in global variable) but it works. <br>I hope some corrections will be made in future clsql versions.<br><br>Regards,<br>Dmitry<br> ------=_Part_7495_14713213.1162902769920-- ------=_Part_7494_21522668.1162902769920 Content-Type: text/x-patch; name=clsql-postgres+clisp-charset.diff; charset=ANSI_X3.4-1968 Content-Transfer-Encoding: base64 X-Attachment-Id: f_eu8aenz1 Content-Disposition: attachment; filename="clsql-postgres+clisp-charset.diff" ZGlmZiAtcnUgYS9kYi1wb3N0Z3Jlc3FsLXNvY2tldC9wb3N0Z3Jlc3FsLXNvY2tldC1hcGkubGlz cCBiL2RiLXBvc3RncmVzcWwtc29ja2V0L3Bvc3RncmVzcWwtc29ja2V0LWFwaS5saXNwCi0tLSBh L2RiLXBvc3RncmVzcWwtc29ja2V0L3Bvc3RncmVzcWwtc29ja2V0LWFwaS5saXNwCTIwMDYtMTAt MDIgMTM6NTc6NTYuMDAwMDAwMDAwICswNDAwCisrKyBiL2RiLXBvc3RncmVzcWwtc29ja2V0L3Bv c3RncmVzcWwtc29ja2V0LWFwaS5saXNwCTIwMDYtMTEtMDcgMTU6MTk6MjkuMDAwMDAwMDAwICsw MzAwCkBAIC0zNiw2ICszNiw3IEBACiAoZGVmbWV0aG9kIGNsc3FsLXN5czpkYXRhYmFzZS10eXBl LWxvYWQtZm9yZWlnbiAoKGRhdGFiYXNlLXR5cGUgKGVxbCA6cG9zdGdyZXNxbC1zb2NrZXQpKSkK ICAgdCkKIAorKGRlZnZhciBwb3N0Z3Jlc3FsLXNvY2tldDoqcGctY2xpZW50LWVuY29kaW5nKiAi S09JOC1SIikKIAogOzs7IE1lc3NhZ2UgSS9PIHN0dWZmCiAKQEAgLTEwNywxMSArMTA4LDE1IEBA CiAoZGVmdW4gc2VuZC1zb2NrZXQtdmFsdWUtc3RyaW5nIChzb2NrZXQgdmFsdWUpCiAgIChkZWNs YXJlICh0eXBlIHN0cmVhbSBzb2NrZXQpCiAJICAgKHR5cGUgc3RyaW5nIHZhbHVlKSkKLSAgIy1z Yi11bmljb2RlCisgICMtKG9yIHNiLXVuaWNvZGUgdW5pY29kZSkKICAgKGxvb3AgZm9yIGNoYXIg YWNyb3NzIHZhbHVlCiAgICAgICAgIGZvciBjb2RlID0gKGNoYXItY29kZSBjaGFyKQogICAgICAg ICBkbyAod3JpdGUtYnl0ZSBjb2RlIHNvY2tldCkKICAgICAgICAgZmluYWxseSAod3JpdGUtYnl0 ZSAwIHNvY2tldCkpCisgICMrKGFuZCBjbGlzcCB1bmljb2RlKQorICAocHJvZ24KKyAgICAod3Jp dGUtc2VxdWVuY2UgKGV4dDpjb252ZXJ0LXN0cmluZy10by1ieXRlcyB2YWx1ZSBwb3N0Z3Jlc3Fs LXNvY2tldDoqcGctY2xpZW50LWVuY29kaW5nKikgc29ja2V0KQorICAgICh3cml0ZS1ieXRlIDAg c29ja2V0KSkKICAgIytzYi11bmljb2RlCiAgICh3cml0ZS1zZXF1ZW5jZSAoc2ItZXh0OnN0cmlu Zy10by1vY3RldHMgdmFsdWUgOm51bGwtdGVybWluYXRlIHQpIHNvY2tldCkKICAgbmlsKQpAQCAt MTU0LDExICsxNTksMjIgQEAKIAogKGRlZnVuIHJlYWQtc29ja2V0LXZhbHVlLXN0cmluZyAoc29j a2V0KQogICAoZGVjbGFyZSAodHlwZSBzdHJlYW0gc29ja2V0KSkKLSAgIy1zYi11bmljb2RlCisg ICMtKG9yIHNiLXVuaWNvZGUgdW5pY29kZSkKICAgKHdpdGgtb3V0cHV0LXRvLXN0cmluZyAob3V0 KQogICAgIChsb29wIGZvciBjb2RlID0gKHJlYWQtYnl0ZSBzb2NrZXQpCiAgICAgICAgICAgdW50 aWwgKHplcm9wIGNvZGUpCiAgICAgICAgICAgZG8gKHdyaXRlLWNoYXIgKGNvZGUtY2hhciBjb2Rl KSBvdXQpKSkKKyAgIysoYW5kIGNsaXNwIHVuaWNvZGUpCisgIChsZXQgKChieXRlcyAobWFrZS1h cnJheSA2NAorICAgICAgICAgICAgICAgICAgICAgICAgICAgOmVsZW1lbnQtdHlwZSAnKHVuc2ln bmVkLWJ5dGUgOCkKKyAgICAgICAgICAgICAgICAgICAgICAgICAgIDphZGp1c3RhYmxlIHQKKyAg ICAgICAgICAgICAgICAgICAgICAgICAgIDpmaWxsLXBvaW50ZXIgMCkpKQorICAgIChsb29wIGZv ciBjb2RlID0gKHJlYWQtYnl0ZSBzb2NrZXQpCisgICAgICAgICAgdW50aWwgKHplcm9wIGNvZGUp CisgICAgICAgICAgZG8gKHZlY3Rvci1wdXNoLWV4dGVuZCBjb2RlIGJ5dGVzKSkKKyAgICAocHJp bnQgIkRvbmUiKQorICAgIChwcmludCBieXRlcykKKyAgICAoZXh0OmNvbnZlcnQtc3RyaW5nLWZy b20tYnl0ZXMgYnl0ZXMgcG9zdGdyZXNxbC1zb2NrZXQ6KnBnLWNsaWVudC1lbmNvZGluZyopKQog ICAjK3NiLXVuaWNvZGUKICAgKGxldCAoKGJ5dGVzIChtYWtlLWFycmF5IDY0CiAgICAgICAgICAg ICAgICAgICAgICAgICAgICA6ZWxlbWVudC10eXBlICcodW5zaWduZWQtYnl0ZSA4KQpAQCAtMjE3 LDExICsyMzMsMTYgQEAKICAgIktNUiAtLSBBZGRlZCB0byBzdXBwb3J0IHJlYWRpbmcgZnJvbSBi aW5hcnkgc3RyZWFtIGludG8gYSBzdHJpbmciCiAgIChkZWNsYXJlIChzdHJlYW0gc3RyZWFtKQog CSAgIChvcHRpbWl6ZSAoc3BlZWQgMykgKHNhZmV0eSAwKSkpCi0gICMtc2ItdW5pY29kZQorICAj LShvciBzYi11bmljb2RlIHVuaWNvZGUpCiAgIChsZXQgKChyZXN1bHQgKG1ha2Utc3RyaW5nIGxl bmd0aCkpKQogICAgIChkb3RpbWVzIChpIGxlbmd0aCByZXN1bHQpCiAgICAgICAoZGVjbGFyZSAo Zml4bnVtIGkpKQogICAgICAgKHNldGYgKGNoYXIgcmVzdWx0IGkpIChjb2RlLWNoYXIgKHJlYWQt Ynl0ZSBzdHJlYW0pKSkpKQorICAjKyhhbmQgY2xpc3AgdW5pY29kZSkKKyAgKGxldCAoKGJ5dGVz IChtYWtlLWFycmF5IGxlbmd0aCA6ZWxlbWVudC10eXBlICcodW5zaWduZWQtYnl0ZSA4KSkpKQor ICAgIChkZWNsYXJlICh0eXBlIChzaW1wbGUtYXJyYXkgKHVuc2lnbmVkLWJ5dGUgOCkgKCopKSBi eXRlcykpCisgICAgKHJlYWQtc2VxdWVuY2UgYnl0ZXMgc3RyZWFtKQorICAgIChleHQ6Y29udmVy dC1zdHJpbmctZnJvbS1ieXRlcyBieXRlcyBwb3N0Z3Jlc3FsLXNvY2tldDoqcGctY2xpZW50LWVu Y29kaW5nKikpCiAgICMrc2ItdW5pY29kZQogICAobGV0ICgoYnl0ZXMgKG1ha2UtYXJyYXkgbGVu Z3RoIDplbGVtZW50LXR5cGUgJyh1bnNpZ25lZC1ieXRlIDgpKSkpCiAgICAgKGRlY2xhcmUgKHR5 cGUgKHNpbXBsZS1hcnJheSAodW5zaWduZWQtYnl0ZSA4KSAoKikpIGJ5dGVzKSkKZGlmZiAtcnUg YS9kYi1wb3N0Z3Jlc3FsLXNvY2tldC9wb3N0Z3Jlc3FsLXNvY2tldC1wYWNrYWdlLmxpc3AgYi9k Yi1wb3N0Z3Jlc3FsLXNvY2tldC9wb3N0Z3Jlc3FsLXNvY2tldC1wYWNrYWdlLmxpc3AKLS0tIGEv ZGItcG9zdGdyZXNxbC1zb2NrZXQvcG9zdGdyZXNxbC1zb2NrZXQtcGFja2FnZS5saXNwCTIwMDYt MTEtMDcgMTU6MjE6MDIuMDAwMDAwMDAwICswMzAwCisrKyBiL2RiLXBvc3RncmVzcWwtc29ja2V0 L3Bvc3RncmVzcWwtc29ja2V0LXBhY2thZ2UubGlzcAkyMDA2LTExLTA3IDE1OjE4OjUzLjAwMDAw MDAwMCArMDMwMApAQCAtMzAsNiArMzAsOCBAQAogCSAgICM6cGdzcWwtZnR5cGUjZmxvYXQ0CiAg ICAJICAgIzpwZ3NxbC1mdHlwZSNmbG9hdDgKIAorCSAgICM6KnBnLWNsaWVudC1lbmNvZGluZyoK KwogCSAgICM6K2NyeXB0LWxpYnJhcnkrCiAJICAgIzpwb3N0Z3Jlc3FsLWNvbmRpdGlvbgogCSAg ICM6cG9zdGdyZXNxbC1jb25kaXRpb24tY29ubmVjdGlvbgo= ------=_Part_7494_21522668.1162902769920 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ CLSQL-Devel mailing list [email protected] http://lists.b9.com/mailman/listinfo/clsql-devel ------=_Part_7494_21522668.1162902769920--