Re: Migrating from UFFI to CFFI

Hello Victor,

I found that your example of
(uffi:convert-from-foreign-**string (uffi:convert-to-foreign-**string
"абв")) returns
"абв"
when it evaluated in Clozure CL REPL on my configurations (CCL 1.8 on
Windows XP and on Debian Squeeze x64)
but returns "^Z^Z^Z" when evaluated in emacs connected to swank.

On Debian i start CCL with "-K utf-8" flag.
Also I attached my cp1251 encoding definition, if you interested in it.

Best regards,
Sergey


2013/10/17 Victor <[email protected]>

> On Thu, 17 Oct 2013 06:27:27 +0300, Kevin Rosenberg <kevin-HJRc7zDS/[email protected]>
> wrote:
>
>  On Oct 16, 2013, at 11:07 AM, Victor <[email protected]> wrote:
>>
>>> ? (uffi:convert-from-foreign-**string (uffi:convert-to-foreign-**string
>>> "абв"))
>>> "^Z^Z^Z"
>>>
>>> I suspect that the problem is in the uffi:convert-to-foreign-string
>>> function.
>>>
>>
>> Your suspicion may well be right. I look a look at the code and see that
>> uffi::%convert-to-foreign-**string for ccl stores two bytes of zeros
>> after
>> storing the bytes returned by ccl:encode-string-to-octets.
>>
>> On the reverse, uffi:convert-from-foreign-**string uses the internal
>> ccl::%get-utf-8-cstring for decoding :utf-8 encoded foreign strings.
>>
>> Perhaps something has changed in the internals of ccl utf-8 representation
>> since I wrote, tested, and committed that code in Feb 2010. If you see
>> some
>> useful ideas for an update, please forward them to me and I'll be glad to
>> commit a fix to work for UFFI to work with the current version of ccl.
>> Ideally, there may be documented, exported functions for encoding/decoding
>> internationalized strings in the current version of ccl.
>>
>
>
> Hi Kevin,
>
> Indeed, OpenMCL-devel subscribers experienced the same problem. This
> is a response that I've got from Sergey.
>
> The mail from Sergey was Cc-ed to the openmcl-devel mailing list but
> it did not appear there. I quote him below:
>
> | 1) First of all I set ccl:*default-file-character-**encoding* and
> |    ccl:*default-socket-character-**encoding* to :utf-8
> |
> | 2) And then I patched uffi soures in quicklisp directory
>
> See these modifications as a patch attached to this message.
>
> | Additionally i've defined encoding for still widely used cp1251, but
> | I think it's offtopic.
>
> I've applied these changes and indeed, Sqlite3 backend now is capable
> of dealing with UTF-8 encoded strings. However, I also noticed that
> the problem with UFFI:CONVERT-* functions still remain. Moreover, I've
> also stuck into the problem that Sergey noticed some time before:
>
> http://clozure.com/pipermail/**openmcl-devel/2013-April/**014220.html<http://clozure.com/pipermail/openmcl-devel/2013-April/014220.html>
>
> it appears that the CCL::%GET-UTF-8-CSTRING function is not a reliable
> one.
>
> I think that if confirmed, this function has some internal problems
> and should be either fixed or avoided.
>
> The two zeroes in the end are a UTF-8 version of the 0 character
> terminator that ordinary C strings have. CFFI relies on the babel
> package to get its size, but I guess that this thing is static enough
> to hard-code it for UTF-8.
>
> Thanks!
> Victor
>
> p.s. CP1251 encoding is not defined by the patch but I believe it is
> a good thing to do in the future.

_______________________________________________
CLSQL mailing list
[email protected]
http://lists.b9.com/cgi-bin/mailman/listinfo/clsql