Re: utf-8 output in a terminal

> On Jul 28, 2024, at 6:31=E2=80=AFPM, Gr=C3=A9gory Vanuxem =
<[email protected]> wrote:
>=20
> Found!
>=20
> In level-0/l0-misc.lisp there is exactly what i was looking for.
> (ccl::%get-utf-8-cstring ..) It's a pity I think it is not documented
> since now UTF-8 is relatively widely used.
>=20
> All is right now!


The exported functionality is ccl:get-encoded-string.

(get-encoded-string :utf-8 cname (#_strlen cname))

https://ccl.clozure.com/docs/ccl.html#f_get-encoded-string

The manual also contains examples of using both ccl:with-encoded-cstrs =
and ccl:get-encoded-string:

https://ccl.clozure.com/docs/ccl.html#brief-examples



>=20
> Thanks
>=20
> Greg
>=20
> Le lun. 29 juil. 2024 =C3=A0 01:34, Gr=C3=A9gory Vanuxem =
<[email protected]> a =C3=A9crit :
>>=20
>> Hello Matthew,
>>=20
>> Le dim. 28 juil. 2024 =C3=A0 07:31, R. Matthew Emerson =
<[email protected]> a =C3=A9crit :
>>>=20
>>>=20
>>>=20
>>>> On Jul 27, 2024, at 8:23=E2=80=AFPM, Gr=C3=A9gory Vanuxem =
<[email protected]> wrote:
>>>>=20
>>>> In FriCAS  built on top of Clozure CL I obtain this output in a =
terminal (WSL2)
>>>>=20
>>>> (gamma(x) =C3=83=C3=82=C2=A2=C3=83=C3=82=C3=83=C3=82 reals && Re(x) =
> 0) || (x =C3=83=C3=82=C2=A2=C3=83=C3=82=C3=83=C3=82 integers &&
>>>> gamma(x) =C3=83=C3=82=C2=A2=C3=83=C3=82=C3=83=C3=82 reals)
>>>>=20
>>>> instead of (SBCL)
>>>>=20
>>>> (gamma(x) =E2=88=88 reals && Re(x) > 0) || (x =E2=88=89 integers && =
gamma(x) =E2=88=88 reals)
>>>>=20
>>>> if I (print ...) a string with, for example, =E2=88=88, in a pure =
Clozure CL
>>>> (recently git cloned) this is correctly printed, no problem.
>>>>=20
>>>> So I wonder if I have to modify the output stream character =
encoding
>>>> or the output routine.
>>>=20
>>> Check that the external format of the FriCAS output stream is what =
you expect. In other words, if you pass an explicit :external-format =
argument to cl:open, make sure it matches what your terminal requires.
>>=20
>> I don't think it is at this stage in fact, I was wrong. Better
>> checking what is going on, since the string comes from C (sorry I
>> forgot to mention that) I tend to think my problem comes from
>> ccl::%get-cstring. Tracing it seems to show it's not "decoding" it =
and
>> considers it as a 8859-1 string. But to test this i.e. force CCL to
>> use :utf8 format to "read it" I have not found. There is no option
>> from what I know to get-cstring like :external-format and using
>> with-encoded-cstrs does not seem to work.
>>=20
>> Right now I am using some FFI macros from FriCAS to simplify my code,
>> I will try to revert to pure CCL code first to check my claims,
>> TIMTOWTDI.
>>=20
>>> Doing (describe *terminal-io*) might show something interesting. =
Here=E2=80=99s what it shows for me:
>>>=20
>>> ? (describe *terminal-io*)
>>> #<ECHOING-TWO-WAY-STREAM input #<BASIC-CHARACTER-INPUT-STREAM UTF-8 =
(TTY/0) #x1824C7D6>, output #<BASIC-CHARACTER-OUTPUT-STREAM UTF-8 =
(TTY/1) #x1824C2A6> #x1825FBE6>
>>> Class: #<STANDARD-CLASS ECHOING-TWO-WAY-STREAM>
>>> Wrapper: #<CLASS-WRAPPER ECHOING-TWO-WAY-STREAM #x18039A16>
>>> Instance slots
>>> SHARED-RESOURCE: NIL
>>> OPEN-P: T
>>> INPUT-STREAM: #<BASIC-CHARACTER-INPUT-STREAM UTF-8 (TTY/0) =
#x1824C7D6>
>>> OUTPUT-STREAM: #<BASIC-CHARACTER-OUTPUT-STREAM UTF-8 (TTY/1) =
#x1824C2A6>
>>>=20
>>> The default external format has been #<EXTERNAL-FORMAT :UTF-8/:UNIX =
#x18249C6E> for quite a long time now.
>>>=20
>>> =
https://ccl.clozure.com/docs/ccl.html#characters-and-external-formats
>>=20
>> This is exactly what I have when I use this function in FriCAS. I =
have
>> read the documentation and the internal utf32 representation but here
>> I am stuck. This is not the most important thing I need to do but it
>> will be imperative for some next stuff.
>>=20
>> In anyway, thanks!
>>=20
>> Greg
>>=20
>>>=20
>>>>=20
>>>> And more importantly, how can I achieve that?
>>>>=20
>>>> - Greg
>>>>=20
>>>> PS:
>>>> (1) -> )lisp CCL:*DEFAULT-EXTERNAL-FORMAT*
>>>> Value =3D :UNIX
>>>=20