Re: Using -dshow-available-fonts with LilyPond v2.2 5.12 throws error with non–UTF-8 encoded font info

Thanks for making the merge request. Unfortunately, it didn’t fix the issue, although it really seems like it should have. I’m starting to think this may be a bug in Guile.

With the patch applied, running `lilypond -dshow-available-fonts` outputs:

GNU LilyPond 2.25.12 (running Guile 3.0)
ERROR: In procedure ly:font-config-display-fonts:
Throw to key `decoding-error' with args `("scm_from_stringn" "input locale conversion error" 92 #vu8([very long bytevector])

The “very long bytevector” is the bytes of the C string that’s being converted. In this C string, there appears to be a single encoding issue related to a macOS system font that causes conversion to fail. Attached is a much reduced (though still long) LilyPond file that demonstrates an issue (but maybe not the exact same issue). In the LilyPond file, I’m converting a bytevector to a string using bytevector->string <https://www.gnu.org/software/guile/manual/html_node/Representing-Strings-as-Bytes.html#index-bytevector_002d_003estring> (which is documented <https://www.gnu.org/software/guile/manual/html_node/Conversion-to_002ffrom-C.html#index-scm_005ffrom_005fstringn> as the Scheme interface to scm_from_stringn).

When I use a conversion strategy of 'substitute, the conversion works as expected (with a � where the encoding issue occurs). When I use the default strategy ('error, presumably), LilyPond outputs:

string-test.ly:3:2: error: Guile signaled an error for the expression beginning here
#
 (begin
Throw to key `decoding-error' with args `("scm_from_utf8_stringn" "input locale conversion error" 2 #vu8([long bytevector])

For what it’s worth, I tried—

SCM string_port = scm_open_output_string ();
scm_set_port_encoding_x (string_port, scm_from_utf8_string ("UTF-8"));
scm_set_port_conversion_strategy_x (string_port, scm_from_utf8_symbol ("substitute"));
SCM scheme_string = scm_from_port_stringn (str.c_str (), str.length (), string_port);
scm_close_port (string_port);
scm_write_line (scheme_string, port);

—only to realize that scm_from_port_stringn <https://codeberg.org/rlb/guile/src/branch/main/libguile/strings.c#L1726> is a convenience function that calls scm_from_stringn (so it results in the same error). (This is based on the bytevector->string implementation <https://codeberg.org/rlb/guile/src/branch/main/module/ice-9/iconv.scm#L63>, which uses read-string <https://codeberg.org/rlb/guile/src/branch/main/module/ice-9/rdelim.scm#L152> instead of something like scm_from_port_stringn.)

It seems like there’s something weird happening with the conversion strategy passed to scm_from_stringn. I’m wondering if a conversion strategy from an input port is being used instead of the conversion strategy passed to scm_from_stringn.

Cheers,
Nate



> On Jan 9, 2024, at 2:49 AM, Werner LEMBERG <[email protected]> wrote:
> 
> 
>> On macOS Ventura v13.6.3 (and almost certainly other versions), the
>> string produced when using -dshow-available-fonts need not be
>> encoded as UTF-8.
> 
> Thanks, should be fixed with this MR:
> 
>  https://gitlab.com/lilypond/lilypond/-/merge_requests/2231
> 
> Out of interest: Which font shows this behaviour?
> 
> I suggest you contact the FontConfig people and report the issue, too;
> assuming that the font is valid it should be possible to convert the
> non-UTF8 data to UTF8.
> 
> 
>    Werner