broken file name conversion

Duplicating and updating mail sent to <[email protected]>
on Mon, 19 Sep 2005 20:48:42 +0400.  Perhaps the addresses to use have
changed since last release, and <[email protected]> is now
(largely) defunct.

Versions: 1.3.8, 2.0.1

Hot to repeat.  Unset `LANG' environment variable.  Create directory
with name containing 8-bit characters, and image file(s) in it.  Run
the program with name of image file in that directory ($PHOTOS_DIR),
or of $PHOTOS_DIR itself, as (single) command line argument, `gqview'
outputs the following.  This is output on file descriptor 1, not 2, as
one would expect for error messages.  And the output is repeated
endlessly with interval of second or so, even if program window is
minimized and user issues no commands.  The output is as follows.

Unable to convert to locale from UTF-8:
$PHOTOS_DIR
Invalid byte sequence in conversion input
Unable to convert to locale from UTF-8:
$PHOTOS_DIR/PICT0474_l.JPG
Invalid byte sequence in conversion input
Unable to convert to locale from UTF-8:
$PHOTOS_DIR/PICT0474_l.JPG
Invalid byte sequence in conversion input

It does not depend on whether file names are well formed utf-8 or not.


When user specifies such file names, `gqview' pops up a dialog with
error message, but the message is misleading.  ...<filenames are not
encoded with the preferred locale character set> is of little use
unless specifies what the character set is.  It also advices to set
`G_BROKEN_FILENAMES=1', which changes nothing, and could not, since
file names are really not broken in any way.

If file names are in utf-8, setting `LANG' to something specifying
utf-8 encoding works around this, but it may very well be
unacceptable.  It can break million other things, depending on what
other underlying packages are broken in which ways, starting with
strings returned by `strerror' from libc (will describe such a lossage
in one linux-gnu distribution in detail on request).  Generally, it is
only not setting `LANG', or setting `LANG=C', is a safe play.  And
`gqview' makes even this unacceptable.


Locating writes to standard output is easy, and this indicates a major
`gqview' design flaw as follows.

`printf' calls (again, why should error messages go to standard
output?) occur in the following backtrace (all 8-bit characters in gdb
output are replaced with their octal encoding).

#8  0x08056d04 in path_from_utf8 (
    utf8=0x826fd18 "/home/gin/share/photos/2005_08_\320\232/PICT0474_l.JPG")
    at ../../share/gqview-2.0.1/src/ui_fileops.c:157
#9  0x08056db7 in stat_utf8 (
    s=0x826fd18 "/home/gin/share/photos/2005_08_\320\232/PICT0474_l.JPG", st=0xbffff730)
    at ../../share/gqview-2.0.1/src/ui_fileops.c:193
#10 0x08056ed3 in filetime (
    s=0x826fd18 "/home/gin/share/photos/2005_08_\320\232/PICT0474_l.JPG")
    at ../../share/gqview-2.0.1/src/ui_fileops.c:233
#11 0x08082fb3 in image_auto_refresh_cb (data=0x813eba8)
    at ../../share/gqview-2.0.1/src/image.c:3058
#12 0x40496e78 in g_timeout_dispatch (source=0x813f6c0, 
    callback=0x8082f70 <image_auto_refresh_cb>, user_data=0x813eba8)
    at ../../share/glib-2.8.1/glib/gmain.c:3293
#13 0x404946dc in g_main_dispatch (context=0x8122b78)
    at ../../share/glib-2.8.1/glib/gmain.c:1934
#14 0x40495749 in IA__g_main_context_dispatch (context=0x8122b78)
    at ../../share/glib-2.8.1/glib/gmain.c:2484
#15 0x40495baf in g_main_context_iterate (context=0x8122b78, block=1, 
    dispatch=1, self=0x8101e80) at ../../share/glib-2.8.1/glib/gmain.c:2565
#16 0x404960c0 in IA__g_main_loop_run (loop=0x822a238)
    at ../../share/glib-2.8.1/glib/gmain.c:2769
#17 0x4012285b in IA__gtk_main () at ../../share/gtk+-2.8.3/gtk/gtkmain.c:976
#18 0x08092b1a in main (argc=2, argv=0xbffff9c4)
    at ../../share/gqview-2.0.1/src/main.c:1363

It, in turn, is caused by attempt to convert `image_auto_refresh_cb'
`imd->image_path' from utf-8 to <preferred character set used for
filenames>, which `get_filename_charset' under conditions specified
above returns to be "ANSI_X3.4-1968", that is, ascii.  Certainly such
a conversion will fail even if the string is correctly encoded in
utf-8, but contains non-ascii characters.

What is worst is that such a conversion is done to obtain file names
to pass to `stat' system call.  That it is unclear how conversion
input (which is non-ascii utf-8 `imd->image_path') was obtained, is a
separate matter.  File name indeed was in utf-8, but nothing specified
to `gqview' that it may expect so; and attempting to convert it from
the same ascii <preferred character set> would have returned error.
So `imd->image_path' was not obtained by converting from ascii to
utf-8, and assumption that it may be converted back in any way is just
wrong.

But even if file name (or other string) was indeed encoded in
<preferred character set> for particular locale, decoding it from that
character set (actually, encoding, for this is what glib / glibc
`iconv' subsystem generally applies to strings) to utf-8 and then
encoding back to the same encoding do not necessary yield the same
string.  Even utf-8 has different (however, redundant and
non-canonical) encoding for the same sequence of unicode code points.
If encoding is stateful (including ones based on iso2022), there are
even more encoding variants for the same unicode string (or utf-8 one,
for that matter).  So file name obtained that way may easily be quite
different from the original one specified to `gqview' by user, and
file with that name may even not exist.  Converting file names to any
universal encoding and then converting them back to initial one is
inherently incorrect.

The fix is to keep image file name, as specified to program and exists
in file system, and pass it to file system interface calls without any
conversions.  The human-readable representation of file name, as
presented to user in graphical interface, is completely another
matter, and it may easily not even consist of a single utf-8 (or
other) string at all, since (absolute) file name may easily contain
directory components from several file systems using different file
name encodings.  (The issue is the same for any program that has GUI
and operates on user specified files / directories.  For development
branch: how do other programs resolve it, in particular,
<http://www.gnome.org/projects/nautilus/>?  Can `gqview' share with it
code that does the job?)

Never investigated how `imd->image_path' is actually obtained, so do
not submitting a patch.  After investigating this and related matters,
the patch is very likely get fairly big and *look* "too radical a
change for stable branch".

Anyway, this is what definitely has to be changed.  The exact changes
are up to program maintainer(s).


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. 
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php