Re: Strings again

Mark Hahn wrote:
> Thanks for the lengthy detailed reply.  Can you give me any pointers
> to Python's implementation or support of locale?

The locale module is available on Windows and Linux and presumably
Mac.  It deals with setting the process wide locale as well as
several formatting functions.

There is also a gettext module for translations.  time.strftime
can also do the right thing for the locale.

On the gui side I use wxWidgets.  It does have Unicode but that
doesn't go too deep.  There is a wxLocale class that is the
same thing as the Python locale module.  There is also some
stuff to deal with font mapping.

Even just trying to display a unicode string is hard.  For
example no font actually has all unicode characters.  They
usually just have subsets.  Usually it isn't an issue if
all the text comes from the same locale (eg all Greek or
all Cyrillic), but if you try to display text with both
you get boxes for which charcters the font doesn't have.
Just getting a Unicode string to display right is hard code.
See Raymond Chen's blog at
http://blogs.msdn.com/oldnewthing/archive/2004/07/16/185261.aspx

And from the comments you will see that it doesn't handle
text that goes right to left or bidirectional text.

To get an idea of where you will have to end up, look at
Java.  See the java.text package which does the locale
and gettext style stuff:

http://java.sun.com/j2se/1.5.0/docs/api/

See also this tutorial on internationalising/localising
a Java program:

http://java.sun.com/docs/books/tutorial/i18n/intro/

Quite frankly most people look at that and decide that
there is no way they will go to that much effort unless
absolutely forced to.  I agree :-)  I also put the
blame fairly and squarely on the language and libraries
which make it tedious and verbose.

This is what you have to do to do a web app in J2EE
(where each client could have a different locale
than the server program).

http://tinyurl.com/59o55

And here is the main entry point for the i18n/l10n doc:

http://java.sun.com/j2se/corejava/intl/index.jsp

Current "state of the art" amongst all programming environments
is that i18n/l10n programs have to manually call extra functions
to setup locales, manually mark text that needs to be looked up
in external catalogs (with a fair amount of setup to find them),
manually make many extra calls if you want parameter positioning,
locale correct number/date/currency/string formatting/sorting,
and various bizarre platform specific APIs if you want to
deal with things like bidirectional text display or unicode/locale
interaction issues.  Oh, and they all do something like "internally
all strings are stored as Unicode using X bits which should
be enough for anyone".  (You picked 24 for X :-)

In addition to the tedious manual crap listed above, few environments
have done anything about making I/O easier.  If you have never
done Java programming before, try to write a program that opens a
text file and prints the first line.  Experienced Java programmers
usually have to lookup which three classes they need :-)

Why is it so hard to change how a program inputs and outputs
stuff?  I remind you of the test I posted in an earlier message:

  Here is the true test.  Take a sample "Hello World" program
  and also have it output todays date, and how many days of the
  month have passed as a percentage.

  How much extra effort (just on the coding side) is it to also
  output in German?  Japanese? Hebrew? Arabic?  HTML?  XML?
  teletype?

  For the next generation of languages I believe the answer
  is going to be "none".

Roger