Re: BeOS port news and public patch

Please stop posting to multiple lists.  That's just going to get your
posts lost and disconnected from their threads---many of us suppress
duplicates.  Send patches you think are ready for inclusion in the
development tree to xemacs-patches.  Send all other discussion to
xemacs-design.

Redirecting to xemacs-design.

>>>>> "François" == François Revol <[email protected]> writes:

    François> Maybe a "virtual" utf-8 charset would do, but it seems

UTF-8 can't possibly be a charset.  It is a specific transformation
format for Unicode, but all Unicode variants share the same charset.

Unicode is not treated as a Mule charset in XEmacs; it has special
status because (in general) a Unicode code point may be 31 bits (cf
the UTF-32 transformation format and ISO 10646), and Emacs characters
only have 30 (and a large block of them is unavailable since they are
already used for other character sets).

So Unicodes are represented as integers, and all positive integers are
Unicodes.  (In principle; of course we have translation tables you can
use to discover whether an integer actually has a definition in our
version of Unicode, see etc/unicode for the tables we use; they can be
trivially updated from www.unicode.org.)  Nor do we directly output
Unicodes to the GUI.  For one thing, this is very poorly documented
except for the Utf8* extensions in XFree86 (and I assume the Unicode
APIs for recent Win32 have decent docs from MSDN, but I don't do
Windows).  For another, since there are very few reasonably complete
free Unicode fonts, we need to support "filling in" from legacy
character sets, but even XFree86's Utf8* extensions don't support
that.  So instead we translate to legacy character sets, and there's
no need for a "Unicode charset".

However, there are a whole set of Unicode coding systems, which are
used to translate blocks of text from one encoding to another.  See
src/mule-coding.c for more information on those.

You really need to get a clue about this stuff if you want to work
with X?Emacs; for better or worse, Mule is based on ISO 2022 and that
is not going to change without a large amount of work, which is barely
begun yet.  If you create a variant of XEmacs which only works with
UTF-8, even if that's only on BeOS, it will not get in to the mainline
CVS tree.  You need to work with the existing framework.

You can download a copy of ISO 2022 from ECMA's website (I think it's
www.ecma.ch) as ECMA-35.  I believe the Unicode standard is online at
www.unicode.org, but ISTR it's (deliberately) hard to download the
whole thing.

For Emacs-specific stuff, there are long comments in most of the files
related to Mule in src/, in particular look in mule-charset.*,
charset.h, mule-coding.*, file-coding.*, text.*, and unicode.c.  There
is a fair amount of information in the Lisp Reference Manual and the
Internals Manual (this is pretty sketchy, unfortunately) available
from the Info directory node.

    François> Emacs has a very specific idea about what charsets are
    François> (256 or 256^2 ?).

Try 94^n or 96^n.  UTF-2000 (see below) has extensions for 256 and
256^n charsets and I think some of it was in 21.2.x before UTF-2000
forked from the XEmacs mainline, but if any of that is left in XEmacs
21.4 or 21.5 it's dead code.

    François> Has any of those attempts been published somewhere ?

There is a working version of XEmacs based on XEmacs 21.4 called
XEmacs/UTF-2000 or CHISE.  See http://www.m17n.org/utf-2000, I think
(it's all in Japanese, but you can download from there, still).
XEmacs/UTF-2000 uses UTF-8 internally but translates from Unicode to
legacy charsets, and then outputs using corresponding fonts AFAIK.

The various patches for Xft have all been posted to xemacs-patches;
search the archives for "xft".  There's link to the archives at
http://www.xemacs.org/Lists/#xemacs-patches.  The Windows code is in
CVS for ages; look at the files named src/something-msw.*.

You're working with 21.5, current CVS, right?  If not, get it and port
your changes to it; your code will probably never get into 21.4.

If you are already working on 21.5, have you bothered to list the src/
and lisp/ directories yet?  If you do, you should immediately find
unicode.c and unicode.el.  The first contains the utilities needed for
translating Emacs characters to Unicode code points.  The second turns
out to be irrelevant to your current purposes.

I suspect you may have to steal int-to-utf8 code from somewhere; there
is such code in mule-charsets.c, IIRC.  Please try to make sure that
any such utility functions end up in unicode.c, and are not duplicated
in several places.

-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.