Re: Issues with japanese support

From: Ralf Angeli <[email protected]>
Subject: Re: Issues with japanese support
Date: Mon, 14 Feb 2005 10:19:33 +0100

> Here are some comments regarding the changes:
> 
Hi Ralf, thank you for your comments.
 
I fixed as followed:

| --- latex.el  15 Feb 2005 07:20:23 -0000      5.328
| +++ latex.el  15 Feb 2005 07:24:16 -0000      5.329
| @@ -2442,9 +2442,8 @@
|         ;; For Japanese (FIXED on 2005-02-11)
|         (when (featurep 'mule)
|           (goto-char from)
| -         (while (re-search-forward "\\(\\cj\\)\n" to t)
| -           (replace-match "\\1")
| -           (setq to (1- to))))
| +         (while (re-search-forward "\\(\\cj\\)\n\\(\\cj\\)" to t)
| +           (replace-match "\\1\\2")))
|         ;; Make sure sentences ending at end of line get an extra space.
|         (if (or (not (boundp 'sentence-end-double-space))
|                 sentence-end-double-space)

> 1) The search/regexp doesn't cater for the situation where there is a
>    line ending with Japanese characters and the following line starts
>    with a non-Japanese character.  In this case the newline must not
>    be deleted.
> 
Catered.  Thanks.

> 2) `to' is a marker.  With the change above you are degrading it to an
>    integer.  Not good.  Why does it need to be changed at all?
> 
Fixed.  It was all my fault that misunderstanding a marker.

> 3) I am not sure about the performance implications of this change.
>    The function to get rid of newlines in the non-Japanese case is
>    `subst-char-in-region' which is implemented in C and should be
>    quite fast.  The `re-search-forward-statement' is likely to slow
>    things down.
> 
Which character do you use for TOCHAR?
We want to substitute `\n' by empty, not space.

I'm not sure about the performance implications, too.

Any good idea?

> 4) Is it sufficient to search for \cj or are there other character
>    categories which have to be treated the same way?
> 
At least, it is sufficient for Japanese and not sufficient for 
Chinese and Korean.  But I am not a native of Chinese or Korea,
so that I can't say space should be gotten rid of when filling
in Chinese and Korean like in Japanese.

May be \c| is sufficient, but its document says:

(category-docstring ?|)
 => While filling, we can break a line at this character.

That is not equal that no space should be removed.
Please negatives of Chinese and Korean give us advices.

> |           ;; Make sure sentences ending at end of line get an extra space.
> |           (if (or (not (boundp 'sentence-end-double-space))
> |                   sentence-end-double-space)
> | @@ -2558,7 +2564,12 @@
> |    ;; COMPATIBILITY for Emacs <= 21.3 and XEmacs
> |    (if (fboundp 'fill-move-to-break-point)
> |        (fill-move-to-break-point linebeg)
> | -    (skip-chars-backward "^ \n")
> | +    ;; For Japanese (FIXED on 2005-02-11)
> | +    (if (featurep 'mule)
> | +       (if (TeX-looking-at-backward "\\cj")
> | +           (backward-char 1)
> | +         (skip-chars-backward "^ \n"))
> | +      (skip-chars-backward "^ \n"))
> 
> That's only a cosmetic issue, but I'd write it like this:
> 
> (if (and (featurep 'mule)
> 	 (TeX-looking-at-backward "\\cj"))
>     (backward-char 1)
>   (skip-chars-backward "^ \n"))
> 
Applied.  Thank you.

> One probably should look forward for a Japanese character as well.
> 
> Why do you have to go a character backward?  When
> `LaTeX-fill-move-to-break-point' is executed, point is at the column
> specified in `fill-column'.  Isn't that the correct position in this
> case?
> 
After `move-to-column', `forward-char' is executed once.
And then, `LaTeX-fill-move-to-break-point' is called:

              (move-to-column (current-fill-column))
              (if (when (< (point) (marker-position end-marker))
                    ;; Find the position where we'll break the line.
                    (forward-char 1) ; Use an immediately following
                                     ; space, if any.
                    (LaTeX-fill-move-to-break-point linebeg)

So one character back cancels this forward-char.

--
Masayuki Ataka (Japan)