Re: Issues with japanese support

* Masayuki Ataka (2005-02-15) writes:

> From: Ralf Angeli <[email protected]>
> Subject: Re: Issues with japanese support
> Date: Mon, 14 Feb 2005 10:19:33 +0100
>
>> Here are some comments regarding the changes:
>> 
> Hi Ralf, thank you for your comments.
>  
> I fixed as followed:
[...]
>> 3) I am not sure about the performance implications of this change.
>>    The function to get rid of newlines in the non-Japanese case is
>>    `subst-char-in-region' which is implemented in C and should be
>>    quite fast.  The `re-search-forward-statement' is likely to slow
>>    things down.
>> 
> Which character do you use for TOCHAR?
> We want to substitute `\n' by empty, not space.

I didn't propose to use `subst-char-in-region' as it doesn't allow for
the control we need to distinguish between Japanese and non-Japanese
characters.

>> 4) Is it sufficient to search for \cj or are there other character
>>    categories which have to be treated the same way?
>> 
> At least, it is sufficient for Japanese and not sufficient for 
> Chinese and Korean.  But I am not a native of Chinese or Korea,
> so that I can't say space should be gotten rid of when filling
> in Chinese and Korean like in Japanese.
>
> May be \c| is sufficient, but its document says:
>
> (category-docstring ?|)
>  => While filling, we can break a line at this character.
>
> That is not equal that no space should be removed.
> Please negatives of Chinese and Korean give us advices.

Hm, I just looked at the code doing this newline squashing in CVS
Emacs and Emacs 21.3:

  (goto-char from)
  (if enable-multibyte-characters
      ;; Delete unnecessay newlines surrounded by words.  The
      ;; character category `|' means that we can break a line
      ;; at the character.  And, charset property
      ;; `nospace-between-words' tells how to concatenate
      ;; words.  If the value is non-nil, never put spaces
      ;; between words, thus delete a newline between them.
      ;; If the value is nil, delete a newline only when a
      ;; character preceding a newline has text property
      ;; `nospace-between-words'.
      (while (search-forward "\n" to t)
	(if (get-text-property (match-beginning 0) 'fill-space)
	    (replace-match (get-text-property (match-beginning 0) 'fill-space))
	  (let ((prev (char-before (match-beginning 0)))
		(next (following-char)))
	    (if (and (or (aref (char-category-set next) ?|)
			 (aref (char-category-set prev) ?|))
		     (or (get-charset-property (char-charset prev)
					       'nospace-between-words)
			 (get-text-property (1- (match-beginning 0))
					    'nospace-between-words)))
		(delete-char -1))))))

This is executed after the code for adding an extra space to sentence
endings.

Why don't we just stick this into `LaTeX-fill-region-as-para-do'?  We
could even cut out the whole `(if (fboundp 'fill-delete-newlines) ...'
construct and put it into a new `LaTeX-fill-delete-newlines' function
which would make `LaTeX-fill-region-as-para-do' more readable.

-- 
Ralf