Re: Proposal: New CollationKey Byte Representation API additions

You may file the proposal under my name if you like. Since you are defining 
the UCharacterIterator class, it would be a huge improvement if you could 
create an instance with a character array in addition to a CharacterIterator 
(which introduces too much method calling overhead -- one call per 
character). It would be an additional improvement if there was a way to 
reuse the UCharacterIterator instances. Something like a reset() function 
that takes a character array. Finally, if you could add a version of 
getRawCollationKey() that takes a UCharacterIterator as a source string, 
things would really start cooking.

public RawCollationKey getRawCollationKey(UCharacterIterator source,
	RawCollationKey key)

My need for speed has a lot to do with the way I am using the ICU. In 
addition to building traditional database indexes, I am also constructing 
keyword indexes. The latter requires parsing out separate words from a 
record and building a key for each word. This can easily mean creating 
thousands of collation keys for a single record. Reducing the overhead of 
unnecessary object creation becomes increasingly important as the number of 
objects multiplies. I try to reuse objects and eliminate object sharing (to 
remove the need for synchronization) as much as possible.

Thanks again.

Geoff Mottram
Minaret Corp.
[email protected]


> Internally, the ICU4J collation module uses Strings and 
> CharacterIterators instead of character arrays. 
> CollationElementIterators are used to calculate the collation elements 
> of each character, and to traverse the source text 
> CollationElementIterator uses the class CharacterIterator.
> 
> This would change with jitterbug 2212, due for release 2.8, where ICU4J 
> collation will migrate to using com.ibm.icu.text.UCharacterIterator 
> instead. 
> http://www.jtcsv.com/cgibin/private/icu-bugs-private/collation?id=2212.
> 
> With the migration to using com.ibm.icu.text.UCharacterIterator, it will 
> be alot easier for us to add the method you proposed.
> 
> I would like file a seperate jitterbug for this proposed method, and if 
> you are interested I can file it under your name, so that the 
> implementor of the jitterbug could contact you directly if there are any 
> issues involved.
> 
> Syn Wee.
> 
> Geoff Mottram wrote:
> 
>> This would be a really useful enhancement to the ICU.
>>
>> Since you are almost certainly converting any String arguments into 
>> character arrays internally, is there any chance of adding the 
>> following as well?
>>
>> public RawCollationKey getRawCollationKey(char [] source, int offset,
>>     int count, RawCollationKey key)
>>
>> It's a shame if the caller has a character array (for efficiency) to 
>> have to create a String to pass to the ICU which then allocates a 
>> character array. Even if the ICU is using an internal buffer and not 
>> allocating one every time, couldn't it just as well copy from a source 
>> array instead of a String?
>>
>> Thanks.
>>
>> Geoff Mottram
>> Minaret Corp.
>> [email protected]
>>
>>
>> syn wee wrote:
>>
>>> Proposal: Addition of ByteArrayWrapper class, RawCollationKey class 
>>> and getRawCollationKey method.
>>>
>>> Please respond with comments by Oct 3rd 2003.
>>>
>>> This proposal is related to jitterbug 2732, 
>>> http://www.jtcsv.com/cgibin/icu-bugs?findid=2732.
>>>
>>> ICU recommends the use and storage of CollationKeys in a system where 
>>> comparison are to be done to the same String multiple times. See 
>>> http://oss.software.ibm.com/icu4j/doc/com/ibm/icu/text/RuleBasedCollator.html#getCollationKey(java.lang.String). 
>>>
>>> However, calling the current getCollationKey() method creates a new 
>>> CollationKey object everytime and access to the internal CollationKey 
>>> byte representation returns a copy of the byte array. To enable 
>>> further optimization for speed critical systems, ICU proposes the 
>>> following classes and method to allow user a more direct access to 
>>> the byte representation of the CollationKey.
>>>
>>> New class ByteArrayWrapper to reside in package com.ibm.icu.util
>>> /**
>>> * A simple wrapper utility class
>>> */
>>> public class ByteArrayWrapper {
>>>    public byte[] bytes;
>>>    /**
>>>     * Size of the internal byte array used. Different from 
>>> bytes.length, size will be <= bytes.length.
>>>     */
>>>    public int size;
>>>    /**
>>>     *Ensure that the byte array is at least of length 
>>> minCapacity.     *If the byte array is null or its length is less 
>>> than minCapacity, a new byte array of
>>>     * length minCapacity will be allocated.  The contents of the 
>>> array (between 0 and size)
>>>     * remain unchanged.
>>>     */
>>>    public void ensureCapacity(int capacity)
>>> }
>>>
>>> New class RawCollationKey to reside in package com.ibm.icu.text.
>>>
>>> /**
>>> * Simple class wrapper to store the internal byte representation of a 
>>> CollationKey.
>>> * Unlike the CollationKey, this class do not contain information on 
>>> the source
>>> * string the sort order represents.
>>> * RawCollationKey is mutable and users can reuse its objects with the 
>>> method in
>>> * RuleBasedCollator.getRawCollationKey(..).
>>> */
>>> public final class RawCollationKey extends ByteArrayWrapper {
>>>    /**
>>>     * Default constructor, internal byte array is null
>>>     */
>>>    public RawCollationKey() {
>>>    /**
>>>     * RawCollationKey created with a internal byte array of length 
>>> capacity
>>>     */
>>>    public RawCollationKey(int capacity) {
>>>    /**
>>>     * RawCollationKey created with a internal byte array bytes
>>>     */
>>>    public RawCollationKey(byte[] bytes) {
>>>    /**
>>>     * Compares 2 RawCollationKey objects.
>>>     * @return 0 if the sort order is the same,
>>>     *                 <0 if this RawCollationKey has a smaller sort 
>>> order than target,
>>>     *                 >0 if this RawCollationKey has a bigger sort 
>>> order than target.
>>>     */
>>>    public int compare(RawCollationKey target) {
>>> }
>>>
>>> New method getRawCollationKey to reside in the class 
>>> com.ibm.icu.text.RuleBasedCollator
>>> /**
>>> * Calculates the sort order of the String source and stores the byte 
>>> array representation of
>>> * the key into the user provided argument.
>>> * If key has a internal byte array of length that's too small for the 
>>> result,
>>> * the internal byte array will be grown to the exact required size.
>>> * @return If key is null, a new instance of RawCollationKey will be 
>>> created and returned,
>>> *                otherwise the user provided key will be returned.
>>> * @see #getCollationKey
>>> */
>>> public RawCollationKey getRawCollationKey(String source, 
>>> RawCollationKey key)
>>>
>>> Example of use:
>>>
>>> RuleBasedCollator collator = (RuleBasedCollator)Collator.getInstance();
>>> RawCollationKey key = new RawCollationKey(128);
>>> for (int i = 0; i < str.length; i ++) {
>>>    collator.getRawCollationKey(str[i], key);
>>>    // do something with key.bytes
>>> }
>>>
>>> _______________________________________________
>>> icu mailing list
>>> [email protected]
>>> http://oss.software.ibm.com/developerworks/oss/mailman/listinfo/icu
>>
>>
>>
>> _______________________________________________
>> icu mailing list
>> [email protected]
>> http://oss.software.ibm.com/developerworks/oss/mailman/listinfo/icu
>>
>>
>