Re: Proposal: New CollationKey Byte Representation API additions

Hi Geoff,

Yes, I agree, we would have to add extra APIs to allow reuse of 
UCharacterIterator.
I have just submitted a jitterbug under your name for the 2 proposed API
public RawCollationKey getRawCollationKey(UCharacterIterator source, 
RawCollationKey key)
public RawCollationKey getRawCollationKey(char[] source, RawCollationKey 
key)
The bug number is 3270.
http://www.jtcsv.com/cgibin/icu-bugs?findid=3270

Syn Wee.

Geoff Mottram wrote:

> You may file the proposal under my name if you like. Since you are 
> defining the UCharacterIterator class, it would be a huge improvement 
> if you could create an instance with a character array in addition to 
> a CharacterIterator (which introduces too much method calling overhead 
> -- one call per character). It would be an additional improvement if 
> there was a way to reuse the UCharacterIterator instances. Something 
> like a reset() function that takes a character array. Finally, if you 
> could add a version of getRawCollationKey() that takes a 
> UCharacterIterator as a source string, things would really start cooking.
>
> public RawCollationKey getRawCollationKey(UCharacterIterator source,
>     RawCollationKey key)
>
> My need for speed has a lot to do with the way I am using the ICU. In 
> addition to building traditional database indexes, I am also 
> constructing keyword indexes. The latter requires parsing out separate 
> words from a record and building a key for each word. This can easily 
> mean creating thousands of collation keys for a single record. 
> Reducing the overhead of unnecessary object creation becomes 
> increasingly important as the number of objects multiplies. I try to 
> reuse objects and eliminate object sharing (to remove the need for 
> synchronization) as much as possible.
>
> Thanks again.
>
> Geoff Mottram
> Minaret Corp.
> [email protected]
>
>
>> Internally, the ICU4J collation module uses Strings and 
>> CharacterIterators instead of character arrays. 
>> CollationElementIterators are used to calculate the collation 
>> elements of each character, and to traverse the source text 
>> CollationElementIterator uses the class CharacterIterator.
>>
>> This would change with jitterbug 2212, due for release 2.8, where 
>> ICU4J collation will migrate to using 
>> com.ibm.icu.text.UCharacterIterator instead. 
>> http://www.jtcsv.com/cgibin/private/icu-bugs-private/collation?id=2212.
>>
>> With the migration to using com.ibm.icu.text.UCharacterIterator, it 
>> will be alot easier for us to add the method you proposed.
>>
>> I would like file a seperate jitterbug for this proposed method, and 
>> if you are interested I can file it under your name, so that the 
>> implementor of the jitterbug could contact you directly if there are 
>> any issues involved.
>>
>> Syn Wee.
>>
>> Geoff Mottram wrote:
>>
>>> This would be a really useful enhancement to the ICU.
>>>
>>> Since you are almost certainly converting any String arguments into 
>>> character arrays internally, is there any chance of adding the 
>>> following as well?
>>>
>>> public RawCollationKey getRawCollationKey(char [] source, int offset,
>>>     int count, RawCollationKey key)
>>>
>>> It's a shame if the caller has a character array (for efficiency) to 
>>> have to create a String to pass to the ICU which then allocates a 
>>> character array. Even if the ICU is using an internal buffer and not 
>>> allocating one every time, couldn't it just as well copy from a 
>>> source array instead of a String?
>>>
>>> Thanks.
>>>
>>> Geoff Mottram
>>> Minaret Corp.
>>> [email protected]
>>>
>>>
>>> syn wee wrote:
>>>
>>>> Proposal: Addition of ByteArrayWrapper class, RawCollationKey class 
>>>> and getRawCollationKey method.
>>>>
>>>> Please respond with comments by Oct 3rd 2003.
>>>>
>>>> This proposal is related to jitterbug 2732, 
>>>> http://www.jtcsv.com/cgibin/icu-bugs?findid=2732.
>>>>
>>>> ICU recommends the use and storage of CollationKeys in a system 
>>>> where comparison are to be done to the same String multiple times. 
>>>> See 
>>>> http://oss.software.ibm.com/icu4j/doc/com/ibm/icu/text/RuleBasedCollator.html#getCollationKey(java.lang.String). 
>>>>
>>>> However, calling the current getCollationKey() method creates a new 
>>>> CollationKey object everytime and access to the internal 
>>>> CollationKey byte representation returns a copy of the byte array. 
>>>> To enable further optimization for speed critical systems, ICU 
>>>> proposes the following classes and method to allow user a more 
>>>> direct access to the byte representation of the CollationKey.
>>>>
>>>> New class ByteArrayWrapper to reside in package com.ibm.icu.util
>>>> /**
>>>> * A simple wrapper utility class
>>>> */
>>>> public class ByteArrayWrapper {
>>>>    public byte[] bytes;
>>>>    /**
>>>>     * Size of the internal byte array used. Different from 
>>>> bytes.length, size will be <= bytes.length.
>>>>     */
>>>>    public int size;
>>>>    /**
>>>>     *Ensure that the byte array is at least of length 
>>>> minCapacity.     *If the byte array is null or its length is less 
>>>> than minCapacity, a new byte array of
>>>>     * length minCapacity will be allocated.  The contents of the 
>>>> array (between 0 and size)
>>>>     * remain unchanged.
>>>>     */
>>>>    public void ensureCapacity(int capacity)
>>>> }
>>>>
>>>> New class RawCollationKey to reside in package com.ibm.icu.text.
>>>>
>>>> /**
>>>> * Simple class wrapper to store the internal byte representation of 
>>>> a CollationKey.
>>>> * Unlike the CollationKey, this class do not contain information on 
>>>> the source
>>>> * string the sort order represents.
>>>> * RawCollationKey is mutable and users can reuse its objects with 
>>>> the method in
>>>> * RuleBasedCollator.getRawCollationKey(..).
>>>> */
>>>> public final class RawCollationKey extends ByteArrayWrapper {
>>>>    /**
>>>>     * Default constructor, internal byte array is null
>>>>     */
>>>>    public RawCollationKey() {
>>>>    /**
>>>>     * RawCollationKey created with a internal byte array of length 
>>>> capacity
>>>>     */
>>>>    public RawCollationKey(int capacity) {
>>>>    /**
>>>>     * RawCollationKey created with a internal byte array bytes
>>>>     */
>>>>    public RawCollationKey(byte[] bytes) {
>>>>    /**
>>>>     * Compares 2 RawCollationKey objects.
>>>>     * @return 0 if the sort order is the same,
>>>>     *                 <0 if this RawCollationKey has a smaller sort 
>>>> order than target,
>>>>     *                 >0 if this RawCollationKey has a bigger sort 
>>>> order than target.
>>>>     */
>>>>    public int compare(RawCollationKey target) {
>>>> }
>>>>
>>>> New method getRawCollationKey to reside in the class 
>>>> com.ibm.icu.text.RuleBasedCollator
>>>> /**
>>>> * Calculates the sort order of the String source and stores the 
>>>> byte array representation of
>>>> * the key into the user provided argument.
>>>> * If key has a internal byte array of length that's too small for 
>>>> the result,
>>>> * the internal byte array will be grown to the exact required size.
>>>> * @return If key is null, a new instance of RawCollationKey will be 
>>>> created and returned,
>>>> *                otherwise the user provided key will be returned.
>>>> * @see #getCollationKey
>>>> */
>>>> public RawCollationKey getRawCollationKey(String source, 
>>>> RawCollationKey key)
>>>>
>>>> Example of use:
>>>>
>>>> RuleBasedCollator collator = 
>>>> (RuleBasedCollator)Collator.getInstance();
>>>> RawCollationKey key = new RawCollationKey(128);
>>>> for (int i = 0; i < str.length; i ++) {
>>>>    collator.getRawCollationKey(str[i], key);
>>>>    // do something with key.bytes
>>>> }
>>>>
>>>> _______________________________________________
>>>> icu mailing list
>>>> [email protected]
>>>> http://oss.software.ibm.com/developerworks/oss/mailman/listinfo/icu
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> icu mailing list
>>> [email protected]
>>> http://oss.software.ibm.com/developerworks/oss/mailman/listinfo/icu
>>>
>>>
>>
>
> _______________________________________________
> icu mailing list
> [email protected]
> http://oss.software.ibm.com/developerworks/oss/mailman/listinfo/icu
>
>