[core-dev] [limewire jira] Commented: (CORE-5) asian language search stifled by 3+ character search requirement
[email protected] Tue, 7 Sep 2004 12:26:00 -0400 (EDT)
| Newsgroups | gmane.network.gnutella.limewire.core.devel |
|---|---|
| Message-ID | <[email protected]> |
DO NOT REPLY TO THIS EMAIL. PLEASE POST YOUR COMMENTS THROUGH THE WEB INTERFACE AT http://www.limewire.org/jira/browse/CORE-5 ANY REPLY MADE TO THIS MESSAGE WILL BE DISCARDED. The following comment has been added to this issue: Author: Philippe Verdy Created: Tue, 7 Sep 2004 12:25 PM Body: Instead of counting characters (i.e. UTF-16 code units with Java Strings), we may just need to count UTF-8 encoded bytes corresponding to the tested keywords. This make 1 byte for 7-bit ASCII characters, 2 bytes for most European characters up to U+03FF, 3 bytes for all other characters in the BMP, 4 bytes for rare characters out of the BMP (in Java they are represented by a UTF-16 surrogate pair, so we may count 6 bytes if we don't want to convert surrogate pairs to a single UTF-32 codepoint). The three bytes limit corresponds then to: - 3 characters if only ASCII - 2 characters if there's at least 1 European non-ASCII character. - 1 character for ideographic characters. We don't need to make the actual conversion to UTF-8. We just need to see if the keyword length if lower than 3 and then check if one of them is not ASCII (lower than U+0080). This is very fast to do, and will work on the network because search strings are UTF-8 encoded and their incoming length is checked against the UTF-8 encoding... --------------------------------------------------------------------- View this comment: http://www.limewire.org/jira/browse/CORE-5?page=comments#action_10040 --------------------------------------------------------------------- View the issue: http://www.limewire.org/jira/browse/CORE-5 --------------------------------------------------------------------- This message was generated by JIRA, LimeWire.org's issue tracking system. If you think it was sent incorrectly contact one of the administrators: http://www.limewire.org/jira/secure/Administrators.jspa _______________________________________________ core-dev mailing list [email protected] http://www.limewire.org/mailman/listinfo/core-dev