Re: Regular Expressions Proposal

When dealing with word boundaries, is it going to be Unicode default 
behavior or will it be locale sensitive?

George Rhoten
IBM Globalization Center of Competency/ICU  San José, CA, USA




Andy Heninger <[email protected]>
Sent by: [email protected]
10/22/2003 03:41 PM
Please respond to Andy Heninger
 
        To:     [email protected]
        cc: 
        Subject:        Regular Expressions Proposal


This is a proposal for enhancement of the ICU Regular Expression
package, to address a few things that have come up with usage so far.

1.  Add new API
UBool RegexMatcher::lookingAt(int32_t stringIndex,
UErrorCode &status);

UBool RegexMatcher::matches(int32_t stringIndex,
UErrorCode &status);

These would function like the already existing lookingAt() and
matcthes() methods, but would check for a match starting at
the specified position.  The existing methods start at the
current position, which is either the start, or, with repeated
calls, the position following the previous match.


2.  Add new API
RegexMatcher &reset(int32_t stringIndex,
UErrorCode &status)

Removes any memory of previous matches, and sets the current
position of the RegexMatcher, causing subsequent find()
or matches() operations to begin at the specified position.

Report an error if the string index is out of bounds.

The existing reset() function sets the current position to
the beginning of the input string.

2.  Add an option for word boundaries ( \b in regex patterns) to
be determined according to the rules from Unicode UAX 29.
The current implementation is based on Perl's word boundaries,
and this (Perl conventions) would remain the default.

Unicode word boundaries mode will be selected by means of an
option flag.

To the enum of option flags in file regex.h, add
UREGEX_UWORD

To the list of flags that can be used within a pattern, add 'w'.
The existing flags are 'ismx', so the set of flags becomes 'ismwx'

Examples:

"(?iw)[aeiou].*?\b"   A pattern to match any word beginning with
a vowel [aeiou], with the flags
i  (case insensitive) and
w  (Unicode style word boundaries).

---


// Initialize a matcher object from a pattern in a string,
//   with the default word match mode set to
//   Unicode word boundaries.
RegexMatcher myMatcher(patternString, UREGEX_UWORD, status);


3.  Add new API
UBool  RegexMatcher::touchedEnd();

Return true if the most recent attempted match or match touched the
end of the input string.  For failed matches, this normally
means that some amount of additional input, appended to the
existing input string, could have resulted in a match.  (Note
that there are some degenerate cases involving look-around
operations where this is not true, but these generally involve
non-sensical patterns.)

The intent of this function is to allow a determination of
potential matches to be made while input text is incrementally
received.

These additions to ICU regular expressions are scheduled for ICU 2.8.
Any comments would be most appreciated.

-- Andy Heninger
[email protected]


_______________________________________________
icu mailing list
[email protected]
http://oss.software.ibm.com/developerworks/oss/mailman/listinfo/icu