Re: str:split, str:tokenize and empty tokens

On 25/10/2012 15:34, Daniel Veillard wrote:
> On Tue, Oct 23, 2012 at 05:00:31PM +0200, Nick Wellnhofer wrote:
>>
>> This seems wrong to me, and I couldn't find anything in the EXSLT
>> spec that mandates this behavior. Thoughts?
>
>    Do other implementations do the same, if yes too bad, work around
> it, if no and EXSLT doesn't mandate that behaviour, feel free to fix :-)

I tried to check Saxon and Xalan, but they don't seem to support 
str:split. But I had a look at the implementations in plain XSLT on the 
EXSLT website:

http://www.exslt.org/str/functions/split/str.split.template.xsl
http://www.exslt.org/str/functions/split/str.split.function.xsl

They both remove empty tokens intentionally. So we should keep the 
current behavior.

The fn:tokenize function in XPath 2.0 keeps empty tokens btw:

http://www.w3.org/TR/xpath-functions/#func-tokenize

"Zero-length strings will also occur in the result sequence if a 
separator occurs at the end of the $input string, or if two adjacent 
substrings match the supplied $pattern."

Nick