Re: 7.1 string operations (was Re: Ann: SWI-Prolog 6.5.3)

Hi Richard,

On 11/26/2013 04:29 AM, Richard A. O'Keefe wrote:

> I agree that a Unicode-aware system doesn't need to mention Unicode
> over and over, but that's *except* where bytes are explicitly mentioned.
> I understand from earlier remarks in your message that this help text
> is now different.

Yes.  The old section on strings has been moved to the `extensions'
chapter, most predicates are renamed for consistency, predicates
were added and the general introduction and description of use
cases has changed.  Actually also considerably compared to what
is on the website today.  Read man/extensions.doc from the git
repo if you want to see the current status.

>>>     span_left(String, Set, Before[, Length[, After]])
>>>     span_right(String, Set[[, Before], Length], After)
>>> predicates that Xerox Quintus Prolog did?  -- quite helpful for
>>> tokenising.
>>
>> Found the docs.  I'm not really convinced.  Set is an atom,
>> code list or not(Set) as I understand it.
>
> You have mistaken the pips for the fruit.
> The fruit here is having *some* way to match a sequence
> of characters of some kind.  The particular representation
> of a set is just pips.  (As you may know, apple pips contain
> cyanide (:-).)

Point taken.

> It's *because* implementing sets of Unicode characters well is so
> complex that it doesn't belong in user code.

Which might be a good reason to leave that to an external regex library
if we can.  At least we can blame someone else :-)

>> We probably want a split-like predicate, similar to atomic_list_concat/3.
>> The naming is a bit hard as there is no atom in this name.
>> Maybe atomic_list_string(?List, +Separator, ?String)?
>
> Such a thing may well be useful, but it would be hideously
> inappropriate for the tasks that span_{left,right} were invented for,
> where you *just* want the first (or last) chunk, and the next time
> you will be using a *different* set.

I wasn't saying it was a replacement. I was only searching for things
that need to be added. Many languages have split-like operation. My only
problem with split is that it is commonly used as a cheap replacement
for proper parsing, introducing bugs.

>> Then there is long pending thing called regular expressions.  Strings
>> is the right moment to introduce them.  But, how?  Notable, which code
>> base and according to which of the many regex standards?
>
> I'd recommend re2.
> http://code.google.com/p/re2/
> Fast, low memory, multithread-friendly, and about as Perl-compatible
> as you can get while still being guaranteed linear time.
> Actively maintained, too; the latest version is 2013-10-24.

Thanks.  That is two votes.  We'll deal with that in a separate topic.

> This would do more than the span_left family, although it wouldn't
> substitute for the less commonly useful span_right.

It would avoid that we need to do Unicode character class classification
:-)  That makes me quite happy.

	Cheers --- Jan