Re: Unicode strings and invalid codepoints

When working with arrays of integers, strings are much more efficient than
array(int). They use less memory, are generally faster and can share memory
of many identical strings are created. This makes them ideal for storing
byte data (files), audio samples, vector graphics and many other things.
If array(int) was more cleverly implemented (I made some early attempts at
this, but eventually gave up) then the use case for string-as-data pretty
much goes away.  In fact, if array() was smart enough, then string would be
an alias for array(UCS32Character).

     /Hubbe

On Fri, Mar 20, 2015 at 10:31 AM, Chris Angelico <[email protected]> wrote:

> On Sat, Mar 21, 2015 at 4:27 AM, Fredrik Hubinette <[email protected]>
> wrote:
> > Pike strings are arrays of 32-bit numbers.
> > Some functions assume that they contain unicode characters, most don't.
> >
> > What you suggest requires implementing a way to type strings depending on
> > their content, and then enforce the validity of the content based on the
> > type. Doing so would seem to be a lot of work for very little gain. How
> many
> > minutes/hours of developer time have you personally lost because pike
> didn't
> > detect U+D800 in unicode strings early enough?
>
> That was where this question started. Is it something that's simply
> "not worth the effort of disallowing", or is there actually a solid
> use-case for needing those codepoints? If the easy implementation had
> been to disallow them, would it have been worth putting effort into
> allowing them?
>
> So far, all I'm seeing is that it's not going to make any difference
> either way, so the better option is the easier one - ie check nothing,
> and permit them all. Haven't heard from anyone who actually needs
> them.
>
> ChrisA
>
>
>