Re: Unicode strings and invalid codepoints

Pike strings are arrays of 32-bit numbers.
Some functions assume that they contain unicode characters, most don't.

What you suggest requires implementing a way to type strings depending on
their content, and then enforce the validity of the content based on the
type. Doing so would seem to be a lot of work for very little gain. How
many minutes/hours of developer time have you personally lost because pike
didn't detect U+D800 in unicode strings early enough?

      /Hubbe


On Fri, Mar 20, 2015 at 10:08 AM, Chris Angelico <[email protected]> wrote:

> On Sat, Mar 21, 2015 at 2:24 AM, Henrik Grubbström <[email protected]>
> wrote:
> >>> Well, as U+D800 is legal in UTF-16 strings (which BTW is why it is
> >>> reserved), I don't see a reason to prohibit it.
> >>>
> >>>> Are there situations in which it's necessary to be able to store these
> >>>> kinds of noncharacters in a string?
> >>>
> >>>
> >>>
> >>> Yes.
> >>
> >>
> >> UTF-16 is represented with individual bytes, though:
> >>
> >>> string_to_unicode("\U00012345");
> >>
> >> (1) Result: "\330\b\337E"
> >>>
> >>> String.string2hex(string_to_unicode("\U00012345"));
> >>
> >> (2) Result: "d808df45"
> >
> >
> > Well, take UCS-2 then.
>
> Which, I believe, disallows U+D800, bringing us back to the start.
>
> ChrisA
>
>
>