Re: Unicode Character 'PILE OF POO' (U+1F4A9) and validator test suite

Hi Mark,

Mark Rogers <[email protected]>, 2014-11-08 13:59 -0600:

> Is the Unicode character U+1F4A9 used in the conformance checker test
> suite for URLs really invalid?

No, it's valid. Thanks for catching this and taking time report it.

> It’s marked as novalid in test suite files like:
> 
> conformance-checkers/html/elements/a/href/userinfo-username-contains-pile-of-poo-novalid.html

Yeah, I'll need to fix that. But before I do, I'll wait for a fix to the
upstream code of the URL parsing library the validator uses, called
galimatias. I've already filed a pull request with a proposed fix:

  https://github.com/smola/galimatias/pull/46

I expect that'll get fixed relatively soon.

> In RFC 3987 this character is listed in the 10000-1FFFD  range in the
> iuserinfo  -> iunreserved -> ucschar production:
> 
> iuserinfo      = *( iunreserved / pct-encoded / sub-delims / ":" )
> 
> iunreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~" / ucschar
> 
>    ucschar        = %xA0-D7FF / %xF900-FDCF / %xFDF0-FFEF
>                   / %x10000-1FFFD / %x20000-2FFFD / %x30000-3FFFD
>                   / %x40000-4FFFD / %x50000-5FFFD / %x60000-6FFFD
>                   / %x70000-7FFFD / %x80000-8FFFD / %x90000-9FFFD
>                   / %xA0000-AFFFD / %xB0000-BFFFD / %xC0000-CFFFD
>                   / %xD0000-DFFFD / %xE1000-EFFFD
> 
> In the Whatwg URL standard it’s listed as a valid URL code point, and
> will be converted to percent encoding during the normalisation process,
> but won’t flag an error. See
> https://url.spec.whatwg.org/#url-code-points
> https://url.spec.whatwg.org/#authority-state

Yup. Your reading of the spec is right. I'd made the mistake of being lazy
and having the test suite just follow the (buggy in this particular case)
behavior galimatias on this, rather than checking it against the spec.

I'll follow up here after I've got it all fixed.

  --Mike

-- 
Michael[tm] Smith https://people.w3.org/mike

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iQIcBAEBCgAGBQJUYBYxAAoJEIfRdHe8OkuVg4cP/1TlAIgNjaiXEncBD0BQG7dY
REOR9YCdKGWKaBdB+8lOwcZ39Dew7MjezxYtMHF9dY8gjsVmI9OCaRKqRNPp8OmD
Sq2LlhmWBMJfPVarQr/tnjJUS2CvvSVXU7ghuyc/+bU+DY4VyU1XvxpD2hKdKoeg
sPmDZ8GMCy/gSo9ZZHz2HRMIfsCHlqp4VXZAeB4PQMRLjAtv6IvZjOUX1Dr8I0us
hms69frigHtmAfGXqQO1uawOlvmgicXiTy5TOp/74T3v56uEDMWcdaTOzbBaV+VP
KpHVfUCIe3ZkQt9T96kmzYOlluXoO+9AiYakiRBjrvTWq21XRYbSwcA2s3qtBLss
V37dPz8EV+6hRK/g7heYI0MP2QXZnzQehobCibVFY5IXs/4avoXy52mDAKwI3BbQ
+NQJuBxrTiNcnD33Pza5AabqU+hwZS5EOv0acHX1ryu0kzcXsj1heKvm+x7K2/cf
gHgn0XIwSbcD8IPaGjxMJ+2aCoqFxwx+M0JE+YFePOkBPGhxf0Nr8+iMkRHdNXcD
VcgMKj8CPaWvWMGQt0iRgYZGMxV24zykExDX1H3rEMdOx7xhRDRt33FXUxpnjKsW
ZBRERjG9OufVhOVRJ/M2Q4oRrZKdyuRbReenD3lT/deJa/P5MQBNW3tZT20ayo3p
SmqdlKc409XDaoANVgU/
=TLgF
-----END PGP SIGNATURE-----