[PHP-DEV] [Concept] declare(strict_identifiers=1)

Hi internals,

I would like to gauge reaction before writing an RFC.

PHP's scanner defines identifiers on bytes rather than code points:

     LABEL  [a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]*

Every byte >= 0x80 is accepted, so UTF-8 identifiers work by accident;
the manual says as much. There is no encoding validation, no
normalisation requirement and no UAX #31 conformance. As a result
${"\xFF\xFE"} is a valid variable name, and $x and $x<U+00A0> are two
distinct variables that render identically.

I am not proposing that PHP accept more characters -- it already accepts
everything. I am proposing a per-file declare under which the accepted
set is specified: well-formed UTF-8, UAX31-R1-2 with the standard
Default-Ignorable Exclusion Profile, and NFC required rather than
applied. Identifiers consisting only of bytes below 0x80 are never
examined, so existing code pays nothing.

To find out what this would break I surveyed the 250 most-downloaded
Packagist packages and 250 GitHub repositories -- 168,604 PHP files --
using ext/tokenizer. The Packagist corpus contains exactly one
non-ASCII identifier, and no identifier in either corpus is non-NFC.
Tooling, raw data and per-identifier CSVs are here:
     https://github.com/Otzie2023/PHP

The patch would not touch the re2c scanner: the byte rule is already
maximally permissive and cannot split a UTF-8 sequence, so a strict
mode only ever rejects and the check can run after the token is formed.
About 13.7 KiB of generated tables, with no dependency on intl, ICU or
mbstring. I would write and maintain it.

Is this worth an RFC, or is there an objection I should know about
before I spend the time?


Regards,
Luca Rodenhaeuser