[Perl/perl5] 70c62a: Downgradable OP_CONST UTF-8 strings will not be pr...

  Branch: refs/heads/blead
  Home:   https://github.com/Perl/perl5
  Commit: 70c62a078bd23b42c996a8a69dafa570ddbba9b3
      https://github.com/Perl/perl5/commit/70c62a078bd23b42c996a8a69dafa570ddbba9b3
  Author: Richard Leach <[email protected]>
  Date:   2026-03-16 (Mon, 16 Mar 2026)

  Changed paths:
    M op.c
    M pod/perldelta.pod
    M pp_hot.c
    M t/comp/parser.t

  Log Message:
  -----------
  Downgradable OP_CONST UTF-8 strings will not be preemptively hekified

OP_CONST SVs containing valid strings that will be used directly as
hash keys can be converted at compile time to use shared HEKs.
Doing so improves the performance of hash operations.

Two main functions control this:
* `Perl_check_hash_fields_and_hekify()` - used for OP_MULTIDEREF
* `S_check_alt_hash_fields_hekify()` - used in hash initializers

GH #24266 revealed that the UTF-8 origins of strings downgraded
to single byte representations are lost during the conversion
and any subsequent use in hash _store_ operations. This breaks
the expectation that when keys come back out of the hash, such as
through the `keys` keyword, the encodings in the resulting SVs
are identical to the originally specified encodings.

Comprehensive tracking of UTF-8 origins, via the `HVhek_WASUTF8`
HEK flag, will require changes to `Perl_newSVpvn_share` and
`Perl_hv_common`, as well as additional functions TBD.

Considering the closeness to the v5.44.0 stable release date,
this commit takes the crude approach of pre-scanning UTF8
strings ahead of hekifying, rejecting those strings that can
be downgraded.

This will likely incur a performance decrease in `OP_MULTIDEREF`
when the `OP_CONST` SVs it subsumes contain downgradable UTF8.

Hopefully, all changes required to track the UTF-8 origins of
downgraded HEKs can be identified and implemented soon,
allowing this limitation to be reversed.



To unsubscribe from these emails, change your notification settings at https://github.com/Perl/perl5/settings/notifications