Re: [Perl/perl5] 502da7: make /p a no-op

On Thu, 12 Feb 2026 at 16:36, Philippe Bruhat (BooK) <[email protected]>
wrote:

> Hi,
>

Hey man! Hope you are good.


> On Thu, Feb 12, 2026 at 06:00:35PM +0800, demerphq wrote:
> > On my phone so forgive the double reply. The code i mean is something
> like
> > this:
> >
> > while (<>) {
> >   while (/(.)/g) { $count++}
> > }
> >
> > Try that on a large file. If it appears to hang then /p should not be
> > removed.
>
> Thanks Yves for bringing up this issue.
>
> I tried this (also reported on the GitHub ticket):
>
>     $ time perl -E 'say $^V;while(<>){while(/(.)/g){$count++}}say$count'
> /tmp/big
>     v5.36.0
>     101876091
>
>     real    0m16.232s
>     user    0m16.208s
>     sys     0m0.020s
>
>     # this is the Perl built from https://github.com/Perl/perl5/pull/24191
>
>     $ time ./perl -Ilib -E 'say
> $^V;while(<>){while(/(.)/g){$count++}}say$count' /tmp/big
>     v5.43.8
>     101876091
>
>     real    0m48.173s
>     user    0m48.110s
>     sys     0m0.060s
>
> So there's definitely a degradation
>

Did you double check you were using the same build options? I did something
similar and ended up comparing a DEBUGGING and release build.  My other
post about COW and $_ and /p modifier is related to this. I havent had time
yet to dig more deeply to see what the story is.


> > > Every time people proposed removing /p when I have  checked while /g it
> > > has been broken.
>
> I'm a little bit confused about the whole $& vs ${^MATCH} situation.
> The docs keep saying there was horrible degradation before 5.20, but
> that everything is fixed since 5.20 and COW. And still things don't
> appear as simple.
>

As always this is a bit of a twisty set of circumstances... And
involves/involved multiple bugs/issues/design flaws.

Let me just summarize the situation and history a bit as I understand it
just for the record.

So, the core problem is that when we match and have capture buffers in the
pattern we lazily populate the buffers based on offsets into the matched
string, and is $1 is a magic variable which behaves like a tied scalar
which reads the appropriate segment of the matched string.  This means we
have to manage the string being updated in between the match and the use of
$1 etc.  In order to manage this in most cases when there is a capture
buffer we copy the string before the match, and store the copy in the
current match context object.

Similarly the use of $& *anywhere in the running perl context* would force
this copy behavior for all matches, even when the pattern did not contain
capture buffers. (PL_sawampersand). This meant that use English was avoided
by most people because it mapped $MATCH to $&. (The same rules applied to
$` and $'.) At some point we changed use English to only support $& on
request and it became a little more commonly used, but was still generally
avoided by CPAN code that needed to work on older perl with the older use
English implementation.  This had to be a global var because it is hard
(maybe Turing Complete hard, but definitely harder than was reasonable for
the perl internals) to determine which match a $& is going to access.

There was one place where this copying was disabled (i'm not sure if it was
always disabled, or just when PL_sawampersand was not used), and that is
for scalar context /g matches, and this was because we don't want

while (/(.)/g) { }

to be quadratic. It should be linear. But this meant that if you messed
with the target of the pattern, you could end up messing up capture
buffers. So in older perls code like this:

$str = "abcdef";
$str =~ /(\w+)/g;
$str = "123456";
print $1;

will output "123456" and not "abcdef" and if you go far enough back, if the
modified string was shorter than the original you could even trigger a SEGV.

At some point I added the support for /p and created ${^PREMATCH},
${^MATCH} and ${^POSTMATCH}.  The idea of this was that these vars are only
populated if the /p modifier was used on a match. So if you really wanted
to use $& without paying a global performance penalty you could use
${^MATCH} instead, and it was up to you make sure that when you did was
after a successful match which used /p.

Even later on COW was added. COW in theory is a solution to the quadratic
problem, as it makes copying the string in the match relatively efficient.
(O(1) performance for the copy instead of O(length_of_string).

So then support for /p was removed from various places.

Unfortunately the COW stuff didn't work very well with readline(). We do
some gnarly speed optimizations to make readline fast, which disabled COW
as well.

So the two changes cancelled each other out and scalar /g matches were
again quadratic. We eventually fixed this somewhat by changing how readline
works. (I fixed it, but i dont rememebr exactly how, i wrote a huge comment
about it at the time if you care to read it.

However, this fix wasnt a complete fix. It didn't fix the performance
problem of scalar /g matches, it just avoided the quadratic behavior by
making sure readline returned COWable strings. The old quadratic behavior
still existed and could be produced by using a non COW-able string, which
are rare and but occasionally encountered beasts in various contexts.

So I essentially reverted the NO-OP of the /p modifier. I guess when I did
it I missed some docs so it continued to be documented as a no-op even
though it was not.

>
> Is there a difference between the punctuation variables ($`, $& and $')
> and the long name versions (${^PREMATCH}, ${^MATCH} and ${^POSTMATCH})?
> What is it? And what is the relation with PL_sawampersand, which has
> apparently been disabled since 1a904fc88069e249a4bd0ef196a3f1a7f549e0fe
> in 2012?
>

I thought I reverted the removal. Maybe i only did a partial revert which
didnt affect $& directly but did affect scalar /g matches.


> Should we open an issue to completely remove the code related to
> PL_sawampersand?
>

It depends. I dont think we should until we eliminate the quadratic vector
or alternatively as I posted earlier, when we try to make the copy if the
string is not COWable we can warn. If we warn when people are doing scalar
/g matches on an unCOWable string at least the user will know and have some
guidance on how to avoid the problem: typically simply copying the value
into a "normal" sv before the scalar /g match will do the trick.


>
> I've tried these in the hope of understanding the relationship
> between the puncation variables and the long name ones:
>
>     # Using $& doesn't seem to make a difference
>
>     $ time perl -E 'say
> $^V;while(<>){while(/(.)/g){$&;$count++}}say$count' /tmp/big
>     v5.36.0
>     101876091
>
>     real    0m15.719s
>     user    0m15.705s
>     sys     0m0.013s
>
>     # Using /p and ${^MATCH} doesn't seem to make a difference either
>
>     $ time perl -E 'say
> $^V;while(<>){while(/(.)/gp){${^MATCH};$count++}}say$count' /tmp/big
>     v5.36.0
>     101876091
>
>     real    0m15.739s
>     user    0m15.723s
>     sys     0m0.017s
>
> (Both of those suck like the first example with my patch bleadperl,
> of course.)
>
> In my limited understanding, the change I made (removing a conditional
> goto) was equivalent to having /p always enabled. That doesn't seem to
> be the actual result.
>
> I'm clearly out of my depth here.
>
> I started looking at this, because I believe that for PPC0014 (English
> name aliases) to be complete, we want the caret long name match variables
> to be simple aliases to the punctuation ones. Which I think means /p
> must be a true no-op.
>

I dont see why resolving the other vars should be blocked by these
particular vars.  They have always been special, even with 'use English' in
operation.

Having said that, these days probably the best is to warn when we match
against an unCOWable string, and then let the user deal with it.

Yves

-- 
perl -Mre=debug -e "/just|another|perl|hacker/"