CPAN-based Experiments: A Reminder

I've been adding new features to core perl for a number of years. For
the most part, all of the big things I've been adding have been
near-copies of existing abilities in existing CPAN modules. As a
reminder, this list contains:

  perl 5.34:
    try {} catch {} -- from Syntax::Keyword::Try

  perl 5.36:
    builtins     -- from Scalar::Util and friends
    defer {}     -- from Syntax::Keyword::Defer
    booleans     -- this was actually new

  perl 5.38:
    class { }               -- from Object::Pad
    ||=, //= in signatures  -- from Sublike::Extended

  perl 5.40:
    field :reader, __CLASS__ -- from Object::Pad

  perl 5.42:
    my method, field :writer   -- from Object::Pad
    lexical method invoke ->&  -- from Object::Pad::LexicalMethods
    builtin any, all           -- from List::Util + List::Keywords

  (upcoming in perl 5.44):
    named :$params in signatures  -- from Sublike::Extended

I specifically bring attention to the distinguished booleans ability of
perl 5.36, because it is the *only* major addition I believe I've added
in recent years that didn't just come from a CPAN experiment. I would
have done that on CPAN if I could have, but it involved such deep
internal changes to the very fundamental building blocks in perl that I
don't think it would have been possible. [1].

Everything else was.

At conferences and discussions I enjoy telling the story of the first
item on this list. I had spent about 3 years experimenting with
`Syntax::Keyword::Try` as a CPAN module. The nature of the experiment
was not "can we get this working?". Getting it working was the easy bit.
The hard part was the Perl-visible design of the syntax. We went
through a number of iterations there before we ended up with something
that looked and felt right, and seemed to gel nicely with other ideas.
This was the point at which the idea got copied into core perl. That
process involved just copy-pasting the unit tests, rewriting some
documentation, and actually reïmplementing the code. The entire time
for implementing the feature once it was solidly designed and tested
took me about 5 days. That's all it took because we had that design.

When it came time to review the change to put it into core perl, it was
easy to make the case that the design of it was right. It had existed
for years on CPAN and been battle-tested by lots of users in lots of
ways.

This overall principle has two key advantages:

 * Because every idea has plenty of time (often years) of existing as a
   CPAN module, we have had time to feel around for any design flaws or
   usability issues. Remember the disaster that was perl 5.10's
   smartmatch and given/when? We're still to this day trying to
   back-pedal out of that one. I'm not going to confidently say that
   kind of thing wouldn't have happened if given/when was first a
   CPAN-based experiment, but it feels likely that if more people had
   tried using it in more ways, at least some of the issues would have
   been more apparently and the whole idea redesigned or abandoned
   entirely.

 * Every one of these existing CPAN experiments works on existing
   versions of perl; often going back to quite old ones. Even the most
   complex of my syntax modules, Object::Pad, runs just fine on perl
   5.22 which was released over 10 years ago now. This means that people
   don't have to wait for the next bleadperl or development point
   release just to try out these new ideas. They can install something
   from CPAN and use these new things right now in existing code, on
   existing systems.

These CPAN-based experiments have all worked because of the much faster
iteration time on CPAN modules as compared to core perl. We get one
major new release of perl every year. Sometimes I've been known to
release three versions of a given CPAN module in a single week. That's
at least a hundred times faster iteration speed to work on a new idea.
I feel quite confident in saying that almost none of the features I
listed above would have been possible by now had I been given only core
perl to start experimenting in.

We also get a much gentler "adoption curve", if such is a way to phrase
it. In core perl, we have "experimental" features, but once something
becomes non-experimental, it's declared stable and basically must be
supported by perl for all of time. CPAN modules feel like they have
much more finer levels than that. Within `Object::Pad` for instance,
there's definitely quite a range from "this is something I thought up
the other day so I want to see how it plays out", to "here is a stable
supported feature that should be copied into core perl". Ideas can
slowly graduate along the scale at their own pace.

As a direct consequence of all of the above, we then have the
`Feature::Compat::` modules. Because core's `try` feature was directly
and deliberately implemented as compatible with `Syntax::Keyword::Try`,
there is a module called `Feature::Compat::Try` which simply enables
the `try` feature on a sufficiently new perl, or pulls in the CPAN
module for it on older perls. It gives authors of Perl code a smooth
upgrade path towards using stable core-supported features. Once a
`Feature::Compat` module exists for a feature you want to use, you can
just use that module and not worry whether it is yet supported by the
perl version you are running.

I started writing this post because I had a specific point to make
about `Object::Pad` and roles, but I feel this has gone on quite long
enough already so I will write that in a follow-up shortly.

Instead, for now I will end by reminding people not to be afraid of the
word "experimental". It doesn't mean "we're not sure this thing works";
we know it works. The focus of the experiment is "do we like this?" -
and by "we" here I specifically mean the entire Perl community - the
core maintainers, the CPAN authors, the end-user developers. We don't
want to end up in another smartmatch scenario of having a
poorly-designed feature suddenly dropping into core untested but
needing to be supported. This gradual experiment path through CPAN
allows people to try out these new ideas and give feedback which helps
guide their design to something solid and stable, that everyone can be
happy about supporting long-term. But this entire mechanism only works
if people use these modules and give that feedback.

It's probably no surprise to folks that I dogfood[2] a lot of my own
modules, but in some cases I am the only user of those modules that I
know about. I am personally happy to continue designing core perl
features just suited to my own personal use, but I suspect many other
of you would have your own opinions to provide. Please all, try to
remember to try things out. Talk to me about these things, use some of
these CPAN experiments where and when you can. Or even if you can't
directly use them for some reason, just look at them and talk to me
anyway. The more feedback I get from more varied people across more
varied use-cases, the more confidence I have in the correctness of
these designs, and hence the more likely I am to spend the time to
migrate them into the core language.


[1]: I would also have done 5.32's `isa` operator on CPAN too, but at
     the time we didn't have the `PL_infix_plugin` mechanism, which I
     later added in 5.38 to allow exactly this sort of thing in future.
     It has been very helpful in designing a number of new operators
     since then.

[2]: https://en.wikipedia.org/wiki/Eating_your_own_dog_food


This post also appears at
https://leonerds-code.blogspot.com/2026/03/cpan-based-experiments-reminder.html

-- 
Paul "LeoNerd" Evans

[email protected]
http://www.leonerd.org.uk/  |  https://metacpan.org/author/PEVANS