Re: RFC on ${^RNG} callback API for overriding rand()

On Thu, 20 Aug 2026 at 09:38, Michael Conrad <[email protected]> wrote:

> On 8/17/26 09:34, demerphq wrote:
> > Add support for an ${^RNG} hook
>
> The ^RNG variable makes sense to me, though adding a perl method call to
> every invocation of rand() seems less than ideal.
>
> > At the same time we dont really want to change Perls internal code to
> > use something other than drand48(). There is lots of code that expects
> > that doing srand(1234) will produce test results that look a specific way
> >
> Perhaps, but that pattern already experienced breakage with 5.20


Actually that isn't really true. Up till that point we used the system
random implementation, which on most of our target systems was already
Drand48, so it wasn't actually a change for most of our deployed surface,
as I recall the primary system that was affected (and was the motivator for
the change) was Win32 which had at the time I believe a 16 bit RNG.

Also up until that point we did not promise that using srand() would allow
you to randomize your tests safely.   Until we internalized our RNG
implementation you could not depend on srand() doing the same thing
everywhere, so that commitment only really started in 5.20

So we have really never experienced the fallout of changing the RNG.
Closest I am aware of is when I changed the hash function but in that case
the situation is a little different, we *wanted* to break tests, because we
*wanted* people to not build dependencies on the randomized order of keys.
In this case its the reverse.


> and in
> tests it's not *that* much extra hassle to globally define a custom
> CORE::rand, which is the better way to future-proof the tests anyway.
> It's basically the same problem that modules like Test::Mock::Time solve.
>

You are right, overriding is not that much hassle, although it is more
sensitive to load order than one would wish. But as I stated in my doc,
upgrading the RNG only solves a small part of the problem of dealing with
entropy in a programming language. I'd like to knock them all out with one
design.


>
> It might be nice if the rand implementation was selected lexically as a
> feature, so `use feature "rand_pcg";` could give you values from an
> internal PCG-XSH-RR by rand() calls in that scope, and would not affect
> rand() calls outside that scope.


Well that is nearly what I am proposing, except that the you have the
option of dynamic control over it because the var is exposed. Doing this as
a feature would involve hints, and essentially would be a similar solution
under the hood to using a global var, but it would just hide the var and be
less flexible. For instance how do you envisage that working with
srand()?*  Making this a compile time attribute just means you have to
answer a bunch of questions at compile time that are much more
appropriately delegated to the user and handled at run time.

* Eg, if i call srand() in one scope on one RNG does it affect the other?
With a global var we (the language designers) don't need to answer these
questions, its up to you at run time to choose one of the many options that
there are to set things up as you wish.


> Then it becomes a default in `use
> v5.48;` or something.


I can see some advantage to that but im not sure the advantage outweighs
the compleixty it will bring. I might give it a try tho just so see how it
plays together.


>   That preserves the srand()-in-tests behavior.
> ...unless they depend on the behavior of a module that changes to use
> this feature, but that was a potential breakage inherent in the choice
> to depend on srand() in the first place.
>

Yeah, but that is precisely the problem I want to solve. And solving *that*
problem means you get a solution to everything else, and can implement
everything else in terms of it. But the opposite isn't true.


>
> If the ${^RAND} variable doesn't add too much overhead, that could be
> added in addition to the feature flag, and override rand() regardless of
> feature flag.
>

Checking a package variable is relatively cheap.  If we make the var magic
then we can have updates to that var modify an internal attribute and
eliminate the package lookup overhead entirely. I haven't actually
implemented the latter yet, but only because it can be done later without
affecting the validity of the proposal.


>
> > I chose to make RNG use a UV as its base currency, so rand() calls
> > ${^RNG}->rand_uv() which returns a UV, which is then converted into
> > whatever the user requested. I did it this way so that any floating
> > point operations or conversion to floating point operations could be
> > performed by the C code. It feels like a more flexible foundation
> > than using a float.
>
> Randomizing a string of bytes could also be a good base
> currency.


You mean returning a string containing K bytes?  Yes, that is also an
option, but it uses a lot more memory and is less efficient to set up.
SvIV/SvUV are much lighter data structures than SvPV's are. Conceptually
there is little difference between an 8 byte random string and a UV. You
can pack one to be the other no problem.

I should have mentioned previously, one reason I dislike the U01 api for
random numbers is that it does not play nicely with tasks like generating a
fair dice roll with rejection sampling. The float hides some of the
important information to  be able to create a fair die.

For instance, if you have a U64 and you want to choose a random number from
1 to 113, converting the U64 into a float and then multiplying that by 13
doesnt give you a fair die. What gives you a fair die is something like

do {
    r = random_u64();
} while (r >= UINT64_MAX - (UINT64_MAX % 113));
return (r % 113) + 1;

I was actually think where would such tools go if we made perl better at
random number generators. I was thinking maybe in the RNG module.

Regardless of which it uses, it seems like a good idea to
> have a "RNG object" specification that people can standardize around.
>

Yeah I also agree that that would be very helpful.

cheers,
Yves

-- 
perl -Mre=debug -e "/just|another|perl|hacker/"