AV API thoughts: AVs as proxies for other things

Here's a loose collection of ideas that came to mind while thinking of
how to handle the `av_splice()` API, as observed in
  https://github.com/Perl/perl5/pull/24451
There's no immediate questions to answer or action to perform, but I
wanted to write it out while it's in my head in case it struck and
bells with anyone else, and we manage to come to some eventual plan of
work that might become useful one day...


The currently-last comment in that PR I liked above, is some thinking
out loud on the complicated problem with the C-level API it proposes. 
It observes that if at the C level you call `av_splice(...)` on a given
AV that is tied, there's no guarantee that the Perl `SPLICE` method that
gets invoked returns the right number of deleted items. In particular,
it could return more, which would be too many items for the caller to
account for in the C-level array it passed in for the purpose. Much
trouble. My comment ends with the observation that perhaps a workable
API would be to have three variants of the function, which differ in
the way they pass in or out a list (sequence?) of SVs for the splice
operation. The three variants could use C-level arrays of SV pointers,
perl's AV structures, or the perl data stack.

But the more I thought on this, the more I realise this is definitely
not a design problem specific to `av_splice()`, and could apply
basically anywhere in perl APIs that wants to receive or return lists
of SVs. That's quite a lot. I don't think we'd want three copies of
every single API function that might want to take (or return) a list of
SVs. And of course it gets even worse with things like `av_splice` that
might want to take and return lists in two different forms - do we
write nine variants of that? Madness!

So to avoid all that madness I started thinking that between various
hackery on AVs (things like adjusting AvALLOC, AvMAX and AvREAL), it
would be fairly easy to make an AV that really just acts as a proxy to
a C array directly. A complex function such as `av_splice()` could use
a reduced subset of the AV API to operate on its input and output list
arguments, and not have to care if that AV is actually a proper
perl-level array data structure, or was in fact a proxy to a C array.
Or perhaps even the perl data stack directly. Thus, rather than us
having to write three functions for three variants of "values from AV,
values from C array, values from perl stack", what if everything just
took an AV, but we had a few functions to build little proxy AVs for C
arrays or the perl stack.

For example: Imagine we had an `av_push_av`, which pushes all the
values from a source AV onto the end of its destination AV.

  void av_push_av(AV *av, AV *src);

Well we wouldn't need variants to push from a C array or from the perl
stack, if instead we had some API like

  AV *newAVproxy_array(SV **svs, size_t len);
  AV *newAVproxy_from_stack(size_t count);

then we wouldn't need any other variants of `av_push_av`. We'd just be
able to pass them in a little proxy AV for one of these scenarios, and
all would work out.

  /* push from a C array */
  av_push_av(av, newAVproxy_array(more_svs, count_of_svs));

  /* push the top 5 items from the perl values stack */
  av_push_av(av, newAVproxy_from_stack(5));


There would be a few concerns here: most notably, that any of these
proxy AVs would likely only support some subset of operations. I think
we'd have to distinguish between "source-only AVs" that get looked at
as inputs but not modified; from "sink-only AVs" that accumulate
results but don't need to be read from beforehand. Additionally, there
would need to be some subtle thoughts about how to encode the required
behaviour with refcount bumping and so on. But I think that would be
easily workable within this overall shape.

As I said at the top, this isn't a fully-formed plan, or a set of
questions. I just wanted to write it out while it was in my head, for
future reference in case someone else came to similar ideas before, or
in future looking back at this.

-- 
Paul "LeoNerd" Evans

[email protected]
http://www.leonerd.org.uk/  |  https://metacpan.org/author/PEVANS