MethodHandle as type slots: result

I'm well along with some experiments to see if we can use MethodHandle 
as a "pointer to function" in the slots of a type object. The idea works 
well enough that we could use it confidently in Jython 3.

There was never much doubt that a MethodHandle could do the job of 
pointer to function. There was some question in my mind whether it could 
be used "neatly", with low overhead and the readably to avoid mistakes. 
In my initial dabbling with MethodHandle there always seemed to be a lot 
of boiler-plate, and I'll admit I was never very clear a given handle 
would match its arguments, or whether I was inducing a whole load of 
checked casts behind the scenes.

I shared this work off list, when the idea was less mature, but it was 
never hidden, so if you've been following along, there's nothing much 
new, except that in my mind, this part has reached a convincing state. 
Adding new opcodes goes smoothly so I think the idea is in the bag: not 
the details, I have more changes in mind, but in principle. I've managed 
to encapsulate in supporting classes all the handle creation and 
reflection that makes the MethodHandle examples you encounter so ugly.

I'm using this in a (woefully incomplete) interpreter for Python 3.8 
byte code. It does arithmetic, indexing and calls. I can report that in 
trivial tests, I'm getting something better than half the speed of 
CPython. TBH I was disappointed not to equal or marginally exceed. I can 
think of improvements, but I'm resisting the temptation.

https://the-very-slow-jython-project.readthedocs.io/en/latest/generated-code/type-and-arithmetic.html

Archtecturally, I would sum up the idea as dropping Jython 2's use of 
the JVM virtual function table in PyObject to represent the slots, and 
adopting a design more more like CPython's, with explicit slots in 
PyType instances that one can re-write. I expected to do away with some 
confusing differences between Jython and CPython. A clean start has 
helped, but looking ahead I can see some of the things returning, e.g. 
the way reflected functions have their own slots, and the *Derived 
classes. The revenants don't have the same form, but the reasons for 
them (which I now appreciate better) seem permanent in the landscape.

Long-term, we would have a compiler to JVM byte code, of course. It 
could simply generate calls to the methods for the abstract operations 
(avoiding the loop-fetch-switch cost) or it could use proper indy calls, 
with call sites embedding the same method handles that appear in a type. 
But we need a Python byte code interpreter in Jython, and with 
performance in the CPython ball-park, it could be enough to get a Jython 
3 out.

-- 
Jeff Allen