Jython buffer protocol

Dear Jeff,

- Your suggestion for #1 with the new PyBUF-flag seems fine for me. My only worry would be that in most situations PyBuffers would support PyBuffer.Pointer even if requested without the flag. This itself is okay, but since programmers are not familiar with the new flag, they might get used to get away with not using it. Maybe with a clear warning in the doc this would go fine.

- I was not aware that package-private stuff prevents one from extending ByteBuffer. Stack-Overflow says this allows the JVM to do significant optimization, but indeed ByteBuffers appear less useful to me than before.

- Your suggestion for #1 made me recall that also CPython's BufferProtocol takes into account that an exporter might raise an error if it can't deal with the requested PyBUF-flags. I can use this behaviour in #2 to deal with the case that JNI can't provide direct access to an array's elements. However the downside of this "solution" would be inconsistent behavior between Jython's and JyNI's BufferProtocols - PyBUF flags that work on Jython might fail on native side.

- I agree on your concerns for #2 regarding efficiency (although I can imagine, that ByteBuffers would still perform rather well as far as one sticks to bulk get/put-operations; ByteBuffers are said to be well optimized by the JVM). However, I had an implementation in mind that distinguishes direct ByteBuffer vs byte[]-backed buffer case early, i.e. at Jython startup (via the proposed config parameter). Jython could choose the appropriate implementations of BufferProtocol-supporting builtin types and assign them to the builtins in the builtin-type-dictionary. I admit this is a bunch of work, but it is not so urgent and I would offer to provide the alternative implementations or at least help with it. However I would postpone this until an actual use-case comes up (i.e. an extension ap
pears that could not be made working cleanly without it). With your suggestion for #1/new PyBUF-flag, Jython's BufferProtocol would be flexible enough to insert such enhanchments later.

- One more thing. I recently recognized that the method "irepeat" in PyByteArray (used by bytearray___imul__) replaces the storage backend by a new array of changed size. But it does not call resizeCheck(). Is this a bug, or do I miss something? (I admit I am currently not looking at the newest beta-code, so please ignore this if it was already fixed.)

-Stefan

Gesendet: Donnerstag, 17. Juli 2014 um 22:18 Uhr

Von: "Jeff Allen" <[email protected]>

An: "Stefan Richthofer" <[email protected]>

Cc: "Jython Developers" <[email protected]>, "Jim Baker" <[email protected]>

Betreff: Re: Jython buffer protocol

Dear Stefan:

I did indeed mean move it to jython-dev, not the other place. Thanks for the careful exposition of your ideas.

In pursuit of idea #1, I think it would not be too difficult to present a ByteBuffer as a PyBuffer, even when there is no byte[] behind it. There are obvious implementations for most of the "abstract" API, and I imagine getBufferSlice could be done with ByteBuffer.slice().

We could add a getNIOByteBuffer to PyBuffer with a default implementation to wrap a Pointer. That is the most significant use of Pointer in the core already. I think I even added it once and reverted it as gold-plating. I see a reason for it now: an object not presenting byte[] access, could implement that method its own way.

I am not keen to lose the direct access to the underlying byte[] that Pointer gives, in general. It seems to me that this is the essence of the CPython Py_Buffer: it gives you a char* and some dimensions, and it lets you have at the data directly, in whatever pattern of access you need, and quickly. A PyBuffer.Pointer is the Java equivalent of the char *buf member. However, I think it would be acceptable for certain objects not to implement the operations that return a Pointer, or (like PyString) to implement them expensively, but defer the cost. I appreciate that implementation may be practical only for a read-only object.

We could add a PyBUF-flag, whereby a client says whether it expects to make Pointer access, so getBuffer(int) could fail early and helpfully, rather than the client fail later.

Idea #2 sounds very difficult to pull off at all (at the JyNI end). And to make it possible you imply that all core objects that implement BufferProtocol would be rewritten to store their data in a ByteBuffer. BaseBytes has a lot in common with ByteBuffer, it is true, but the difference is that inside BaseBytes I can get at the bytes directly, and the clients can do so through a PyBuffer.Pointer. By comparison, using ByteBuffer feels like working through a keyhole. It cannot even be extended, since its constructor is package-private. It is possible to do this, I would say, but to forbid Jython efficient access to data as byte[], in order to afford C-code direct access to it as char*, seems perverse.

Jeff Allen

On 17/07/2014 13:40, Stefan Richthofer wrote:

Dear Jeff,

the true goal I want to achieve is a clean BufferProtocol support in JyNI. That means two things in my opinion.

1) If a CPython extension features types that support BufferProtocol and these are passed to Jython via JyNI, they shall appear as Jython PyObjects that support Jython's BufferProtocol. The exact memory-data that the CPython object exposes shall be exposed in Java by the Jython object. Optimally this should work for reading and writing and as direct as possible (for efficiency, minimizing memory requirements and -most important- to guarantee sync between Java-view and native view on the data). Using direct ByteBuffers this could be achieved, at least for JVMs that support these. On other JVMs, I would use byte[] as a fallback and try to obtain native access on the memory. However in this case, JNI does not guarantee to provide access to the actual memory of the array. It might onl
y offer a copy, which would hold the risk of loosing sync. I see some techniques how to avoid that somehow, but it won't be very efficient nor elegant, nor absolutely save.

2) If a Jython object implements Jython's BufferProtocol in Java and JyNI is used to pass a PyObject from Jython down to a CPython extension, the native variant of the object shall support CPython's BufferProtocol. The object shall expose the corresponding memory from the JVM to the extension via this protocol. This shall work for reading and writing and as direct as possible for same reasons as above. I admit, this will be hard for Jython objects where a user implements PyBuffer in his own fashion. However, I think at least Jython's built-in types could and should support this. It can also be done with direct ByteBuffers.

I know that direct ByteBuffers also have their disadvantages. They prohibit the JVM from doing memory optimization. Additionally the GC does not consider their memory when it determines what to delete or when to run (it would still clear the memory, if it collects a buffer). So direct ByteBuffers should only be used when really needed. I would propose a configuration-parameter for Jython that tells BaseBytes and BaseBuffer to use direct ByteBuffers. If this parameter is turned off, they could use ByteBuffer.wrap to fall back to the current implementation.

The doc of Jython's BufferProtocol could tell potential other implementers also to look at this parameter and store their data as direct or ordinary ByteBuffer accordingly. However, I would build fallbacks into JyNI to deal as good as possible with situations where implementers don't stick to this.

Both scenarios would mean that the backend in BaseBytes and BaseBuffer had to be of a type that unifies byte[] and ByteBuffer. One variant would be to use ByteBuffer as type and ByteBuffer.wrap for byte[] case. Another variant would be to have the storage of type Object. The PyBuffer implementation would know whether it is byte[] or ByteBuffer, or maybe even String and work accordingly (this would involve lots of explicit type casts though). However PyBuffer.pointer should not provide a byte[] and pretend it to be the actual backend in either variant.

I think the solution from PyString, i.e. SimpleStringBuffer only works well for read-only scenarios, since it would lead to asynchronity between byte[] and String backends if the user uses the obtained byte[] for writing (assuming a mutable String variant like StringBuffer). Additionally, such an approach potentially doubles the memory requirements, which might be significant in some situations.

I mentioned my thoughts about support for (>2^31 bytes)-arrays, because -afaIk- a CPython extension might expose such long data via BufferProtocol and I was wondering how to deal with this. Then I mentioned it as a maybe misleading example. I could imagine to provide kind of LongPyBuffer that enhanches PyBuffer by long-index methods and allows to obtain ordinary int-sized slices to portions of the data. But these are future thoughts - for a first step I would just document it as a JyNI-limitation to support BufferProtocol only for (>2^31 bytes)-buffers.

Again - I would be fine with postponing this decision if the public API was adjusted to be open for more variants than byte[]. I would also offer to help with implementing the adjustments and -if accepted- the refining of Jython's BufferProtocol.

I hope this explains my intentions a bit better.

Cheers

Stefan

P.S. Feel free to move the discussion to jython-dev (assuming 'python-dev' was a typo^^)

Gesendet: Donnerstag, 17. Juli 2014 um 10:26 Uhr

Von: "Jeff Allen" <[email protected]>

An: "Stefan Richthofer" <[email protected]>

Cc: "Jim Baker" <[email protected]>

Betreff: Re: Jython buffer protocol

Stefan:

I see why ByteBuffer is useful for referencing bytes that might not be in an a byte[]. I don't yet see why you want to do this through the PyBuffer interface. I imagine you want to define a PyObject that offers BufferProtocol to represent the data. I think we could make it work, but I don't know if what I'm imagining meets your need. Could you give some toy examples?

If the killer application is to represent very large arrays (>2^31 bytes) then your are blocked, since nothing else in the API will work with long indices. So only normal-sized objects, or normal-sized slices of large data objects can be handled.

PyString provides an example of bytes that are not stored in a byte[]. This is perhaps the model we should use. It *does* give you a Pointer to a real byte[], if you insist, but it tries hard to avoid creating one. Most clients accessing its PyBuffer, don't provoke this action. I think it would be tolerable for a private-use object to throw instead of creating the massive array.

BTW, I'm ok with this appearing on python-dev. This is an e-mail address I'm content to expose there. Jim was the one to answer you previously, because Jim was the one who knew the answers.

Jeff Allen

On 16/07/2014 13:40, Stefan Richthofer wrote:

Dear Jeff,

I agree on most of your concerns. Safety is not the running argument (since safty in Java is mainly an illusion anyway). I mainly mentioned it to enhanch the pro's section a bit. Enforcing some buffer properties would however benefit debugging purposes I think.

Also agree that Java tends to overdo checking constraints etc and that ByteBuffer has its issues like many things in Java.

And I absolutely agree that ByteBuffer is not sufficient to replace PyBuffer, because of the striding features etc.

It should be used as a backend and maybe as a replacement for PyBuffer.Pointer, which is up to you.

The killer-feature is that ByteBuffer fundamentally offers additional functionality over byte[] - it can potentially point to memory outside the JVM. This makes it the truest notion of a "pointer" that Java can offer. Just to give some non-JyNI example what could be done with it:

I'm not 100% sure, but I believe one could even write a ByteBuffer subclass that offers long-index access methods overcoming the (2^31)−1 limitation of array size (in fact even (2^31)−6). (One would have to manage allocation and memory natively). So ByteBuffers would be the best chance to allow working with big data and stuff. In general, direct ByteBuffers would be the way to provide a buffer protocol that even extends to C-level.

Let me put it that way - you don't have to decide this now. Neither in a year. I just ask you to change the public API such that it is not tied to the plain array variant. Hide the storage field in PyBuffer.Pointer. Offer a getStorage()-method instead and state that it is not guaranteed to provide the actual backing array. Add a boolean method that tells whether it does. If you like, you can even guarantee access to the backing array in default configuration of Jython (i.e. provide a flag in the future that leverages advanced buffer functionality). These are minor changes, but they must be done before the beta-phase ends.

Thanks for considering my proposal!

Cheers

Stefan

Gesendet: Mittwoch, 16. Juli 2014 um 10:10 Uhr

Von: "Jeff Allen" <[email protected]>

An: "Jim Baker" <[email protected]>, "Stefan Richthofer" <[email protected]>

Betreff: Re: Jython buffer protocol

Hi Jim, and Stefan - thanks for your careful reading of my work.

I have considered java.nio.ByteBuffer a lot during this development, both as a possible substitute for the API as a whole (but it isn't close enough to CPython's API) and as a candidate for Pointer. It also gives me clues about how the API should extend to elements other than byte, which I'd like to do, and I imagine we need for NumPy. (The precursors of this exist in the API, but may be incorrect.)

So java.nio.Buffer is good, and I'm trying to remember why it didn't make it as Pointer.

I've definitely thought more than once I would like easily to get a ByteBuffer from a PyBuffer, when looking into io and codecs. Adding that to the API seemed to burden the implementer unnecessarily when the client can so easily call ByteBuffer.wrap() on information the Pointer gives. But the idea here seems to be that an object that is a java.nio.ByteBuffer already, should be able to back a PyBuffer. I'm not sure this works.

The CPython API that I'm copying absolutely gives the client a pointer to bytes directly, with the purpose of efficient access, and all the attendant risks accepted. So I'm not convinced by the "safety" argument.

I recall looking at the ptr:limit range-checking and readonly checks in ByteBuffer and thinking I those were at contrary to the intentions of the API. (It's bad enough that the array bounds are checked!) Efficiency is largely a matter of taste, however, and one's faith in the optimiser.

Many operations on ByteBuffer move the pointers and limit around in ways that may surprise. I remember feeling that it was too rich and overly-encapsulated when what I wanted really was just a holder for two/three quantities: array base, offset and length.

I'm trying to remember if there was a real show-stopper. The hard case, I predict is, given a PyBuffer backed by an (opaque) ByteBuffer, can I create a PyBuffer slice that returns a ByteBuffer having the properties a client would expect.

I'll think about it.

Jeff Allen

On 16/07/2014 07:12, Jim Baker wrote:

Adding Jeff Allen to the discussion, since he's the author of this support, and therefore has substantially more insight about PyBuffer and its subtleties than I do.

In general, using ByteBuffer as our basis, as wrapped by PyBuffer, makes sense, given the pros and cons below. Something comparable is seen in http://netty.io/4.0/api/io/netty/buffer/ByteBuf.html , although Netty's ByteBuf does wrap byte[] directly as well. In particular, Python's buffer protocol is heavily influenced by the needs of NumPy support, so our own efficient support of NumPy via JyNI is an important consideration. We also know that at JNI has strict requirements for safety that ByteBuffer helps ameliorate. (Although I would be also curious: to what extent can we sidestep via sun.misc.Unsafe, as we already are doing in JNR?)

My naive reading of PyBuffer.Pointer is that this potentially could make certain indexed ops more expensive, but it's also not clear how much we are using such ops today.

I have probably muddied the discussion at this point, but hopefully we can start a good discussion on the best approach.

- Jim

On Sat, Jul 12, 2014 at 9:11 AM, Stefan Richthofer <[email protected]> wrote:

Hey Jim,

I recently had a look into Jython's implementation of the Buffer protocol (just needed a change from gc stuff and work). It uses a plain byte[] array as storage backend. Thinking of it as a default backend for the PyBuffer interface might be okay, since it could be changed on demand without breaking external code. But there is the result type PyBuffer.Pointer, which contains a public reference to the backing byte[] of the PyBuffer. This makes it obligatory to use byte[] as PyBuffer backends. (BaseBytes and BaseBuffer also expose their byte[]-backends, but at least they use the "protected" access modifier.)

However I see good reasons to use a ByteBuffer from java.nio as storage backend instead of a plain byte[] array and highly recommend to change at least the field in PyBuffer.Pointer to the type java.nio.ByteBuffer. This has no regressions, since one can still have byte[] as backend and use java.nio.ByteBuffer.wrap to create a ByteBuffer on top of the array.

Pros:

- Using ByteBuffer allows for a wider range of possible backends without significantly restricting functionality or efficiency.

- The Buffers from java.nio are Java's equivalent of Python's buffer protocol. They were made for this. So Jython's buffer protocol should be build on top of them. They are optimized for sharing memory, even with native code.

- The ByteBuffer can be constructed such that it enforces some of the properties defined by flags from PyBUF. For instance if the flags indicate a read-only buffer, an according read-only ByteBuffer can be exposed to the user.

- Last but not least, the use of direct ByteBuffers would allow me to emulate CPython's buffer protocol in JyNI. Unfortunately it is not obligatory for JVMs to support direct buffers, but for those that do I believe, I could produce almost 100% CPython behaviour with JyNI. On other JVMs it would always remain an issue to detect changes in the buffer and sync them back to Java, leading to low efficiency and lack of success guarantees (I will still have to deal with this as a fallback).

You might guess, that this is honestly the reason, why I propose this. However, I think the other advantages of using ByteBuffer still hold and there are hardly disadvantages on the other hand.

Con's:

- The only thing that would be lost is that writers of external java code interfacing with Jython had the guarantee that they can have write-access to the buffer via a byte array. But in my opinion they should not have this guarantee anyway. Think of a slice of a PyBuffer. By letting the users have the byte[] backend they would have access to the sliced-out data. Or maybe also to read-only data.

- It would break existing code that interfaces with Jython via the buffer protocol. However it would be trivial to fix such code. The protocol was introduced with Jython 2.7 which is still beta and -hopefully- there is currently no or hardly code out there that uses this feature. So this kind of change should be done urgently. To ease fixing external code one could add a method to PyBuffer.Pointer that returns the buffer values as byte[] (With the doc stating that it might be a copy, if the underlying ByteBuffer has no backing array).

So let me comprehend my proposal:

Step 1 (urgent):

In PyBuffer.Pointer change the type of the "storage"-field from byte[] to java.nio.ByteBuffer.

Fix the implementation of the constructor of PyBuffer.Pointer by using java.nio.ByteBuffer.wrap.

Step 2 (not so urgent):

Change the default backends in BaseBytes and BaseBuffer to be java.nio.ByteBuffer instead of byte[].

Fix current constructors the same way as done with that of PyBuffer.Pointer.

Add new constructors that directly take ByteBuffer (also to PyBuffer.Pointer).

Step 3:

Provide a hook or startup-parameter/flag that tells Jython to use direct ByteBuffers as default backends for PyByteArray and BaseBuffer. These can prohibit some memory optimization of the JVM, so providing it as a flag allows me or others to turn it on in JyNI-case (or in case someone else wants to use the buffer protocol via JNI in native code), i.e. only when it is really needed.

Cheers

Stefan

------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds

_______________________________________________
Jython-dev mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/jython-dev