Re: Jython buffer protocol

Hey Jeff,
thanks a lot for this work. I will take a closer look as soon as I find time.
However, so far - quickly scrolling through some source-files - it looks pretty good.

Best

Stefan



> Gesendet: Samstag, 11. Juni 2016 um 11:37 Uhr
> Von: "Jeff Allen" <[email protected]>
> An: "Stefan Richthofer" <[email protected]>
> Cc: "Jython Developers" <[email protected]>
> Betreff: Re: [Jython-dev] Jython buffer protocol
>
> Stefan:
> 
> A sane version of the nio buffer work now exists for your delight at: 
> https://bitbucket.org/tournesol/jython-nio
> 
> I've chosen configuration options at Bitbucket to avoid anyone forking 
> it and to try to avoid unnecessary merges finding their way into  the 
> main Jython repo. However, I wouldn't have committed it even locally if 
> it weren't pretty good. If these tricks prevent you getting the code, 
> let me know and I'll open it up a bit more.
> 
> test_memoryview fails because my new code does not correctly handle 
> overlapped copy to self. A fix will follow. The strided overlapped case 
> is quite tricky, and I think it must have been broken all along. (I 
> wonder if CPython gets it right.)
> 
> Jeff
> 
> Jeff Allen
> 
> On 04/06/2016 17:50, Jeff Allen wrote:
> > After extensive refactoring, I'm close now to a satisfactory solution
> > based on parallel hierarchies: one for byte[] and one for ByteBuffer,
> > each with its own JUnit test. Tests pass on both, including when I
> > allocate direct buffers.
> >
> > I had a shot at simply replacing the prior implementation of the buffers
> > with one based on ByteBuffer, wrapping an array where that's the real
> > storage. It nearly worked, but involved breaking too many things at once
> > to keep track of, and it was just easier to build a new one next door.
> >
> > I think having two implementations has helped me get the base material
> > (BaseBuffer and the test support) into better shape. It was less clear
> > what should be in the base material, and what belongs to a particular
> > implementation choice, when there was only one. This is the bit I'm
> > polishing now. I'm also enjoying how JUnit4 parameterisation lets me
> > debug just one implementation type at a time.
> >
> > I should have something to show this week.
> >
> > Jeff Allen
> >
> > On 11/05/2016 19:14, Jeff Allen wrote:
> >> On 11/05/2016 14:38, Stefan Richthofer wrote:
> >>> Sounds like good news! Would you put a draft e.g. on github once it is
> >>> somehow at a sane state?
> >> How was it Jim dignified our process ... ah yes, commit-then-review. :)
> >> I'll share the elements somehow to check my thinking.
> >>>> ByteBuffer getByteBuffer(int... indices);
> >>> I wonder what this is supposed to do; afaik ByteBuffer supports no
> >>> multi-index logic. (Correct me if I'm wrong).
> >> No, but PyBuffer does. The above returns a ByteBuffer where the position
> >> has been set corresponding to the index polynomial.
> >>>> // Extract column c
> >>>> ByteBuffer bb = pybuf.getNIOByteBuffer(PyBUF.FULL);
> >>>> for (int r=0; r<x.length; r++)
> >>>>         x[r] = bb.getFloat( pybuf.index(r,c) );
> >>> This looks slow, because method calls are slow (compared to array-access)
> >>> and it requires at least two calls per index.
> >> I prefer it to x[r] = pybuf.getByteBuffer(r, c).getFloat() which has
> >> object construction too. If you want speed you have to do the index
> >> calculation by striding from pybuf.index(0,c), but then you will still
> >> call ByteBuffer.getFloat(int) rather than an array access. I think
> >> efficiency has ceased to be the main objective. Client code that uses
> >> array access (equivalently, Pointer) will break when it encounters your
> >> ByteBuffer-based objects, so we should make access via the ByteBuffer
> >> convenient.
> >>>> I'm following the deprecation route at the moment, but bearing in mind
> >>>> Jim's view that breaking change is acceptable by virtue of low adoption,
> >>> I wonder if there is any evidence how low adoption currently is at all.
> >>> Are there any publicly known projects using this?
> >> I abbreviated Jim's argument here. Few and sophisticated enough to adapt
> >> was his view. I can't find any public projects using our PyBuffer. To
> >> suffer from the change, they would have to be projects that use Pointer,
> >> not just PyBuffer.
> >>
> >> Jeff
> >>
> >
> > ------------------------------------------------------------------------------
> > What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> > patterns at an interface-level. Reveals which users, apps, and protocols are
> > consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> > J-Flow, sFlow and other flows. Make informed decisions using capacity
> > planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> > _______________________________________________
> > Jython-dev mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/jython-dev
> >
> 
> 

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e