Re: [Jython-dev] Jython buffer protocol

[Jeff A. <ja...@fa...>]

Stefan:

A sane version of the nio buffer work now exists for your delight at: 
https://bitbucket.org/tournesol/jython-nio

I've chosen configuration options at Bitbucket to avoid anyone forking 
it and to try to avoid unnecessary merges finding their way into  the 
main Jython repo. However, I wouldn't have committed it even locally if 
it weren't pretty good. If these tricks prevent you getting the code, 
let me know and I'll open it up a bit more.

test_memoryview fails because my new code does not correctly handle 
overlapped copy to self. A fix will follow. The strided overlapped case 
is quite tricky, and I think it must have been broken all along. (I 
wonder if CPython gets it right.)

Jeff

Jeff Allen

On 04/06/2016 17:50, Jeff Allen wrote:
> After extensive refactoring, I'm close now to a satisfactory solution
> based on parallel hierarchies: one for byte[] and one for ByteBuffer,
> each with its own JUnit test. Tests pass on both, including when I
> allocate direct buffers.
>
> I had a shot at simply replacing the prior implementation of the buffers
> with one based on ByteBuffer, wrapping an array where that's the real
> storage. It nearly worked, but involved breaking too many things at once
> to keep track of, and it was just easier to build a new one next door.
>
> I think having two implementations has helped me get the base material
> (BaseBuffer and the test support) into better shape. It was less clear
> what should be in the base material, and what belongs to a particular
> implementation choice, when there was only one. This is the bit I'm
> polishing now. I'm also enjoying how JUnit4 parameterisation lets me
> debug just one implementation type at a time.
>
> I should have something to show this week.
>
> Jeff Allen
>
> On 11/05/2016 19:14, Jeff Allen wrote:
>> On 11/05/2016 14:38, Stefan Richthofer wrote:
>>> Sounds like good news! Would you put a draft e.g. on github once it is
>>> somehow at a sane state?
>> How was it Jim dignified our process ... ah yes, commit-then-review. :)
>> I'll share the elements somehow to check my thinking.
>>>> ByteBuffer getByteBuffer(int... indices);
>>> I wonder what this is supposed to do; afaik ByteBuffer supports no
>>> multi-index logic. (Correct me if I'm wrong).
>> No, but PyBuffer does. The above returns a ByteBuffer where the position
>> has been set corresponding to the index polynomial.
>>>> // Extract column c
>>>> ByteBuffer bb = pybuf.getNIOByteBuffer(PyBUF.FULL);
>>>> for (int r=0; r<x.length; r++)
>>>>         x[r] = bb.getFloat( pybuf.index(r,c) );
>>> This looks slow, because method calls are slow (compared to array-access)
>>> and it requires at least two calls per index.
>> I prefer it to x[r] = pybuf.getByteBuffer(r, c).getFloat() which has
>> object construction too. If you want speed you have to do the index
>> calculation by striding from pybuf.index(0,c), but then you will still
>> call ByteBuffer.getFloat(int) rather than an array access. I think
>> efficiency has ceased to be the main objective. Client code that uses
>> array access (equivalently, Pointer) will break when it encounters your
>> ByteBuffer-based objects, so we should make access via the ByteBuffer
>> convenient.
>>>> I'm following the deprecation route at the moment, but bearing in mind
>>>> Jim's view that breaking change is acceptable by virtue of low adoption,
>>> I wonder if there is any evidence how low adoption currently is at all.
>>> Are there any publicly known projects using this?
>> I abbreviated Jim's argument here. Few and sophisticated enough to adapt
>> was his view. I can't find any public projects using our PyBuffer. To
>> suffer from the change, they would have to be projects that use Pointer,
>> not just PyBuffer.
>>
>> Jeff
>>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev
>