Re: [Jython-dev] Buffer protocol - direct vs. JVM ByteBuffers

[Jeff A. <ja...@fa...>]

Thanks for that. So, the (possible) copy-back semantics of 
Get<PrimitiveType>ArrayElements|| effectively make a direct buffer copy 
of the contents. If the C-code runs in the thread of the Java execution 
that calls it (or equivalently it is suspended) then to first order the 
copy causes no problem. But it is not difficult to cook up a scenario 
where another thread or call-back into Java sees a different state from C.

Alternatively, one uses the "critical" methods, and suffers restrictions 
that are, I expect, unenforcible on arbitrary CPython extension modules, 
such as being short and not yielding the CPU.

I'm reminded of the relationship in CPython between C-code and 
interpreted code, where the GIL must be held, proving all other threads 
are "restubg" between instructions, and a context switch is only allowed 
when surrounded by the appropriate magical incantations. I think the 
Universe is trying to tell us something.

The problem I see with the DIRECT_NIO flag is that one cannot expect to 
choose, at the point of getting a PyBuffer, whether that buffer should 
be direct or heap. The data that hold the state of an object have a 
certain implementation in Java, and so the buffer will be a heap buffer. 
Or one can imagine a PyObject whose state is always in a direct 
ByteBuffer (representing an image mapped from disk, say) and then the 
PyBuffer would always be direct. Just possibly objects whose main 
purpose is to be native-friendly would have that implementation. Just 
possibly, this is a thing you get to choose when the object is constructed.

The only way I can imagine an object with Java fields as storage giving 
you a direct ByteBuffer on demand is to allocate one and copy its state 
there. This seems to bring us full circle to the behaviour of 
Get<PrimitiveType>ArrayElements|| , except that I suppose the object 
knows it has done it and can handle intervening access via the Java API 
... effectively a change of implementation on the fly.

Jeff Allen

On 15/05/2016 16:39, Stefan Richthofer wrote:
> Just an add-on to my recent post:
> >It may depend on storage.hasArray(), but storage.isDirect() seems to 
> make no difference.
> I think it is likely the JVM does not offer a backing array, if the 
> buffer is created as direct (i.e. these flags likely exclude each 
> other), because this would imply array pinning and all the 
> restrictions coming with it. I didn't test it though, but anyway we 
> cannot rely on the one or other behavior, as doc explicitly does not 
> guarantee a backing array for direct buffers, saying this is 
> "implementation specific".
> *Gesendet:* Sonntag, 15. Mai 2016 um 11:57 Uhr
> *Von:* "Jeff Allen" <ja...@fa...>
> *An:* "Stefan Richthofer" <Ste...@gm...>, "Jython 
> Developers" <jyt...@li...>
> *Betreff:* [Jython-dev] Buffer protocol - direct vs. JVM ByteBuffers
>
> Stefan:
>
> https://github.com/jythontools/jython/pull/39
>
> What difference does it make in your use case whether a NIO ByteBuffer 
> is direct or non-direct? I can see why a client might want to know 
> which it had been given, but not why it might want an exception raised 
> in one or other case.
>
> Nothing I'm doing seems to depend on what kind of memory the exporting 
> object has, therefore on the implementation type of ByteBuffer 
> storage. It may depend on storage.hasArray(), but storage.isDirect() 
> seems to make no difference.
>
> Jeff
> -- 
> Jeff Allen
> ------------------------------------------------------------------------------ 
> Mobile security can be enabling, not merely restricting. Employees who 
> bring their own devices (BYOD) to work are irked by the imposition of 
> MDM restrictions. Mobile Device Manager Plus allows you to control 
> only the apps on BYO-devices by containerizing them, leaving personal 
> data untouched! 
> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j_______________________________________________ 
> Jython-dev mailing list Jyt...@li... 
> https://lists.sourceforge.net/lists/listinfo/jython-dev