I continue to make progress. You're not seeing anything from me in
the code base because I resolved not to push the io work to the
repository until my (Java) version scored fewer test_io errors and
failures than the existing (Python) version. If it really doesn't
matter how broken I've made it while I work, I'll happily put it
where others can see.
I'm certainly learning a lot about the Jython plumbing through this.
Writing bytearray required only a little understanding by
comparison: the clever bits are in the Java. Here Philip Jenvey has
mostly done the operations and the challenges stem from the
interplay of Java and Python. Even the simplest operations have to
take into account the possibility that a subclass defined in Python
code has overridden them. This theme accounts for many things in the
CPython code I didn't understand at first and is probably behind
many still-failing tests.
I have basic open, write, read and close going on in Java for
unbuffered binary files.
Operations all involve delegation to Philip's FileIO class, although
I found it necessary to implement the logic of read and readall
myself because they invoke the possibly overridden "readinto". I
think there's more of this Python-overrides-Java stuff to find.
I got this overriding to happen, by exposing __dict__ from (just)
the right place. Thanks to others who helped me to the right
The Python buffered and text wrappers still work, even though they
get their base class from the Java code. (At least, I think they
mostly do: quite a lot of test failure in this area.)
I have adapted the Closer concept from PyFile so that it invokes
"close" by name: hence it invokes the overridden close, rather than
just closing the underlying (delegate) file. Unlike a PyFile, a
PyFileIO remains alive until that can happen.
At the moment I'm working on getting close(), closed(),
_checkClosed() and their delegate versions to play together nicely,
reproducing the behaviour of CPython. Cold towel time.
On 14/10/2012 00:09, Jeff Allen wrote:
Time I checked into this thread with a status update.
My idea has been to copy _pyio.py to _jyio.py (as a second Python
implementation of _io) and then progressively replace parts of it
with Java in _io. This half works, but the _jyio classes expect
features (like a _checkWritable method) that the exposed classes
do not provide (yet).
I've got a better idea now about how methods are invoked,
particularly constructors. The hierarchy around io is complex and
it's pleasing to see Jython copes with multiple inheritance. I
couldn't explain how it works yet, but it's beginning to look
familiar in the debugger. I'm not sure what will happen when we
get to the multiple inheritance of exposed classes.
I've already encountered a multiple inheritance problem with the
exceptions (which do not to work like exposed classes, I know).
io.BlockingError and io.UnsupportedOperation both inherit IOError
and another exception type. Can anyone explain how to do this in
And when I get there, what will the Java for
class BufferedRandom(BufferedWriter, BufferedReader):
The main progress is that I now have a pretty faithful imitation
in Java of _io.open() (from CPython _iomodule.c). It gets called
from Python where it should and can construct an (only slightly
defective) exposed _io.FileIO that delegates to Philip's
org.core.io.PyFileIO. My Java _io.open() method is able to
construct instances of the classes in _jyio and wrap the raw
FileIO in them. It looks like this:
// Get constructor io.BufferedWriter
PyObject globals = __builtin__.globals();
PyObject io = globals.__finditem__("io");
PyObject bufferType = io.__getattr__("BufferedWriter");
PyInteger pyBuffering = new PyInteger(buffering);
PyIOBase buffer = (PyIOBase)bufferType.__call__(raw,
This is exciting as I'd have had no idea how to do that a couple
of weeks ago. But I think it would fail if module io were not
already loaded. This will hardly ever be the case, but what could
I do in my code to make sure it gets loaded?
On 03/10/2012 21:00, Jeff Allen wrote:
Some of this is beginning to make sense. I've discovered:
- The "plausible stuff happening ... difficult to know
whether it is meant to happen", is not supposed to happen. I
made a typo in CoreExposed.includes, and that's why
constructors in my PyRawIOBase were being searched
- If PyRawIOBase is properly exposed, __new__ is called and
as we return, __init__ would be called in some
Now that __new__ is being called, I can see better how this is
supposed to work. But I still don't really have the picture in
my head of the data structure being built and navigated.
Can anyone say where (in Java) MockRawIOWithoutRead.__init__()
should get called from, and when? Is that done in the generated
code defined in Python or in support code written in Java?