I continue to make progress. You're not seeing anything from me in the code base because I resolved not to push the io work to the repository until my (Java) version scored fewer test_io errors and failures than the existing (Python) version. If it really doesn't matter how broken I've made it while I work, I'll happily put it where others can see.

I'm certainly learning a lot about the Jython plumbing through this. Writing bytearray required only a little understanding by comparison: the clever bits are in the Java. Here Philip Jenvey has mostly done the operations and the challenges stem from the interplay of Java and Python. Even the simplest operations have to take into account the possibility that a subclass defined in Python code has overridden them. This theme accounts for many things in the CPython code I didn't understand at first and is probably behind many still-failing tests.

I have basic open, write, read and close going on in Java for unbuffered binary files.

Operations all involve delegation to Philip's FileIO class, although I found it necessary to implement the logic of read and readall myself because they invoke the possibly overridden "readinto". I think there's more of this Python-overrides-Java stuff to find.

I got this overriding to happen, by exposing __dict__ from (just) the right place. Thanks to others who helped me to the right solution.

The Python buffered and text wrappers still work, even though they get their base class from the Java code. (At least, I think they mostly do: quite a lot of test failure in this area.)

I have adapted the Closer concept from PyFile so that it invokes "close" by name: hence it invokes the overridden close, rather than just closing the underlying (delegate) file. Unlike a PyFile, a PyFileIO remains alive until that can happen.

At the moment I'm working on getting close(), closed(), _checkClosed() and their delegate versions to play together nicely, reproducing the behaviour of CPython. Cold towel time.


On 14/10/2012 00:09, Jeff Allen wrote:
Time I checked into this thread with a status update.

My idea has been to copy _pyio.py to _jyio.py (as a second Python implementation of _io) and then progressively replace parts of it with Java in _io. This half works, but the _jyio classes expect features (like a _checkWritable method) that the exposed classes do not provide (yet).

I've got a better idea now about how methods are invoked, particularly constructors. The hierarchy around io is complex and it's pleasing to see Jython copes with multiple inheritance. I couldn't explain how it works yet, but it's beginning to look familiar in the debugger. I'm not sure what will happen when we get to the multiple inheritance of exposed classes.

I've already encountered a multiple inheritance problem with the exceptions (which do not to work like exposed classes, I know). io.BlockingError and io.UnsupportedOperation both inherit IOError and another exception type. Can anyone explain how to do this in Java?

And when I get there, what will the Java for
    class BufferedRandom(BufferedWriter, BufferedReader):
look like?

The main progress is that I now have a pretty faithful imitation in Java of _io.open() (from CPython _iomodule.c). It gets called from Python where it should and can construct an (only slightly defective) exposed _io.FileIO that delegates to Philip's org.core.io.PyFileIO. My Java _io.open() method is able to construct instances of the classes in _jyio and wrap the raw FileIO in them. It looks like this:

        // Get constructor io.BufferedWriter
        PyObject globals = __builtin__.globals();
        PyObject io = globals.__finditem__("io");
        PyObject bufferType = io.__getattr__("BufferedWriter");
        PyInteger pyBuffering = new PyInteger(buffering);
        PyIOBase buffer = (PyIOBase)bufferType.__call__(raw, pyBuffering);

This is exciting as I'd have had no idea how to do that a couple of weeks ago. But I think it would fail if module io were not already loaded. This will hardly ever be the case, but what could I do in my code to make sure it gets loaded?


On 03/10/2012 21:00, Jeff Allen wrote:
Some of this is beginning to make sense. I've discovered:

Now that __new__ is being called, I can see better how this is supposed to work. But I still don't really have the picture in my head of the data structure being built and navigated.

Can anyone say where (in Java) MockRawIOWithoutRead.__init__() should get called from, and when? Is that done in the generated code defined in Python or in support code written in Java?