Jeff,

In Jython 2.5 development, up through 2.5.0 RCs/final, we gave ourselves the freedom to break existing Java APIs such as PythonInterpreter. For subsequent 2.5.x releases, we grew the Java API, but kept it backwards compatible. This required a fair amount of contortions.

The general principle should continue to stand in 2.7. So you should feel free to do this refactoring. More below.

On Sat, May 18, 2013 at 2:12 AM, Jeff Allen <ja.py@farowl.co.uk> wrote:
I have prototyped a solution to some difficulties with JLine (Issue
2046), but to make it work in the general case will require changes to
how we go about initialisation that may impact users who have embedded
the interpreter in applications. I'm not sure which aspects of the
present classes constitute API we ought to keep stable.

The root of these problems with JLine is the direct use by sys.stdin of
a System.in, after its behaviour has been modified by JLine to suit
itself. Use of jline.ConsoleReader.readLine through
JLineConsole.raw_input works fairly well. Application writers probably
also expect to use System.in without nasty surprises. By replacing
System.in with a stream that wraps JLine (using
jline.ConsoleReaderInputStream.setIn), I am able to get interactive
behaviour very much like CPython's and a coherent System.in that
benefits from JLine history and editing.

Sounds like a great wrapping. We probably should also look at the interaction with the readline module, which currently uses a workaround to get at JLine directly (through sys._jy_interpreter.reader).

dev>java -cp build\exposed;build\classes;dist\javalib\*
org.python.util.jython
Jython 2.7b1+ (default:bb71f88cbe80+, May 17 2013, 18:20:38)
[Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)] on java1.6.0_35
>>> import sys
>>> sys.stdin.readline()
abc
'abc\n'
>>> raw_input()
Hello
'Hello'

The purpose of JLine is to provide a more convenient way for a user to
get (encoded) characters into a program than typing everything. I'll
call this its "console" function. However, in our current design,
JLineConsole is also a PythonInterpreter, working with lines of
characters treated as commands. It is only by separating the two
concerns, console and interpreter, that I've been able to make it work.
Although one bug fix shouldn't drive architecture, I think this provides
an architectural hint they should be separate.

Makes sense.
 

The second hint this provides is about initialisation. I've hard-wired
use of JLine in my demonstration, but for the console/interpreter to be
selected through configuration, I believe I have to break initialisation
into phases that give the invoking code a chance to act between them.
These phases are probably fill registry, init static state, create
PySystemState instance. I get the feeling this is not the only case
where it would be useful to respond to configuration, or to respond to
static initialisation, before all the plates start spinning.

readline also provides functionality to get at this; I do *not* remember details, but set_startup_hook/set_pre_input_hook look like the entry points.


What do others think? And where are the pitfalls, particularly API
change breaking others' work? I guess we'd like the text-book examples
still to work as written, but what else?

This might also be the right time to consider revisiting other PythonIntepreter/PySystemState related changes. In particular, I would like to get rid of shadowed statics - that is statics that are not really statics but were left that way for backwards compatibility. There's a few that were made that way, but if they are not final, we really should revisit statics given their opportunity to create resource leaks, at the very least.

Thanks for digging into this issue - it's awesome work!

- Jim