[Jython-dev] State of Java 9 support and proposed deadline extension of one week for 2.7.1 RC

[Jim B. <jim...@py...>]

We have made significant progress recently on the release candidate for
2.7.1, as seen in https://hg.python.org/jython

However, there's one outstanding issue that I would like to address: *full
Java 9 support*. There are two outstanding concerns:

1. Status of our tests in regrtest on Java 9
2. Support for the construct `from X.Y import *`, as implemented by our
"jar" scanning support, as tracked by http://bugs.jython.org/issue2362

Supporting Java 9 is obviously important for Jython. In particular, current
work on Jython-using projects like WebLogic are dependent on Jython 2.7
fully supporting Java 9. It would also be helpful for such projects if we
have good support well in advance of the September 2016 release of Java 9.

Let's now look at these concerns. First, general Java 9 support looks quite
good. Using the latest version of JDK 9 with Project Jigsaw (build b86),
which Rory O'Donnell emailed details about a few days ago this list.
Assuming some exports like so:

export
JAVA_SECURITY_EXPORTS=java.base/sun.security.provider=ALL-UNNAMED,java.base/sun.security.pkcs=ALL-UNNAMED,java.base/sun.security.util=ALL-UNNAMED,java.base/sun.security.x509=ALL-UNNAMED,java.base/sun.security.ssl=ALL-UNNAMED,jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED,jdk.charsets/sun.nio.cs.ext=ALL-UNNAMED
export JAVA_OPTS=-XaddExports:$JAVA_SECURITY_EXPORTS

and then running

$ dist/bin/jython dist/Lib/test/regrtest.py --expected

(because ant regrtest stomps on JAVA_OPTS to ensure no locale issues), then
we get the following test failures on OS X 10.11.1:

7 fails unexpected:
    test_import_jy test_list_jy test_marshal test_os_jy test_re_jy
    test_sort test_unicode_jy

which is pretty good and comparable to Java 8 (test_list_jy test_marshal
test_sort) - the additional failures are due to internationalization
changes in Java (test_os_jy); differences in whitespace definition
(test_re_jy, test_unicode_jy); and lack of support for star imports
(test_import_jy).

(See http://openjdk.java.net/projects/jigsaw/j1/prepare-for-jdk9-j1-2015.pdf
for the source of some of these exports; I also take in account additional
ones for Jython specifically, such as being able to get Big5 support; I
believe this is necessary due to the fact that we have dynamic linkage in
Jython, but we need to dig into this more.)

Back to "jar" scanning, where I quote "jar" because rt.jar and other boot
path jars we have known for Java for all versions prior to 9 are now gone
with project Jigsaw. In its place we get the support in JEP 220 (
http://openjdk.java.net/jeps/220) to get the classes and related resources
from the jrt URL file system provider.

After reviewing the source in org.python.core.packagecache that does this
jar scanning and what is actually written in the cache (see for example,
dist/cachedir), it's pretty clear we can readily implement support against
the new jrt provider and using an API like
java.nio.file.Files::walkFileTree. After that, it's just a simple matter of
programming.

Hopefully that summarizes my reasons for the release candidate deadline
extension I have proposed, from Thur Nov 5 to *Thur Nov 12*.

- Jim