Menu
▾
▴
Re: [Jython-dev] Updating extlibs
|
From: Stefan R. <Ste...@gm...> - 2017-05-26 23:19:15
|
<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div> <div>> You can apply these updates to extlibs</div> <div>Done as of <a href="https://hg.python.org/jython/rev/f6b3ddbc1df8" target="_blank">https://hg.python.org/jython/rev/f6b3ddbc1df8</a>.</div> <div> </div> <div>From my point of view we can launch Jython 2.7.1-RC2 now.</div> <div> </div> <div>- no open PRs on github left</div> <div>- no release blockers left apparently <strike>except #2487</strike></div> <div>- all updates done as far as possible (python std-lib will be next challenge on this front)</div> <div> </div> <div>So, I'd suggest everybody pushes last-minute work and Frank hits the release-button... :-)</div> <div>What do you think?</div> <div> </div> <div> </div> <div>Cheers!</div> <div> </div> <div>-Stefan</div> <div> </div> <div> <div style="margin: 10.0px 5.0px 5.0px 10.0px;padding: 10.0px 0 10.0px 10.0px;border-left: 2.0px solid rgb(195,217,229);"> <div style="margin: 0 0 10.0px 0;"><b>Gesendet:</b> Freitag, 26. Mai 2017 um 20:16 Uhr<br/> <b>Von:</b> "Jim Baker" <jim...@py...><br/> <b>An:</b> "Stefan Richthofer" <Ste...@gm...><br/> <b>Cc:</b> "Jython Developers" <jyt...@li...><br/> <b>Betreff:</b> Re: [Jython-dev] Updating extlibs</div> <div> <div>Stefan, <div> </div> <div>You can apply these updates to extlibs, based on my testing on OSX. More below.</div> <div> </div> <div> </div> <div>I was able to run the following on both Java 7 and Java 8 on OSX:</div> <div> </div> <div>pip/setuptools installation smoke test fully succeeds, using the manual test described in <a href="http://bugs.jython.org/issue2570" target="_blank">http://bugs.jython.org/issue2570</a> (we should look at automating, at least with a simple bash script)</div> <div> </div> <div>regrtest. I did observe flakiness in test_socket_jy, which may require more rounds of waiting, but it does not appear to happen unless it's run in the context of other tests, so full runs of the regrtest. (And IIRC, appeared in the past before extlib update. It has been a well-known flaky test.) For Java 7, test_codecencodings_tw continues to not completely pass, almost certainly due to changes in underlying Java support (a bug fix that was not applied to Java 7). It has nothing to do with these extlib updates.</div> <div> </div> <div>Note that test_socket is shaking out some problems, likely due to the Netty update further refining the exceptions it throws. It's worth noting that error states are not documented very well for either C sockets or Netty's rather different support, so we have to explore the implemented behavior. So one example I saw that should be fixed at some point (possibly not for 2.7.1, there will always be more bugs, and certainly higher priority ones IMHO at this time):</div> <div> </div> <div> <div> [exec] Unhandled exception in thread started by <bound method BasicTCPTest.clientRun of testShutdown (test.test_socket.BasicTCPTest)></div> <div> [exec] Traceback (most recent call last):</div> <div> [exec] File "/Users/jbaker/jythondev/jython-extlibs/dist/Lib/test/test_socket.py", line 186, in clientRun</div> <div> [exec] self.clientSetUp()</div> <div> [exec] File "/Users/jbaker/jythondev/jython-extlibs/dist/Lib/test/test_socket.py", line 246, in clientSetUp</div> <div> [exec] self.cli.connect((self.HOST, self.PORT))</div> <div> [exec] File "/Users/jbaker/jythondev/jython-extlibs/dist/Lib/_socket.py", line 1441, in meth</div> <div> [exec] return getattr(self._sock,name)(*args)</div> <div> [exec] File "/Users/jbaker/jythondev/jython-extlibs/dist/Lib/_socket.py", line 935, in connect</div> <div> [exec] self._connect(addr)</div> <div> [exec] File "/Users/jbaker/jythondev/jython-extlibs/dist/Lib/_socket.py", line 911, in _connect</div> <div> [exec] self._handle_channel_future(self.connect_future, "connect")</div> <div> [exec] File "/Users/jbaker/jythondev/jython-extlibs/dist/Lib/_socket.py", line 384, in handle_exception</div> <div> [exec] raise _map_exception(jlx)</div> <div> [exec] _socket.error: [Errno -1] Unmapped exception: io.netty.channel.AbstractChannel$AnnotatedSocketException: Invalid argument: localhost/<a href="http://127.0.0.1:50100" target="_blank">127.0.0.1:50100</a> </div> </div> <div> </div> <div>I was unable to see this error thrown again, but we may have enough to still fix in the exception mapping code.</div> <div> </div> <div>- Jim</div> <div> </div> </div> <div class="gmail_extra"> <div class="gmail_quote">On Wed, May 24, 2017 at 1:26 PM, Stefan Richthofer <span><<a href="mailto:Ste...@gm..." onclick="parent.window.location.href='Ste...@gm...'; return false;" target="_blank">Ste...@gm...</a>></span> wrote: <blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;"> <div> <div style="font-family: Verdana;font-size: 12.0px;"> <div> <div>Did someone test this on OSX? (reminder: <a href="https://github.com/Stewori/jython" target="_blank">https://github.com/Stewori/jython</a>)</div> <div>Or on Java 7?</div> <div>I just tested on Java 7 and the only test I observe to fail unexpectedly is test_random:</div> <div> </div> <div>test_random crashed -- <type 'exceptions.IOError'>: [Errno 13] Permission denied: '/data/workspace/linux/Jython_extlib/jython/dist/testreports/TEST-test.test_random.WichmannHill_TestBasicOps.xml'</div> <div> </div> <div>The test passes when run individually.</div> <div> </div> <div>However that is on my old laptop running Linux Mint Debian/LMDE2. It seems to be</div> <div>some io error due to access rights and also occurs when running with Java8 on that</div> <div>machine. Note that it passes with Java 8 on my current system. Unless this is due to</div> <div>Java 8_111 vs Java 8_131, I guess it's system dependent. Strange though:</div> <div>worked well before extlib update.</div> <div> </div> <div>Still I'd suggest to get this stuff in for RC-phase, given that the test passes individually.</div> <div>I'd merge this stuff as soon as someone reports OSX results.</div> <div> </div> <div>-Stefan</div> <div> </div> <div> <div style="margin: 10.0px 5.0px 5.0px 10.0px;padding: 10.0px 0 10.0px 10.0px;border-left: 2.0px solid rgb(195,217,229);"> <div style="margin: 0 0 10.0px 0;"><b>Gesendet:</b> Dienstag, 23. Mai 2017 um 23:57 Uhr<br/> <b>Von:</b> "Stefan Richthofer" <<a href="mailto:Ste...@gm..." onclick="parent.window.location.href='Ste...@gm...'; return false;" target="_blank">Ste...@gm...</a>><br/> <b>An:</b> "Jim Baker" <<a href="mailto:jim...@py..." onclick="parent.window.location.href='jim...@py...'; return false;" target="_blank">jim...@py...</a>> <div> <div class="h5"><br/> <b>Cc:</b> "Jython Developers" <<a href="mailto:jyt...@li..." onclick="parent.window.location.href='jyt...@li...'; return false;" target="_blank">jyt...@li...</a>><br/> <b>Betreff:</b> Re: [Jython-dev] Updating extlibs</div> </div> </div> <div> <div> <div class="h5"> <div style="font-family: Verdana;font-size: 12.0px;"> <div> <div>Just noticed guava > 20.0 requires Java 8.</div> <div>Strange thing: When building and running Jython on Java 8 with guava 22.0 all regrtests pass fine, but jar-standalone build is bad. Seems like our ant task fails to copy com.google -> org.python.google. Note that I double and triple checked I adjusted build.xml correctly. Is this related to jarjar? Is it maybe not compatible with Java8 jar-files? This might become a problem at some point.</div> <div>Studying guava 22.0 release notes more properly showed that one shall use guava-22.0-android.jar to target Java 7 also on non-Android platforms. With guava-22.0-android.jar, building Jython jar-standalone works well indeed. regrtests also seem to pass then.</div> <div>Ideas?</div> <div>I'll update the linked repo from last email with guava-22.0-android.jar...</div> <div> <div style="margin: 10.0px 5.0px 5.0px 10.0px;padding: 10.0px 0 10.0px 10.0px;border-left: 2.0px solid rgb(195,217,229);"> <div style="margin: 0 0 10.0px 0;"><b>Gesendet:</b> Montag, 22. Mai 2017 um 17:22 Uhr<br/> <b>Von:</b> "Jim Baker" <<a href="mailto:jim...@py..." onclick="parent.window.location.href='jim...@py...'; return false;" target="_blank">jim...@py...</a>><br/> <b>An:</b> "Stefan Richthofer" <<a href="mailto:Ste...@gm..." onclick="parent.window.location.href='Ste...@gm...'; return false;" target="_blank">Ste...@gm...</a>><br/> <b>Cc:</b> "Jython Developers" <<a href="mailto:jyt...@li..." onclick="parent.window.location.href='jyt...@li...'; return false;" target="_blank">jyt...@li...</a>><br/> <b>Betreff:</b> Re: [Jython-dev] Updating extlibs</div> <div> <div>These changes look good to me. I will test out your patch, but all of this is in line with similar updates we have made in the past, usually around this time in the dev cycle. <div> </div> <div>I'm glad that moving to Gradle will make this re-pinning to upstream dependencies much easier going forward!</div> </div> <div class="gmail_extra"> <div class="gmail_quote">On Mon, May 22, 2017 at 8:46 AM, Stefan Richthofer <span><<a href="mailto:Ste...@gm..." onclick="parent.window.location.href='Ste...@gm...'; return false;" target="_blank">Ste...@gm...</a>></span> wrote: <blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">I uploaded the mentioned updates to<br/> <a href="https://github.com/Stewori/jython" target="_blank">https://github.com/Stewori/jython</a>.<br/> See detailed changes at<br/> <a href="https://github.com/Stewori/jython/commit/ee2b1263306f779bf8e9499afc6d02267648ac36" target="_blank">https://github.com/Stewori/jython/commit/ee2b1263306f779bf8e9499afc6d02267648ac36</a><br/> <br/> This might not yet work smoothly with Java 7, I will check and adjust that tomorrow.<br/> Some packages were built from source using Java 8 and I'm not sure whether the gradle<br/> scripts always configured Java 7 source compatibility properly.<br/> However if some people could test it, especially on OSX, would be nice.<br/> <br/> Best<br/> <br/> Stefan<br/> <br/> <br/> > Gesendet: Montag, 22. Mai 2017 um 04:07 Uhr<br/> > Von: "Stefan Richthofer" <<a href="mailto:Ste...@gm..." onclick="parent.window.location.href='Ste...@gm...'; return false;" target="_blank">Ste...@gm...</a>><br/> > An: "Jython Developers" <<a href="mailto:jyt...@li..." onclick="parent.window.location.href='jyt...@li...'; return false;" target="_blank">jyt...@li...</a>><br/> > Betreff: Updating extlibs <div class="m_-4666966744133675115HOEnZb"> <div class="m_-4666966744133675115h5">><br/> > Hey all,<br/> > I spent some effort to explore feasibility of updating extlibs, especially minor versions (but sometimes even major version, e.g. guava and icu4j).<br/> > My results so far, tested with regrtest on Linux and Windows 10 using Java8 ("okay" means I did not observe additional regrtest failures):<br/> ><br/> > ASM 5.0.4 -> 5.2 okay<br/> > bouncycastle:<br/> > bcpkix-jdk15on-1.54 -> 1.57 okay<br/> > bcprov-jdk15on-1.54 -> 1.57 okay<br/> ><br/> > commons-compress-1.12 -> 1.14 okay<br/> > guava-20.0 -> 22.0rc1 okay<br/> > icu4j-58.1 -> 59_1 okay<br/> > Netty 4.1.6 -> 4.1.11 okay<br/> > java-sizeof-0.0.5 -- still current<br/> > jffi-1.2.13 -> 1.2.15 okay<br/> > jnr-ffi-2.1.0 -> 2.1.5 okay<br/> > jnr-netdb-1.1.6 -- still current<br/> > jnr-posix-3.0.31 -> 3.0.41 okay<br/> > jnr-constants-0.9.5 -> 0.9.9 okay<br/> > New platforms: jffi-aarch64-Linux.jar, jffi-ppc64le-Linux.jar, can be added...?<br/> > Updated various other changed platform specific jars to jffi-1.2.15 (okay as far as tested)<br/> > xercesImpl-2.11.0 -- still current<br/> > jline-2.14.2 -> 2.14.3 okay (didn't try jline-3 this time)<br/> > jarjar-1.4 -- still current<br/> > mysql-connector-java-5.1.6 -> 5.1.42 okay<br/> > postgresql-8.3-603.jdbc4 -> 42.1.1-jre7 okay<br/> ><br/> > ----------These failed, so leaving them as it is:<br/> > Antlr 3.1.3 -> 3.5.2 fails<br/> > junit-4.10 -> 4.12 or 4.11 class file for org.hamcrest.Matcher not found<br/> > (staying with 4.10 for now to avoid new dependency on hamcrest-matcher)<br/> > javax.servlet-api-2.5 -> 3.1.0 fails<br/> > mockrunner (better don't touch; whole structure changed)<br/> > cpptasks (better don't touch)<br/> ><br/> > - will be able to test with Java 7 on Tuesday, because I left my old laptop in the office.<br/> > - will upload a fork containing these updates. Would be good if someone else could also test, especially on OSX.<br/> > Maybe some stuff was not covered by regrtests. However the chance that updates solve issues and that they create issues are probably somewhat equal and I'd prefer to focus on fixing issues with current versions rather than older ones.<br/> > So I'm strongly for getting this in if regrtests go through on Java 7 and OSX. 2.7.1RC-phase will be a good opportunity to confirm workability of these updates. Any concerns?<br/> ><br/> > -Stefan<br/> <br/> ------------------------------------------------------------------------------<br/> Check out the vibrant tech community on one of the world's most<br/> engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot" target="_blank">http://sdm.link/slashdot</a><br/> _______________________________________________<br/> Jython-dev mailing list<br/> <a href="mailto:Jyt...@li..." onclick="parent.window.location.href='Jyt...@li...'; return false;" target="_blank">Jyt...@li...</a><br/> <a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a></div> </div> </blockquote> </div> </div> </div> </div> </div> </div> </div> </div> </div> ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot_______________________________________________" target="_blank">http://sdm.link/slashdot_______________________________________________</a> Jython-dev mailing list <a href="mailto:Jyt...@li..." onclick="parent.window.location.href='Jyt...@li...'; return false;" target="_blank">Jyt...@li...</a> <a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a></div> </div> </div> </div> </div> </div> <br/> ------------------------------------------------------------------------------<br/> Check out the vibrant tech community on one of the world's most<br/> engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot" target="_blank">http://sdm.link/slashdot</a><br/> _______________________________________________<br/> Jython-dev mailing list<br/> <a href="mailto:Jyt...@li..." onclick="parent.window.location.href='Jyt...@li...'; return false;" target="_blank">Jyt...@li...</a><br/> <a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a><br/> </blockquote> </div> </div> </div> </div> </div> </div></div></body></html> |
