Menu
▾
▴
jython-dev — Discussion of developing and enhancing Jython itself(!) (not how to use Jython)
You can subscribe to this list here.
| 2000 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(38) |
Nov
(98) |
Dec
(58) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2001 |
Jan
(114) |
Feb
(123) |
Mar
(96) |
Apr
(66) |
May
(84) |
Jun
(72) |
Jul
(128) |
Aug
(126) |
Sep
(82) |
Oct
(80) |
Nov
(148) |
Dec
(55) |
| 2002 |
Jan
(137) |
Feb
(85) |
Mar
(118) |
Apr
(67) |
May
(71) |
Jun
(28) |
Jul
(69) |
Aug
(48) |
Sep
(83) |
Oct
(79) |
Nov
(54) |
Dec
(32) |
| 2003 |
Jan
(44) |
Feb
(47) |
Mar
(59) |
Apr
(57) |
May
(43) |
Jun
(45) |
Jul
(44) |
Aug
(39) |
Sep
(27) |
Oct
(62) |
Nov
(17) |
Dec
(23) |
| 2004 |
Jan
(41) |
Feb
(51) |
Mar
(38) |
Apr
(30) |
May
(25) |
Jun
(12) |
Jul
(11) |
Aug
(27) |
Sep
(16) |
Oct
(56) |
Nov
(23) |
Dec
(29) |
| 2005 |
Jan
(75) |
Feb
(82) |
Mar
(50) |
Apr
(77) |
May
(19) |
Jun
(104) |
Jul
(47) |
Aug
(42) |
Sep
(28) |
Oct
(143) |
Nov
(62) |
Dec
(13) |
| 2006 |
Jan
(20) |
Feb
(10) |
Mar
(59) |
Apr
(45) |
May
(25) |
Jun
(129) |
Jul
(162) |
Aug
(91) |
Sep
(15) |
Oct
(39) |
Nov
(186) |
Dec
(191) |
| 2007 |
Jan
(134) |
Feb
(140) |
Mar
(106) |
Apr
(77) |
May
(92) |
Jun
(63) |
Jul
(233) |
Aug
(102) |
Sep
(119) |
Oct
(63) |
Nov
(68) |
Dec
(32) |
| 2008 |
Jan
(69) |
Feb
(91) |
Mar
(129) |
Apr
(44) |
May
(18) |
Jun
(53) |
Jul
(50) |
Aug
(25) |
Sep
(11) |
Oct
(28) |
Nov
(67) |
Dec
(36) |
| 2009 |
Jan
(20) |
Feb
(24) |
Mar
(66) |
Apr
(53) |
May
(48) |
Jun
(48) |
Jul
(59) |
Aug
(82) |
Sep
(49) |
Oct
(30) |
Nov
(16) |
Dec
(16) |
| 2010 |
Jan
(52) |
Feb
(25) |
Mar
(36) |
Apr
(34) |
May
(14) |
Jun
(15) |
Jul
(14) |
Aug
(16) |
Sep
(23) |
Oct
(6) |
Nov
(4) |
Dec
(5) |
| 2011 |
Jan
(4) |
Feb
(22) |
Mar
(45) |
Apr
(9) |
May
(8) |
Jun
(13) |
Jul
(12) |
Aug
(4) |
Sep
(6) |
Oct
(10) |
Nov
(21) |
Dec
(5) |
| 2012 |
Jan
(6) |
Feb
(9) |
Mar
(25) |
Apr
(6) |
May
(4) |
Jun
(23) |
Jul
(6) |
Aug
(18) |
Sep
(21) |
Oct
(34) |
Nov
(19) |
Dec
(25) |
| 2013 |
Jan
(8) |
Feb
(34) |
Mar
(35) |
Apr
(4) |
May
(11) |
Jun
(4) |
Jul
(7) |
Aug
(5) |
Sep
(20) |
Oct
(12) |
Nov
(11) |
Dec
(7) |
| 2014 |
Jan
(10) |
Feb
(18) |
Mar
(50) |
Apr
(26) |
May
(53) |
Jun
(21) |
Jul
(12) |
Aug
(39) |
Sep
(43) |
Oct
(26) |
Nov
(8) |
Dec
(6) |
| 2015 |
Jan
(18) |
Feb
(32) |
Mar
(31) |
Apr
(42) |
May
(38) |
Jun
(13) |
Jul
(6) |
Aug
(11) |
Sep
(29) |
Oct
(25) |
Nov
(10) |
Dec
(11) |
| 2016 |
Jan
(24) |
Feb
(12) |
Mar
(13) |
Apr
(15) |
May
(22) |
Jun
(8) |
Jul
(12) |
Aug
(25) |
Sep
(8) |
Oct
(6) |
Nov
(13) |
Dec
(7) |
| 2017 |
Jan
(6) |
Feb
(29) |
Mar
(32) |
Apr
(8) |
May
(82) |
Jun
(42) |
Jul
(20) |
Aug
(17) |
Sep
(27) |
Oct
(14) |
Nov
(22) |
Dec
(6) |
| 2018 |
Jan
(12) |
Feb
(9) |
Mar
(22) |
Apr
(19) |
May
(14) |
Jun
(9) |
Jul
(9) |
Aug
(22) |
Sep
(22) |
Oct
(12) |
Nov
(13) |
Dec
(8) |
| 2019 |
Jan
(22) |
Feb
(3) |
Mar
(30) |
Apr
(20) |
May
(20) |
Jun
(6) |
Jul
(15) |
Aug
(25) |
Sep
(11) |
Oct
(24) |
Nov
(11) |
Dec
(6) |
| 2020 |
Jan
(9) |
Feb
(12) |
Mar
(29) |
Apr
(10) |
May
(22) |
Jun
(11) |
Jul
(15) |
Aug
(5) |
Sep
(6) |
Oct
(7) |
Nov
(7) |
Dec
(13) |
| 2021 |
Jan
(21) |
Feb
(5) |
Mar
(5) |
Apr
(6) |
May
(10) |
Jun
(7) |
Jul
(6) |
Aug
(8) |
Sep
(5) |
Oct
(9) |
Nov
(5) |
Dec
(6) |
| 2022 |
Jan
(5) |
Feb
(4) |
Mar
(8) |
Apr
(6) |
May
(5) |
Jun
(5) |
Jul
(10) |
Aug
(6) |
Sep
(7) |
Oct
(4) |
Nov
(4) |
Dec
(6) |
| 2023 |
Jan
(5) |
Feb
(5) |
Mar
(6) |
Apr
(4) |
May
(5) |
Jun
(6) |
Jul
(5) |
Aug
(5) |
Sep
(5) |
Oct
(5) |
Nov
(7) |
Dec
(8) |
| 2024 |
Jan
(3) |
Feb
(1) |
Mar
|
Apr
(2) |
May
|
Jun
(1) |
Jul
(1) |
Aug
(4) |
Sep
|
Oct
|
Nov
|
Dec
(1) |
| 2025 |
Jan
|
Feb
(2) |
Mar
|
Apr
(1) |
May
|
Jun
(1) |
Jul
|
Aug
(1) |
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Yaqiang W. <yaq...@gm...> - 2017-08-30 06:09:25
|
One of the deal ends of Jython is no multi-dimensional array computing library like Numpy of CPython. JyNI is a good solution to support Numpy from Jython. Another way is to develop Numpy like functions using Java and Jython. I am working on a software of MeteoInfo which has now fulfilled many functions of Numpy, Matplotlib and Basemap only use Jython and Java. MeteoInfo is free and open source under LGPL license ( http://www.meteothinker.com/). Welcome to use and develop it. Regards Yaqiang ------------------------------ ************************************************* Dr. Yaqiang Wang Chinese Academy of Meteorological Sciences (CAMS) 46, Zhong-Guan-Cun South Avenue Beijing, 100081 China yaq...@gm... http://www.meteothinker.com *https://www.researchgate.net/profile/Yaqiang_Wang5 <https://www.researchgate.net/profile/Yaqiang_Wang5>* ************************************************** |
|
From: Adam B. <ada...@gm...> - 2017-08-27 22:21:39
|
Was some content lost from the wiki? https://wiki.python.org/jython/FrontPage?action=fullsearch&titlesearch=0&value=Pytype&context=180 Adam > 在 2017年8月27日，下午11:02，Jeff Allen <ja...@fa...> 写道： > > No problem. Sometimes it's difficult to tell what's intentional. I have done my best to give these *Derived classes too, following the models nearby -- all on my BB fork, not the main repo. (If the fork turns out bent, we can cherry-pick this out.) > > J. > > >> On 26/08/2017 22:51, fwi...@gm... wrote: >> The ast.* cases are oversights (specifically my oversights, I think). I suspect the other is an oversight as well. >> >> Thanks for finding them! >> >> -Frank >> >> On Aug 26, 2017 05:47, "Jeff Allen" <ja...@fa... <mailto:ja...@fa...>> wrote: >> >> Tinkering with the logic of PyType, I have run across cases where >> a class is annotated @ExposedType but does produce a TypeBuilder >> when initialised. This is because they are not listed in >> CoreExposed.includes, are not processed by the exposer, and >> therefore are represented by a PyJavaType object. >> >> I am experimenting with using the annotation, rather than static >> initialisation and looking for a builder, as my way to distinguish >> the two cases, and which reveals this discrepancy whe addBuilder >> is not called. >> >> So far I found: PyNullImporter, org.python.modules.itertools.imap, >> org.python.antlr.ast.SetComp, org.python.antlr.ast.SetComp, >> PyXRangeIter. >> >> Such cases are just an oversight, not some subtle choice I haven't >> understood, right? >> >> Jeff >> >> >> ------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> _______________________________________________ >> Jython-dev mailing list >> Jyt...@li... >> <mailto:Jyt...@li...> >> https://lists.sourceforge.net/lists/listinfo/jython-dev >> <https://lists.sourceforge.net/lists/listinfo/jython-dev> >> > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev |
|
From: Jeff A. <ja...@fa...> - 2017-08-27 15:02:40
|
No problem. Sometimes it's difficult to tell what's intentional. I have done my best to give these *Derived classes too, following the models nearby -- all on my BB fork, not the main repo. (If the fork turns out bent, we can cherry-pick this out.) J. On 26/08/2017 22:51, fwi...@gm... wrote: > The ast.* cases are oversights (specifically my oversights, I think). > I suspect the other is an oversight as well. > > Thanks for finding them! > > -Frank > > On Aug 26, 2017 05:47, "Jeff Allen" <ja...@fa... > <mailto:ja...@fa...>> wrote: > > Tinkering with the logic of PyType, I have run across cases where > a class is annotated @ExposedType but does produce a TypeBuilder > when initialised. This is because they are not listed in > CoreExposed.includes, are not processed by the exposer, and > therefore are represented by a PyJavaType object. > > I am experimenting with using the annotation, rather than static > initialisation and looking for a builder, as my way to distinguish > the two cases, and which reveals this discrepancy whe addBuilder > is not called. > > So far I found: PyNullImporter, org.python.modules.itertools.imap, > org.python.antlr.ast.SetComp, org.python.antlr.ast.SetComp, > PyXRangeIter. > > Such cases are just an oversight, not some subtle choice I haven't > understood, right? > > Jeff > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > <mailto:Jyt...@li...> > https://lists.sourceforge.net/lists/listinfo/jython-dev > <https://lists.sourceforge.net/lists/listinfo/jython-dev> > |
|
From: Jeff A. <ja...@fa...> - 2017-08-27 14:58:01
|
Good. There has been so much in PyType I thought I understood but broke. This could easily, in my mind, have been another. I've corrected as many types as I've come across, and it passes tests, but I don't quite know what I'm doing here. (I had notes from the first time I learned it, but they were on the Wiki.) I've just pushed recent work to https://bitbucket.org/tournesol/jython-fgtype . I still have a few more ideas for things I can break ... Jeff On 26/08/2017 16:51, Jim Baker wrote: > Jeff, > > Agreed that these all must be an oversight. I specifically contributed > PyXRangeIter per the blame info, and I know I would have added this to > CoreExposed.includes. Not being exposed can impact some aspects of > Python integration, eg right now the __doc__ string is not being > exposed properly, but most likely the key thing that is impacted with > these specific classes is possibly higher calling overhead due to > reflection. So that's why we haven't caught it until now. > > Thanks for systematically digging into this! > > I also plan to respond to your earlier analysis about using > ClassValue. I was busy, then on vacation, then busy again. Summer! > > Lastly I was hopeful that exposing this iterator would see at least > some impact in a tight loop, but it's not visible in a trivial > benchmark like > > jython -m timeit -n 100000 'def f():' ' c = 0' ' for i in > xrange(10000):' ' c += 1' 'f()' > > It might be worthwhile trying https://github.com/zware/codespeed, but > obviously a secondary concern. Any performance tuning with respect to > function calling — and the use of fun stuff like InvokeDynamic and > SwitchPoint — should go into Jython 3. > > - Jim > > On Sat, Aug 26, 2017 at 6:46 AM, Jeff Allen <ja...@fa... > <mailto:ja...@fa...>> wrote: > > Tinkering with the logic of PyType, I have run across cases where > a class is annotated @ExposedType but does produce a TypeBuilder > when initialised. This is because they are not listed in > CoreExposed.includes, are not processed by the exposer, and > therefore are represented by a PyJavaType object. > > I am experimenting with using the annotation, rather than static > initialisation and looking for a builder, as my way to distinguish > the two cases, and which reveals this discrepancy whe addBuilder > is not called. > > So far I found: PyNullImporter, org.python.modules.itertools.imap, > org.python.antlr.ast.SetComp, org.python.antlr.ast.SetComp, > PyXRangeIter. > > Such cases are just an oversight, not some subtle choice I haven't > understood, right? > > Jeff > > |
|
From: <fwi...@gm...> - 2017-08-26 21:51:57
|
The ast.* cases are oversights (specifically my oversights, I think). I suspect the other is an oversight as well. Thanks for finding them! -Frank On Aug 26, 2017 05:47, "Jeff Allen" <ja...@fa...> wrote: > Tinkering with the logic of PyType, I have run across cases where a class > is annotated @ExposedType but does produce a TypeBuilder when initialised. > This is because they are not listed in CoreExposed.includes, are not > processed by the exposer, and therefore are represented by a PyJavaType > object. > > I am experimenting with using the annotation, rather than static > initialisation and looking for a builder, as my way to distinguish the two > cases, and which reveals this discrepancy whe addBuilder is not called. > > So far I found: PyNullImporter, org.python.modules.itertools.imap, > org.python.antlr.ast.SetComp, org.python.antlr.ast.SetComp, PyXRangeIter. > > Such cases are just an oversight, not some subtle choice I haven't > understood, right? > > Jeff > > > ------------------------------------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > |
|
From: Jeff A. <ja...@fa...> - 2017-08-26 12:46:58
|
Tinkering with the logic of PyType, I have run across cases where a class is annotated @ExposedType but does produce a TypeBuilder when initialised. This is because they are not listed in CoreExposed.includes, are not processed by the exposer, and therefore are represented by a PyJavaType object. I am experimenting with using the annotation, rather than static initialisation and looking for a builder, as my way to distinguish the two cases, and which reveals this discrepancy whe addBuilder is not called. So far I found: PyNullImporter, org.python.modules.itertools.imap, org.python.antlr.ast.SetComp, org.python.antlr.ast.SetComp, PyXRangeIter. Such cases are just an oversight, not some subtle choice I haven't understood, right? Jeff |
|
From: Jython t. <st...@bu...> - 2017-08-25 16:10:22
|
ACTIVITY SUMMARY (2017-08-18 - 2017-08-25) Jython tracker at http://bugs.jython.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 319 ( +4) closed 2318 ( +0) total 2637 ( +4) Open issues with patches: 28 Issues opened (4) ================= #2615: select.select using subprocess.Popen error http://bugs.jython.org/issue2615 opened by Brian Kurt #2616: Incomplete / broken support for Certificate Revocation Lists http://bugs.jython.org/issue2616 opened by behackett #2617: GZIPInputStream issue http://bugs.jython.org/issue2617 opened by yuli129 #2618: socket.sendall no longer sends all http://bugs.jython.org/issue2618 opened by behackett Most recent 15 issues with no replies (15) ========================================== #2618: socket.sendall no longer sends all http://bugs.jython.org/issue2618 #2617: GZIPInputStream issue http://bugs.jython.org/issue2617 #2616: Incomplete / broken support for Certificate Revocation Lists http://bugs.jython.org/issue2616 #2615: select.select using subprocess.Popen error http://bugs.jython.org/issue2615 #2613: 2.7.1 MANIFEST.MF contains unfilled placeholders http://bugs.jython.org/issue2613 #2611: mkdir() operation in /Lib/os.py has different behavior when ru http://bugs.jython.org/issue2611 #2606: jython launch problem sun.misc.InvalidJarIndexException http://bugs.jython.org/issue2606 #2605: Jython 2.7.0 startup performance regression http://bugs.jython.org/issue2605 #2591: Unable to execute directory or zip file (test_cmd_line_script) http://bugs.jython.org/issue2591 #2567: System state lost during JSR-223 initialisation http://bugs.jython.org/issue2567 #2562: Windows: OSError: unlink(): an unknown error occurred http://bugs.jython.org/issue2562 #2531: Support SNI for SSL/TLS server sockets http://bugs.jython.org/issue2531 #2525: Jython incorrectly buffers file pipe output with Subprocess(co http://bugs.jython.org/issue2525 #2520: Jython does NOT support socket.listen(0) for accepting only on http://bugs.jython.org/issue2520 #2512: Values in built-in modules's __dict__ are â<reflected field http://bugs.jython.org/issue2512 Most recent 15 issues waiting for review (15) ============================================= #2566: inspect does not recognize code objects from bytecode files http://bugs.jython.org/issue2566 #2545: help() does not work on Java-implemented modules http://bugs.jython.org/issue2545 #2429: cStringIO does not work with mutable objects implementing the http://bugs.jython.org/issue2429 #2367: Jython ignores custom __eq__ when hashing dict subclasses http://bugs.jython.org/issue2367 #2330: full-build fails to copy CPython License http://bugs.jython.org/issue2330 #2230: Jython evaluation blocks under heavy load with high multi-core http://bugs.jython.org/issue2230 #2143: site-packages support in standalone jar http://bugs.jython.org/issue2143 #2142: Set Thread classloader when entering Jython context http://bugs.jython.org/issue2142 #2121: Jython jar on Maven central embeds other third party libraries http://bugs.jython.org/issue2121 #2077: marshal doesn't raise error when fed unmarshalable object http://bugs.jython.org/issue2077 #1925: Support loading java.sql.Drivers that aren't on the boot class http://bugs.jython.org/issue1925 #1917: No ctypes.c_char http://bugs.jython.org/issue1917 #1866: Parser does not have mismatch token error messages caught by B http://bugs.jython.org/issue1866 #1842: Add IBM i support to Jython http://bugs.jython.org/issue1842 #1796: Jython doesn't support jar dir with colon's in it http://bugs.jython.org/issue1796 |
|
From: Jython t. <st...@bu...> - 2017-08-18 16:10:24
|
ACTIVITY SUMMARY (2017-08-11 - 2017-08-18) Jython tracker at http://bugs.jython.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 315 ( +0) closed 2318 ( +0) total 2633 ( +0) Open issues with patches: 28 Most recent 15 issues with no replies (15) ========================================== #2613: 2.7.1 MANIFEST.MF contains unfilled placeholders http://bugs.jython.org/issue2613 #2611: mkdir() operation in /Lib/os.py has different behavior when ru http://bugs.jython.org/issue2611 #2606: jython launch problem sun.misc.InvalidJarIndexException http://bugs.jython.org/issue2606 #2605: Jython 2.7.0 startup performance regression http://bugs.jython.org/issue2605 #2591: Unable to execute directory or zip file (test_cmd_line_script) http://bugs.jython.org/issue2591 #2567: System state lost during JSR-223 initialisation http://bugs.jython.org/issue2567 #2562: Windows: OSError: unlink(): an unknown error occurred http://bugs.jython.org/issue2562 #2531: Support SNI for SSL/TLS server sockets http://bugs.jython.org/issue2531 #2525: Jython incorrectly buffers file pipe output with Subprocess(co http://bugs.jython.org/issue2525 #2520: Jython does NOT support socket.listen(0) for accepting only on http://bugs.jython.org/issue2520 #2512: Values in built-in modules's __dict__ are â<reflected field http://bugs.jython.org/issue2512 #2510: TypeError when monkey-patching time.time with an unbound funct http://bugs.jython.org/issue2510 #2507: Run "invokeFunction" in a thread does not inherit the "ScriptC http://bugs.jython.org/issue2507 #2494: Support for pydoc_data http://bugs.jython.org/issue2494 #2484: Codec encodings can exhaust permgen http://bugs.jython.org/issue2484 Most recent 15 issues waiting for review (15) ============================================= #2566: inspect does not recognize code objects from bytecode files http://bugs.jython.org/issue2566 #2545: help() does not work on Java-implemented modules http://bugs.jython.org/issue2545 #2429: cStringIO does not work with mutable objects implementing the http://bugs.jython.org/issue2429 #2367: Jython ignores custom __eq__ when hashing dict subclasses http://bugs.jython.org/issue2367 #2330: full-build fails to copy CPython License http://bugs.jython.org/issue2330 #2230: Jython evaluation blocks under heavy load with high multi-core http://bugs.jython.org/issue2230 #2143: site-packages support in standalone jar http://bugs.jython.org/issue2143 #2142: Set Thread classloader when entering Jython context http://bugs.jython.org/issue2142 #2121: Jython jar on Maven central embeds other third party libraries http://bugs.jython.org/issue2121 #2077: marshal doesn't raise error when fed unmarshalable object http://bugs.jython.org/issue2077 #1925: Support loading java.sql.Drivers that aren't on the boot class http://bugs.jython.org/issue1925 #1917: No ctypes.c_char http://bugs.jython.org/issue1917 #1866: Parser does not have mismatch token error messages caught by B http://bugs.jython.org/issue1866 #1842: Add IBM i support to Jython http://bugs.jython.org/issue1842 #1796: Jython doesn't support jar dir with colon's in it http://bugs.jython.org/issue1796 Top 10 most discussed issues (1) ================================ #2614: "SSLError: [Errno 1] Illegal state exception" error message wh http://bugs.jython.org/issue2614 6 msgs |
|
From: Jeff A. <ja...@fa...> - 2017-08-17 08:01:17
|
Jim, all: I had a first go at using ClassValue to provide non-blocking look-up. It seems to work. Fork at https://bitbucket.org/tournesol/jython-fgtype . I have moved the relevant static methods of PyType into an inner class (PyType.Registry) with the data structures they manipulate. These methods are essentially unchanged by the move. One needs forwarding methods for fromClass and addBuilder at the outer level like this: public static PyType fromClass(Class<?> c, boolean hardRef) { return Registry.fromClass(c, hardRef); } Data structures in the Registry relevant to this discussion are: private static final class Registry { /** Mapping of Java classes to their PyTypes. */ private static final Map<Class<?>, PyType> classToType = ... ; /** Classes added to classToType where the PyType in not completely init'd */ private static final Set<Class<?>> classToTypeNotReady = new HashSet<>(); /** Acts as a non-blocking cache for PyType look-up. */ private static ClassValue<PyType> cv = new ClassValue<PyType>() {...}; Answering the points about ClassValue below, the set classToTypeNotReady tracks unfinished inners and bootstrap types. The way to prevent caching a result (point 1), I suddenly realised, is to terminate ClassValue.computeValue abnormally: private static ClassValue<PyType> cv = new ClassValue<PyType>() { @Override protected PyType computeValue(Class<?> c) throws NotReady { synchronized (Registry.class) { PyType type = classToType.get(c); if (type == null || classToTypeNotReady.contains(c)) { // Avoid anything being cached in the class value. throw new NotReady(); } return type; } } }; In that case, the outer PyType.fromClass is made to fall back on the familiar version, which (as now) allows re-entrancy in the owning thread, but blocks concurrent access: public static PyType fromClass(Class<?> c, boolean hardRef) { try { return Registry.cv.get(c); } catch (NotReady nr) { return Registry.fromClass(c, hardRef); } } This all passes the regression tests (ant regrtest), slightly quicker if anything. Answering point 2, it does this without holding onto PyTypes when we have finished with them. (See the revised test in test_jy_internals.) This last observation surprised me. classToType is still a double-weak map, but I my use of cv has no explicit weakness or removal code. It must hold the PyType reachable as long as the Class it refers to is reachable, by contract, but apparently no longer. Is this not exactly what we need? Jeff On 09/08/2017 08:24, Jeff Allen wrote: > ... > On 05/08/2017 20:34, in Issue #2387 > (http://bugs.jython.org/issue2487), Jim Baker wrote: >> ... >> Lastly, >> https://docs.oracle.com/javase/7/docs/api/java/lang/ClassValue.html >> looks potentially useful as an alternative for publishing PyType >> objects into a cache. But we need to address two key questions: >> >> 1. Whether ClassValue#computeValue would work properly in the >> re-entrant case where the class graph from a given class C is >> explored, as is done for inners, possibly referring back to C. This >> is why we could not get the publication of the type in the map done >> after the init (as one would usually want to do!) in >> https://github.com/jythontools/jython/blob/master/src/org/python/core/PyType.java#L1515 >> >> (putIfAbsent); class C would be referenced, and it would attempt to >> build again (and stack overflow). No solution I tried could break >> this problem. >> >> 2. When to call ClassValue#removeValue ! We still have to keep track >> of the cache with respect to PyType. > |
|
From: Jeff A. <ja...@fa...> - 2017-08-13 21:54:10
|
PyType really is quite hairy code. It has certainly tested my understanding of Java initialisation. I've not changed anything materially yet, but produced a version plastered with print statements that others might find amusing: https://bitbucket.org/tournesol/jython-fgtype/commits/094f87e1ddaa5d30b4d1be9fcf3209e0ff6432bd It produces output like this: addBuilder(class o.p.c.PyShadowString, class o.p.c.PyShadowString$PyExposer) --- --- --- --- fromClass(class o.p.c.PyShadowString, true) -> null | addFromClass(class o.p.c.PyShadowString) | createType(class o.p.c.PyShadowString) | getBuilder(class o.p.c.PyShadowString) | Class.forName(class o.p.c.PyShadowString) with static init. | fromClass(class o.p.c.PyType, false) -> <type 'type'> | PyType.init(class o.p.c.PyShadowString) | fromClass(class o.p.c.PyString, false) -> <type 'str'> | fromClass(class o.p.c.PyMethodDescr, false) -> <type 'method_descriptor'> | fromClass(class o.p.c.PyMethodDescr, false) -> <type 'method_descriptor'> | fromClass(class o.p.c.PyMethodDescr, false) -> <type 'method_descriptor'> | fromClass(class o.p.c.PyMethodDescr, false) -> <type 'method_descriptor'> | fromClass(class o.p.c.PyMethodDescr, false) -> <type 'method_descriptor'> | fromClass(class o.p.c.PyMethodDescr, false) -> <type 'method_descriptor'> | +-- init'd <type 'shadowstr'> (from class o.p.c.PyShadowString) | +-- created <type 'shadowstr'> from class o.p.c.PyShadowString +-> <type 'shadowstr'> (from class o.p.c.PyShadowString) --- --- --- --- There is some odd-but-not-incorrect behaviour amongst the 15000 or so lines, including PyArray initialised twice and _codecs$EncodingMap left in the bootstrap types at the end (until you look up the right kind of codec). Now I understand the logic better, and have a baseline, I'll try the ClassValue idea, and exhibit it in the same place. Jeff |
|
From: Jython t. <st...@bu...> - 2017-08-11 16:10:23
|
ACTIVITY SUMMARY (2017-08-04 - 2017-08-11) Jython tracker at http://bugs.jython.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 315 ( +1) closed 2318 ( +0) total 2633 ( +1) Open issues with patches: 28 Issues opened (1) ================= #2614: "SSLError: [Errno 1] Illegal state exception" error message wh http://bugs.jython.org/issue2614 opened by Neha Most recent 15 issues with no replies (15) ========================================== #2613: 2.7.1 MANIFEST.MF contains unfilled placeholders http://bugs.jython.org/issue2613 #2611: mkdir() operation in /Lib/os.py has different behavior when ru http://bugs.jython.org/issue2611 #2606: jython launch problem sun.misc.InvalidJarIndexException http://bugs.jython.org/issue2606 #2605: Jython 2.7.0 startup performance regression http://bugs.jython.org/issue2605 #2591: Unable to execute directory or zip file (test_cmd_line_script) http://bugs.jython.org/issue2591 #2567: System state lost during JSR-223 initialisation http://bugs.jython.org/issue2567 #2562: Windows: OSError: unlink(): an unknown error occurred http://bugs.jython.org/issue2562 #2531: Support SNI for SSL/TLS server sockets http://bugs.jython.org/issue2531 #2525: Jython incorrectly buffers file pipe output with Subprocess(co http://bugs.jython.org/issue2525 #2520: Jython does NOT support socket.listen(0) for accepting only on http://bugs.jython.org/issue2520 #2512: Values in built-in modules's __dict__ are â<reflected field http://bugs.jython.org/issue2512 #2510: TypeError when monkey-patching time.time with an unbound funct http://bugs.jython.org/issue2510 #2507: Run "invokeFunction" in a thread does not inherit the "ScriptC http://bugs.jython.org/issue2507 #2494: Support for pydoc_data http://bugs.jython.org/issue2494 #2484: Codec encodings can exhaust permgen http://bugs.jython.org/issue2484 Most recent 15 issues waiting for review (15) ============================================= #2566: inspect does not recognize code objects from bytecode files http://bugs.jython.org/issue2566 #2545: help() does not work on Java-implemented modules http://bugs.jython.org/issue2545 #2429: cStringIO does not work with mutable objects implementing the http://bugs.jython.org/issue2429 #2367: Jython ignores custom __eq__ when hashing dict subclasses http://bugs.jython.org/issue2367 #2330: full-build fails to copy CPython License http://bugs.jython.org/issue2330 #2230: Jython evaluation blocks under heavy load with high multi-core http://bugs.jython.org/issue2230 #2143: site-packages support in standalone jar http://bugs.jython.org/issue2143 #2142: Set Thread classloader when entering Jython context http://bugs.jython.org/issue2142 #2121: Jython jar on Maven central embeds other third party libraries http://bugs.jython.org/issue2121 #2077: marshal doesn't raise error when fed unmarshalable object http://bugs.jython.org/issue2077 #1925: Support loading java.sql.Drivers that aren't on the boot class http://bugs.jython.org/issue1925 #1917: No ctypes.c_char http://bugs.jython.org/issue1917 #1866: Parser does not have mismatch token error messages caught by B http://bugs.jython.org/issue1866 #1842: Add IBM i support to Jython http://bugs.jython.org/issue1842 #1796: Jython doesn't support jar dir with colon's in it http://bugs.jython.org/issue1796 Top 10 most discussed issues (1) ================================ #2614: "SSLError: [Errno 1] Illegal state exception" error message wh http://bugs.jython.org/issue2614 6 msgs |
|
From: Jeff A. <ja...@fa...> - 2017-08-09 07:24:48
|
Jim, all: Apologies for the near repeat, but my first attempt to give this a life of its own on Jython-dev ended up nested under issue 2487 again. Hope this lands as a fresh topic. On 05/08/2017 20:34, in Issue #2387 (http://bugs.jython.org/issue2487), Jim Baker wrote: > ... > > Next, let's look at our caching, and its corresponding > performance. Using weak keys/weak values in class_to_type means that > when working with Java objects (calling methods, accessing > attributes), Python code that does not maintain a strong reference to > the Java class will cause the Jython runtime to constantly rebuild > these entries in class_to_type. Here are the ways that such strong > references would be established: > > 1. exposedTypes (so Jython's internals) > https://hg.python.org/jython/file/1a14d360dc8d/src/org/python/core/PyType.java#l104 > > > 2. imports of Java classes into a Python namespace > > 3. references to `type(java_obj)` > > 4. subclassing of a Java class by a Python class > > References to Java *objects*, including objects from factory methods; > indirect construction, such as `java.util.HashMap()`; and callbacks > (Netty, re/json/strptime) would not introduce such references to > Python types for Java *classes*. But calling methods, using > attributes, or otherwise using getting the Python type on the Java > object, would require this entry in class_to_type to be added, > potentially to be potentially quickly discarded by generational > GC. Computing these entries is relatively expensive, especially if a > given class C has inner classes (and this is a general type graph > expansion). > > See also the analysis in https://bugs.openjdk.java.net/browse/JDK-6389107 > (scenario 3); our mapping in class_to_type is a common pattern seen in > Java systems. > > What if instead of current one-level cache, we had a two-level cache? > Level 1 uses weak key/weak value for the class/type entries (C, T); > (C, T) entries that are expired from Level 1 are moved to Level 2; > Level 2 uses an expiration time so a ClassLoader could be unloaded in > say some reasonable amount of time. > > To implement: > > 1. Level 1 cache is a direct replacment of class_to_type with > CacheBuilder weak keys/weak values/removal listener. Also we will > not attempt to expose it as a Map going forward; adjust > class_to_type usage accordingly. > > 2. Level 2 cache is weak keys/strong values/expires after write. We > can tune this expiration as was done with the regex cache and the > python.sre.cachespec option, > https://github.com/jythontools/jython/blob/master/src/org/python/modules/sre/SRE_STATE.java#L1253). > > > 3. For removalListener: (C, T) entries that are removed from Level 1 > will use removalListener to get placed in Level 2 (safe because > Level 1 is concurrent; and obviously hard refs would prevent it > from being removed while under construction). > > 4. Try both caches when looking up the type T for class C. Because > synchronized, there's no visible inconsistency; if no entry for C > because it was removed, simply recompute T'. There is a small > window of time that it could be in the process of being moved to > Level 2, but this does not affect correctness; (C, T) will be > eventually expired from Level 2 and (C, T') is a valid repllacement > for it (because there can be no hard refs to T outside the level 2 > cache itself). > > Lastly, > https://docs.oracle.com/javase/7/docs/api/java/lang/ClassValue.html > looks potentially useful as an alternative for publishing PyType > objects into a cache. But we need to address two key questions: > > 1. Whether ClassValue#computeValue would work properly in the > re-entrant case where the class graph from a given class C is > explored, as is done for inners, possibly referring back to C. This > is why we could not get the publication of the type in the map done > after the init (as one would usually want to do!) in > https://github.com/jythontools/jython/blob/master/src/org/python/core/PyType.java#L1515 > > (putIfAbsent); class C would be referenced, and it would attempt to > build again (and stack overflow). No solution I tried could break > this problem. > > 2. When to call ClassValue#removeValue ! We still have to keep track > of the cache with respect to PyType. I have an idea that I can use ClassValue with this, but it sits as a (lock-free) cache in front of the same (properly locked) apparatus we have now. Clients see a static PyType.forClass() that is not synchronised and simply calls the (ClassValue) pyType.get(c). A miss results in pyType.computeValue(c), which uses the existing synchronised apparatus. When we wrestle with types internally to the type registry, we call an internal Registry.forClass(), and other methods, that are essentially the ones we're familiar with. We don't use the ClassValue pyType for look-up internally because that would expose incomplete types to other threads. In the first instance, this means I can leave the existing code essentially unmodified, apart from turning it into a private delegate. The limitation of ClassValue is that it doesn't let you supply information in the look-up call that can be passed to computeValue: it has to be possible using only the class c itself, and information provided beforehand. In our case, the extra argument is just the boolean hardRef that is supposed to decide whether the PyType created ends up in exposedTypes. I think I can deal with it, awkwardly, but that API seems questionable: many places the first call is PyType.forClass(this, false) and a later (default) call has hardRef=true, but comes too late. The creation of 'object' and 'type' are notable examples of core types missing from exposedTypes. In other places, an explicit false or implicit true is given when the thought may only be to look something up, and the possibility it will be created at that point may not have been considered. I wonder if we could decide, either by looking at the class (name) or through a list specified in advance, that certain classes should be strongly referenced? I think I understand the intent of the two-level cache, and trying ClassValue doesn't invalidate it. It might do part of the work for us. In particular, a ClassValue does not prevent unloading of the class. (It's a weak-keys cache.) It keeps its value alive until the corresponding class is unloaded (unless the value is explicitly a weak reference). Thing is, I don't understand in what circumstances we ought to create the strong reference: what need are we actually seeking to meet? Is the critical use case that the PyType could be freed when Python code stops using it, and yet the Java class has reason to remain? Jeff |
|
From: Javen O'N. <jav...@gm...> - 2017-08-08 10:35:49
|
What is sys.path for both? If the python library you're trying to import is installed outside of the python installation or virtualenv, make sure you append an absolute path to sys.path rather than a relative one. On Aug 8, 2017 2:16 AM, "Venkatesh Adiga" <rv...@gm...> wrote: Hello, I am jconsole to connect to remote machine and execute remote python script. The remote python script has dependency of python package, so does import. This python script executes correctly on command line. But when trying to execute through jconsole, the same script fails to import. The cataline log indicates that specific package failed to import. Can anyone please throw some light on this ? Thank you in advance Regards Venkatesh ------------------------------------------------------------ ------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Jython-dev mailing list Jyt...@li... https://lists.sourceforge.net/lists/listinfo/jython-dev |
|
From: Rory O'D. <ror...@or...> - 2017-08-08 09:52:40
|
Hi Alan, Thank you very much for all your testing of JDK 9 during its development! Such contributions have significantly helped shape and improve JDK 9. Now that we have reached the JDK 9 Final Release Candidate phase [1] , I would like to ask if your project can be considered to be 'ready for JDK 9', or if there are any remaining show stopper issues which you've encountered when testing with the JDK 9 release candidate. JDK 9 b181 is available at http://jdk.java.net/9/ If you have a public web page, mailing list post, or even a tweet announcing you project's readiness for JDK 9, I'd love to add the URL to the upcoming JDK 9 readiness page on the Quality Outreach wiki. Looking forward to hearing from you, Rory [1] http://openjdk.java.net/projects/jdk9/ -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA , Dublin, Ireland |
|
From: Venkatesh A. <rv...@gm...> - 2017-08-08 09:16:12
|
Hello, I am jconsole to connect to remote machine and execute remote python script. The remote python script has dependency of python package, so does import. This python script executes correctly on command line. But when trying to execute through jconsole, the same script fails to import. The cataline log indicates that specific package failed to import. Can anyone please throw some light on this ? Thank you in advance Regards Venkatesh |
|
From: Jeff A. <ja...@fa...> - 2017-08-07 22:08:52
|
Moving this onto Jython-Dev for a wider readership. Rest of my message bottom-posted because you want to read Jim's first. On 05/08/2017 20:34, in Issue #2387 (http://bugs.jython.org/issue2487), Jim Baker wrote: > ... > > Next, let's look at our caching, and its corresponding > performance. Using weak keys/weak values in class_to_type means that > when working with Java objects (calling methods, accessing > attributes), Python code that does not maintain a strong reference to > the Java class will cause the Jython runtime to constantly rebuild > these entries in class_to_type. Here are the ways that such strong > references would be established: > > 1. exposedTypes (so Jython's internals) > https://hg.python.org/jython/file/1a14d360dc8d/src/org/python/core/PyType.java#l104 > > 2. imports of Java classes into a Python namespace > > 3. references to `type(java_obj)` > > 4. subclassing of a Java class by a Python class > > References to Java *objects*, including objects from factory methods; > indirect construction, such as `java.util.HashMap()`; and callbacks > (Netty, re/json/strptime) would not introduce such references to > Python types for Java *classes*. But calling methods, using > attributes, or otherwise using getting the Python type on the Java > object, would require this entry in class_to_type to be added, > potentially to be potentially quickly discarded by generational > GC. Computing these entries is relatively expensive, especially if a > given class C has inner classes (and this is a general type graph > expansion). > > See also the analysis in https://bugs.openjdk.java.net/browse/JDK-6389107 > (scenario 3); our mapping in class_to_type is a common pattern seen in > Java systems. > > What if instead of current one-level cache, we had a two-level cache? > Level 1 uses weak key/weak value for the class/type entries (C, T); > (C, T) entries that are expired from Level 1 are moved to Level 2; > Level 2 uses an expiration time so a ClassLoader could be unloaded in > say some reasonable amount of time. > > To implement: > > 1. Level 1 cache is a direct replacment of class_to_type with > CacheBuilder weak keys/weak values/removal listener. Also we will > not attempt to expose it as a Map going forward; adjust > class_to_type usage accordingly. > > 2. Level 2 cache is weak keys/strong values/expires after write. We > can tune this expiration as was done with the regex cache and the > python.sre.cachespec option, > https://github.com/jythontools/jython/blob/master/src/org/python/modules/sre/SRE_STATE.java#L1253). > > 3. For removalListener: (C, T) entries that are removed from Level 1 > will use removalListener to get placed in Level 2 (safe because > Level 1 is concurrent; and obviously hard refs would prevent it > from being removed while under construction). > > 4. Try both caches when looking up the type T for class C. Because > synchronized, there's no visible inconsistency; if no entry for C > because it was removed, simply recompute T'. There is a small > window of time that it could be in the process of being moved to > Level 2, but this does not affect correctness; (C, T) will be > eventually expired from Level 2 and (C, T') is a valid repllacement > for it (because there can be no hard refs to T outside the level 2 > cache itself). > > Lastly, https://docs.oracle.com/javase/7/docs/api/java/lang/ClassValue.html > looks potentially useful as an alternative for publishing PyType > objects into a cache. But we need to address two key questions: > > 1. Whether ClassValue#computeValue would work properly in the > re-entrant case where the class graph from a given class C is > explored, as is done for inners, possibly referring back to C. This > is why we could not get the publication of the type in the map done > after the init (as one would usually want to do!) in > https://github.com/jythontools/jython/blob/master/src/org/python/core/PyType.java#L1515 > (putIfAbsent); class C would be referenced, and it would attempt to > build again (and stack overflow). No solution I tried could break > this problem. > > 2. When to call ClassValue#removeValue ! We still have to keep track > of the cache with respect to PyType. > Jim: I've read this a couple of times now and understand the intent, if not the detail. I'd like try ClassValue as a way to improve concurrency. I see what you mean about the needsInners list. I've watched that work and I'm still a bit puzzled what it does. It stores PyJavaType objects whose initialisation we couldn't complete because they depend on other PyType objects existing first? I write "existing" here because in the creation of T for some class C, it must be acceptable to have return a PyType the initialisation of which will be completed later, and only unacceptable to leave the monitor with T (or any PyType) incomplete. At any rate, that's what we do now, and I have half an idea how to reproduce it. The other thing I'm not sure about is when we would like a PyType to be collected. A ClassValue will be freed when the Class C is freed. And the fact that the value (PyType T) hard-refers to C will not prevent this. If something else hard-refers to T, then C won't be unloaded. I dare say JVMs try to avoid premature unloading of C, from which T benefits. The difficult case may be where C continues in use, but we'll never want T again. T would be elegible for collection except it is in the ClassValue for C. Is this a significant case? If it's common enough, then I think we need ClassValue<WeakReference<PyType>>, and if you want to defer that collection, a ReferenceQueue (not a thing I've used before) leading to a sort of recycling bin, your second cache. I'm pretty sure that could be thread safe, although it needs a secondary garbage collection thread to empty it. I'll try without this death-row cache, first. Jeff |
|
From: Jython t. <st...@bu...> - 2017-08-04 16:10:21
|
ACTIVITY SUMMARY (2017-07-28 - 2017-08-04) Jython tracker at http://bugs.jython.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 314 ( +0) closed 2318 ( +0) total 2632 ( +0) Open issues with patches: 28 Most recent 15 issues with no replies (15) ========================================== #2613: 2.7.1 MANIFEST.MF contains unfilled placeholders http://bugs.jython.org/issue2613 #2611: mkdir() operation in /Lib/os.py has different behavior when ru http://bugs.jython.org/issue2611 #2606: jython launch problem sun.misc.InvalidJarIndexException http://bugs.jython.org/issue2606 #2605: Jython 2.7.0 startup performance regression http://bugs.jython.org/issue2605 #2591: Unable to execute directory or zip file (test_cmd_line_script) http://bugs.jython.org/issue2591 #2567: System state lost during JSR-223 initialisation http://bugs.jython.org/issue2567 #2562: Windows: OSError: unlink(): an unknown error occurred http://bugs.jython.org/issue2562 #2531: Support SNI for SSL/TLS server sockets http://bugs.jython.org/issue2531 #2525: Jython incorrectly buffers file pipe output with Subprocess(co http://bugs.jython.org/issue2525 #2520: Jython does NOT support socket.listen(0) for accepting only on http://bugs.jython.org/issue2520 #2512: Values in built-in modules's __dict__ are â<reflected field http://bugs.jython.org/issue2512 #2510: TypeError when monkey-patching time.time with an unbound funct http://bugs.jython.org/issue2510 #2507: Run "invokeFunction" in a thread does not inherit the "ScriptC http://bugs.jython.org/issue2507 #2494: Support for pydoc_data http://bugs.jython.org/issue2494 #2484: Codec encodings can exhaust permgen http://bugs.jython.org/issue2484 Most recent 15 issues waiting for review (15) ============================================= #2566: inspect does not recognize code objects from bytecode files http://bugs.jython.org/issue2566 #2545: help() does not work on Java-implemented modules http://bugs.jython.org/issue2545 #2429: cStringIO does not work with mutable objects implementing the http://bugs.jython.org/issue2429 #2367: Jython ignores custom __eq__ when hashing dict subclasses http://bugs.jython.org/issue2367 #2330: full-build fails to copy CPython License http://bugs.jython.org/issue2330 #2230: Jython evaluation blocks under heavy load with high multi-core http://bugs.jython.org/issue2230 #2143: site-packages support in standalone jar http://bugs.jython.org/issue2143 #2142: Set Thread classloader when entering Jython context http://bugs.jython.org/issue2142 #2121: Jython jar on Maven central embeds other third party libraries http://bugs.jython.org/issue2121 #2077: marshal doesn't raise error when fed unmarshalable object http://bugs.jython.org/issue2077 #1925: Support loading java.sql.Drivers that aren't on the boot class http://bugs.jython.org/issue1925 #1917: No ctypes.c_char http://bugs.jython.org/issue1917 #1866: Parser does not have mismatch token error messages caught by B http://bugs.jython.org/issue1866 #1842: Add IBM i support to Jython http://bugs.jython.org/issue1842 #1796: Jython doesn't support jar dir with colon's in it http://bugs.jython.org/issue1796 Top 10 most discussed issues (2) ================================ #2609: strptime in Jython2.7.1 not working correctly in multithreaded http://bugs.jython.org/issue2609 11 msgs #2610: Unexpected program error: ImportError: Cannot import site modu http://bugs.jython.org/issue2610 4 msgs |
|
From: Jim B. <jim...@py...> - 2017-07-30 21:16:59
|
+jython-dev because of the seriousness of this issue Jeff, I increasingly believe this issue is almost certainly http://bugs.jython.org/issue2487 as Ivan pointed out earlier in this thread. Note that the synchronization in PyType.fromClass, which is responsible for constructing this map, was causing deadlocks was removed in 2.7.1b3 and replaced with a busy wait in 2.7.1rc1. I want to ask this question: *should we revisit our original decision to remove this synchronization*? (In https://hg.python.org/jython/rev/cffe5b824a21 and at least another followup commit set). It is possible that we got this mixed up with the low memory problem, and how it had a nasty action at a distance problem, see http://bugs.jython.org/issue2536 At the very least this is a low cost thing to try out, although it may not be conclusive. I should point out that I spent too much time trying to revisit how this publishes, and I never found a workable solution that didn't do the busy waiting that Darjus proposed as an alternative. Clearly we don't care about this code path being synchronized, only that it doesn't deadlock; it is not going to be hot except in some pathological usage scenario (constant reloading of Java classes through distinct class loaders, then used in some sort of callback scenario is the only thing that I can think of; otherwise this cost is only a startup cost). - Jim On Sun, Jul 30, 2017 at 3:35 AM, Jeff Allen <re...@bu...> wrote: > > Jeff Allen added the comment: > > Hmmm ... I'm not sure that actual concurrency (large numbers of parsers > working at the same time) has much to do with this. I run my test as: > > > while ../inst/bin/jython parse3a.py ; do echo "ok"; done > > which lets me fiddle with the source as it runs. It can be quite a wait. > The times I've caught this, the numbers were small. Here's one: > > REPEATS=2, THREADS=10, LENGTH=2. > barfoo > foofoo > foobar > barfoo > foofoo > barfoo > barfoo > barfoo > barfoo > barfoo > barfoo > barfoo > foofoo > foobar > foobar > barfoofoofoo > > barfoo > barfoo > *** Caught one! *** > Traceback (most recent call last): > File "parse3a.py", line 25, in parse > if m.groups(): short_print(m.group(1)) > AttributeError: 'org.python.modules.sre.MatchObject' object has no > attribute 'groups' > > /home/jeff/Jython/iss2609/work/parse3a.py(25)parse() > -> if m.groups(): short_print(m.group(1)) > (Pdb) p m > <org.python.modules.sre.MatchObject object at 0x2> > (Pdb) m.groups() > ('foobar', '52536') > (Pdb) p tests > ['barfoo84894', 'barfoo38345E', 'foofoo8017', 'barfoo42953', > 'foobar52536E', 'foobar18661', 'barfoo35695E', 'foofoo89916', > 'barfoo94921', 'barfoo32379'] > (Pdb) p i > 0 > > Now, as you can see, the object m behaved in the thread as if it had no > groups() method, but later in the debugger it is present, and it yields the > right result. This happens in the first block of threads. > > My theory is that use of this object by the thread is running ahead of > initialising the object or (quite likely) its class dictionary. I'll test > that, see if I can get an incriminating snapshot. > > ---------- > assignee: -> jeff.allen > components: +Core -Library > Added file: http://bugs.jython.org/file1594/parse3a.py > > _______________________________________ > Jython tracker <re...@bu...> > <http://bugs.jython.org/issue2609> > _______________________________________ > |
|
From: Jython t. <st...@bu...> - 2017-07-28 16:10:22
|
ACTIVITY SUMMARY (2017-07-21 - 2017-07-28) Jython tracker at http://bugs.jython.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 314 ( +2) closed 2318 ( +0) total 2632 ( +2) Open issues with patches: 28 Issues opened (2) ================= #2612: 2.7.1 NPE while trying to load class http://bugs.jython.org/issue2612 opened by jamesmudd #2613: 2.7.1 MANIFEST.MF contains unfilled placeholders http://bugs.jython.org/issue2613 opened by jamesmudd Most recent 15 issues with no replies (15) ========================================== #2613: 2.7.1 MANIFEST.MF contains unfilled placeholders http://bugs.jython.org/issue2613 #2611: mkdir() operation in /Lib/os.py has different behavior when ru http://bugs.jython.org/issue2611 #2606: jython launch problem sun.misc.InvalidJarIndexException http://bugs.jython.org/issue2606 #2605: Jython 2.7.0 startup performance regression http://bugs.jython.org/issue2605 #2591: Unable to execute directory or zip file (test_cmd_line_script) http://bugs.jython.org/issue2591 #2567: System state lost during JSR-223 initialisation http://bugs.jython.org/issue2567 #2562: Windows: OSError: unlink(): an unknown error occurred http://bugs.jython.org/issue2562 #2531: Support SNI for SSL/TLS server sockets http://bugs.jython.org/issue2531 #2525: Jython incorrectly buffers file pipe output with Subprocess(co http://bugs.jython.org/issue2525 #2520: Jython does NOT support socket.listen(0) for accepting only on http://bugs.jython.org/issue2520 #2512: Values in built-in modules's __dict__ are â<reflected field http://bugs.jython.org/issue2512 #2510: TypeError when monkey-patching time.time with an unbound funct http://bugs.jython.org/issue2510 #2507: Run "invokeFunction" in a thread does not inherit the "ScriptC http://bugs.jython.org/issue2507 #2494: Support for pydoc_data http://bugs.jython.org/issue2494 #2484: Codec encodings can exhaust permgen http://bugs.jython.org/issue2484 Most recent 15 issues waiting for review (15) ============================================= #2566: inspect does not recognize code objects from bytecode files http://bugs.jython.org/issue2566 #2545: help() does not work on Java-implemented modules http://bugs.jython.org/issue2545 #2429: cStringIO does not work with mutable objects implementing the http://bugs.jython.org/issue2429 #2367: Jython ignores custom __eq__ when hashing dict subclasses http://bugs.jython.org/issue2367 #2330: full-build fails to copy CPython License http://bugs.jython.org/issue2330 #2230: Jython evaluation blocks under heavy load with high multi-core http://bugs.jython.org/issue2230 #2143: site-packages support in standalone jar http://bugs.jython.org/issue2143 #2142: Set Thread classloader when entering Jython context http://bugs.jython.org/issue2142 #2121: Jython jar on Maven central embeds other third party libraries http://bugs.jython.org/issue2121 #2077: marshal doesn't raise error when fed unmarshalable object http://bugs.jython.org/issue2077 #1925: Support loading java.sql.Drivers that aren't on the boot class http://bugs.jython.org/issue1925 #1917: No ctypes.c_char http://bugs.jython.org/issue1917 #1866: Parser does not have mismatch token error messages caught by B http://bugs.jython.org/issue1866 #1842: Add IBM i support to Jython http://bugs.jython.org/issue1842 #1796: Jython doesn't support jar dir with colon's in it http://bugs.jython.org/issue1796 Top 10 most discussed issues (1) ================================ #2609: strptime in Jython2.7.1 not working correctly in multithreaded http://bugs.jython.org/issue2609 10 msgs |
|
From: Jeff A. <ja...@fa...> - 2017-07-23 16:54:42
|
Working with Ivan's demonstration, and a more stressful version (see tickethttp://bugs.jython.org/issue2609), I was unable to reproduce the problem on my main dev box (Windows 10). I'm thinking hardware multiprocessing might be a factor. Anyone with lots of CPUs (and a different OS) like to take this on? Jeff On 22/07/2017 09:01, Ivan Ponomarev via Jython-dev wrote: > Hello all! > > We are using jython heavily from Java application in multiple threads. > This didn't cause any problems in 2.7.1b3 and earlier versions. > > After upgrading to 2.7.1 we faced the following: a simple call to > datetime.strptime(...) or json.loads(...) can lead to > "'org.python.modules.sre.MatchObject' object has no attribute..." error. > > As I suspect, this is related to the fact that RegexObject is not > thread-safe anymore. It never was in Java, and never promised to be in > Python, though in earlier versions of Jython this was possible: > > ############################################## > import thread > import re > import time > REGEX = re.compile('((?:foo|bar)+)(\d*)E?') > > def parse(line): > m = REGEX.search(line) > if m.groups(): > print m.group(1) > > thread.start_new_thread(parse,('foofoofoofoofoofoofoofoofoo234',)) > thread.start_new_thread(parse, ('foobarbarbarbarbarbarbarfoo4E',)) > .... > thread.start_new_thread(parse, ('foobarbarbarbarbarbarbarfoo4E',)) > time.sleep(1) > ############################################### > > In Jython 2.7.1 this (sometimes! you might need to run this script a > couple of times) cause "'MatchObject' object has no attribute" errors. > > Instead of calling shared RegexObject one can simply do this: > > def printtime(time): > print datetime.datetime.strptime(time, '%Y-%m-%dT%H:%M:%S') > thread.start_new_thread(printtime, ('2017-07-01T11:11:11',)) > ... > thread.start_new_thread(printtime, ('2017-07-01T11:11:11',)) > > > The same with json.loads. > > OK, we never share RegexObjects between threads in our code. But we > cannot refuse to use json.loads and datetime.strptime! > > This became a major issue for us (http://bugs.jython.org/issue2609). > What would you suggest? How can we help to solve this? > > Regards, > > Ivan Ponomarev > > ------------------------------------------------------------------------------ > > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > |
|
From: Ivan P. <ipo...@ma...> - 2017-07-22 08:01:48
|
Hello all!
We are using jython heavily from Java application in multiple threads.
This didn't cause any problems in 2.7.1b3 and earlier versions.
After upgrading to 2.7.1 we faced the following: a simple call to
datetime.strptime(...) or json.loads(...) can lead to
"'org.python.modules.sre.MatchObject' object has no attribute..." error.
As I suspect, this is related to the fact that RegexObject is not
thread-safe anymore. It never was in Java, and never promised to be in
Python, though in earlier versions of Jython this was possible:
##############################################
import thread
import re
import time
REGEX = re.compile('((?:foo|bar)+)(\d*)E?')
def parse(line):
m = REGEX.search(line)
if m.groups():
print m.group(1)
thread.start_new_thread(parse,('foofoofoofoofoofoofoofoofoo234',))
thread.start_new_thread(parse, ('foobarbarbarbarbarbarbarfoo4E',))
....
thread.start_new_thread(parse, ('foobarbarbarbarbarbarbarfoo4E',))
time.sleep(1)
###############################################
In Jython 2.7.1 this (sometimes! you might need to run this script a
couple of times) cause "'MatchObject' object has no attribute" errors.
Instead of calling shared RegexObject one can simply do this:
def printtime(time):
print datetime.datetime.strptime(time, '%Y-%m-%dT%H:%M:%S')
thread.start_new_thread(printtime, ('2017-07-01T11:11:11',))
...
thread.start_new_thread(printtime, ('2017-07-01T11:11:11',))
The same with json.loads.
OK, we never share RegexObjects between threads in our code. But we
cannot refuse to use json.loads and datetime.strptime!
This became a major issue for us (http://bugs.jython.org/issue2609).
What would you suggest? How can we help to solve this?
Regards,
Ivan Ponomarev
|
|
From: Jython t. <st...@bu...> - 2017-07-21 16:10:23
|
ACTIVITY SUMMARY (2017-07-14 - 2017-07-21) Jython tracker at http://bugs.jython.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 312 ( +6) closed 2318 ( +0) total 2630 ( +6) Open issues with patches: 28 Issues opened (6) ================= #2606: jython launch problem sun.misc.InvalidJarIndexException http://bugs.jython.org/issue2606 opened by rafpar #2607: Jython launcher failure (Windows error 14001) http://bugs.jython.org/issue2607 opened by jeff.allen #2608: Encoding problem in os.uname with non-ascii host name http://bugs.jython.org/issue2608 opened by jeff.allen #2609: 'MatchObject object has no attribute 'end'' when calling strpt http://bugs.jython.org/issue2609 opened by Ivan #2610: Unexpected program error: ImportError: Cannot import site modu http://bugs.jython.org/issue2610 opened by rkanumola #2611: mkdir() operation in /Lib/os.py has different behavior when ru http://bugs.jython.org/issue2611 opened by jochenhebbrecht Most recent 15 issues with no replies (15) ========================================== #2611: mkdir() operation in /Lib/os.py has different behavior when ru http://bugs.jython.org/issue2611 #2609: 'MatchObject object has no attribute 'end'' when calling strpt http://bugs.jython.org/issue2609 #2607: Jython launcher failure (Windows error 14001) http://bugs.jython.org/issue2607 #2606: jython launch problem sun.misc.InvalidJarIndexException http://bugs.jython.org/issue2606 #2605: Jython 2.7.0 startup performance regression http://bugs.jython.org/issue2605 #2591: Unable to execute directory or zip file (test_cmd_line_script) http://bugs.jython.org/issue2591 #2567: System state lost during JSR-223 initialisation http://bugs.jython.org/issue2567 #2562: Windows: OSError: unlink(): an unknown error occurred http://bugs.jython.org/issue2562 #2531: Support SNI for SSL/TLS server sockets http://bugs.jython.org/issue2531 #2525: Jython incorrectly buffers file pipe output with Subprocess(co http://bugs.jython.org/issue2525 #2520: Jython does NOT support socket.listen(0) for accepting only on http://bugs.jython.org/issue2520 #2512: Values in built-in modules's __dict__ are â<reflected field http://bugs.jython.org/issue2512 #2510: TypeError when monkey-patching time.time with an unbound funct http://bugs.jython.org/issue2510 #2507: Run "invokeFunction" in a thread does not inherit the "ScriptC http://bugs.jython.org/issue2507 #2494: Support for pydoc_data http://bugs.jython.org/issue2494 Most recent 15 issues waiting for review (15) ============================================= #2566: inspect does not recognize code objects from bytecode files http://bugs.jython.org/issue2566 #2545: help() does not work on Java-implemented modules http://bugs.jython.org/issue2545 #2429: cStringIO does not work with mutable objects implementing the http://bugs.jython.org/issue2429 #2367: Jython ignores custom __eq__ when hashing dict subclasses http://bugs.jython.org/issue2367 #2330: full-build fails to copy CPython License http://bugs.jython.org/issue2330 #2230: Jython evaluation blocks under heavy load with high multi-core http://bugs.jython.org/issue2230 #2143: site-packages support in standalone jar http://bugs.jython.org/issue2143 #2142: Set Thread classloader when entering Jython context http://bugs.jython.org/issue2142 #2121: Jython jar on Maven central embeds other third party libraries http://bugs.jython.org/issue2121 #2077: marshal doesn't raise error when fed unmarshalable object http://bugs.jython.org/issue2077 #1925: Support loading java.sql.Drivers that aren't on the boot class http://bugs.jython.org/issue1925 #1917: No ctypes.c_char http://bugs.jython.org/issue1917 #1866: Parser does not have mismatch token error messages caught by B http://bugs.jython.org/issue1866 #1842: Add IBM i support to Jython http://bugs.jython.org/issue1842 #1796: Jython doesn't support jar dir with colon's in it http://bugs.jython.org/issue1796 Top 10 most discussed issues (1) ================================ #2610: Unexpected program error: ImportError: Cannot import site modu http://bugs.jython.org/issue2610 6 msgs |
|
From: Jeff A. <ja...@fa...> - 2017-07-18 07:10:07
|
I don't know if Frank improved anything on Maven Central or his blog, but although Maven Central still reports our jars as 0 bytes, I haven't in practice had any trouble downloading from there. This also came up in https://github.com/jythontools/jython/issues/84 and it seems to have worked for the user in https://github.com/jythontools/jython/issues/83 . Jeff On 12/07/2017 00:02, fwi...@gm... wrote: > On Tue, Jul 11, 2017 at 2:04 PM, Stefan Richthofer > <Ste...@gm...> wrote: >> Note that the link from Franks blogpost to maven seems to be broken. I was not able to find Jython 2.7.1 on Maven Central. 2.7.0 is there. >> > Really? That's no good. They all work for me, but I know that isn't > necessarily enough. Which link and where does it take you? > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > |
|
From: Rory O'D. <ror...@or...> - 2017-07-17 12:07:29
|
Hi Alan, *JDK 9 Early Access* build 178 is available at : - jdk.java.net/9/ A summary of all the changes in this build are listed here <http://download.java.net/java/jdk9/changes/jdk-9+178.html>. Changes which were introduced since the last availability email that may be of interest : * b175 - Module system implementation refresh**(6/2017 update) * b175 - no longer has "-ea" in the version string and the system property "java version" is now simply "9" o *java -version* >java version "9" >Java(TM) SE Runtime Environment (build 9+175) >Java HotSpot(TM) 64-Bit Server VM (build 9+175, mixed mode) o *Bundle name changes:* e.g. jdk-9+175_linux-x86_bin.tar.gz *JDK 8u152 Early Access* build 05 is available at : - jdk.java.net/8/ <http://jdk.java.net/8/> A summary of all the changes in this build are listed here <http://download.java.net/java/jdk8u152/changes/jdk8u152-b05.html>. Rgds,Rory -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA , Dublin, Ireland |
|
From: Jython t. <st...@bu...> - 2017-07-14 16:10:23
|
ACTIVITY SUMMARY (2017-07-07 - 2017-07-14) Jython tracker at http://bugs.jython.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 306 ( +0) closed 2318 ( +0) total 2624 ( +0) Open issues with patches: 28 Most recent 15 issues with no replies (15) ========================================== #2605: Jython 2.7.0 startup performance regression http://bugs.jython.org/issue2605 #2591: Unable to execute directory or zip file (test_cmd_line_script) http://bugs.jython.org/issue2591 #2567: System state lost during JSR-223 initialisation http://bugs.jython.org/issue2567 #2562: Windows: OSError: unlink(): an unknown error occurred http://bugs.jython.org/issue2562 #2531: Support SNI for SSL/TLS server sockets http://bugs.jython.org/issue2531 #2525: Jython incorrectly buffers file pipe output with Subprocess(co http://bugs.jython.org/issue2525 #2520: Jython does NOT support socket.listen(0) for accepting only on http://bugs.jython.org/issue2520 #2512: Values in built-in modules's __dict__ are â<reflected field http://bugs.jython.org/issue2512 #2510: TypeError when monkey-patching time.time with an unbound funct http://bugs.jython.org/issue2510 #2507: Run "invokeFunction" in a thread does not inherit the "ScriptC http://bugs.jython.org/issue2507 #2494: Support for pydoc_data http://bugs.jython.org/issue2494 #2484: Codec encodings can exhaust permgen http://bugs.jython.org/issue2484 #2482: Publish U.S. ECCN for Jython http://bugs.jython.org/issue2482 #2422: test_classpathimporter fauls on Linux http://bugs.jython.org/issue2422 #2418: test_chdir subprocess tests fail on Windows http://bugs.jython.org/issue2418 Most recent 15 issues waiting for review (15) ============================================= #2566: inspect does not recognize code objects from bytecode files http://bugs.jython.org/issue2566 #2545: help() does not work on Java-implemented modules http://bugs.jython.org/issue2545 #2429: cStringIO does not work with mutable objects implementing the http://bugs.jython.org/issue2429 #2367: Jython ignores custom __eq__ when hashing dict subclasses http://bugs.jython.org/issue2367 #2330: full-build fails to copy CPython License http://bugs.jython.org/issue2330 #2230: Jython evaluation blocks under heavy load with high multi-core http://bugs.jython.org/issue2230 #2143: site-packages support in standalone jar http://bugs.jython.org/issue2143 #2142: Set Thread classloader when entering Jython context http://bugs.jython.org/issue2142 #2121: Jython jar on Maven central embeds other third party libraries http://bugs.jython.org/issue2121 #2077: marshal doesn't raise error when fed unmarshalable object http://bugs.jython.org/issue2077 #1925: Support loading java.sql.Drivers that aren't on the boot class http://bugs.jython.org/issue1925 #1917: No ctypes.c_char http://bugs.jython.org/issue1917 #1866: Parser does not have mismatch token error messages caught by B http://bugs.jython.org/issue1866 #1842: Add IBM i support to Jython http://bugs.jython.org/issue1842 #1796: Jython doesn't support jar dir with colon's in it http://bugs.jython.org/issue1796 |
101 messages has been excluded from this view by a project administrator.
