From: Oti <oh...@gm...> - 2007-06-12 16:22:23
|
Hi all, to have a single place of resources, I opened bug 1735864 https://sourceforge.net/tracker/index.php?func=detail&aid=1735864&group_id=12867&atid=112867 and will attach the current test program and the results tomorrow. best wishes, Oti. On 6/11/07, Oti <oh...@gm...> wrote: > Andy (or Michael) > > Hopefully you don't mind if I change to jython-dev on this thread. > > I can reproduce it on my at-work box. My environment is: > - Pentium 4 HT > - Ubuntu 6.10 > (see the attached cpuinfo.txt) > > huo@bisonws0135:~$ java -version > java version "1.5.0_08" > Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03) > Java HotSpot(TM) Client VM (build 1.5.0_08-b03, mixed mode, sharing) > > > From this it should be clear that we cannot blame windows. > > I think I can explain the 'seems to hang forever' effect: > If you get an error on the > Test.n += 1 > line of test.py, the public static variable n of the Test class is not > incremented. Your main thread will then yield (=pause) forever. > Introducing a hasAliveThreads() method solved the problem. > (see attached Test.java and test.py) > Strangely, the error you get in this case is a verbatim > 'no module named n' > error. This means that Test.n is not known at this point. > > I could not reproduce the 'memory seems to balloon' effect. Even with > 500 threads, memory only increases a little, while CPU is >= 90% for a > few seconds. > > The magic number of threads in my case is 12. > With 12 threads, I have >90% probability of errors, > with <= 11 threads it decreases rapidly. > But I also got errors with 5 threads, rarely though. > > That's the first impression, now I'll have to dive in deeper. > Hints are welcome ! > > Best wishes, > Oti. > > On 5/22/07, Michael Chisholm <chi...@mi...> wrote: > > It's been a few months; other higher-priority tasks got in the way of > > investigating this further. But it is again moving up the priority > > queue... > > > > Attached is the code we came up with the demonstrate the error. We have > > been using XP and Jython 2.2b2 to show the error. Unfortunately, it is > > rather difficult to reproduce on single-core computers. This code > > simply creates a bunch of threads that instantiate their own > > PythonInterpreter object and run the same simple python script. The > > script does a lot of importing, to try to produce an import error. > > Output goes to a file, since we found that writing to stdout was > > error-prone. Sometimes the output just stops working. E.g. with 10 > > threads, where the script prints to stdout, you might see only 3 or 4 > > prints. But this is of secondary importance to us. > > > > The test program is set to create 500 threads (there's a variable at the > > top you can change), and that doesn't even reliably produce it. If you > > are watching the Task Manager's process list when you run it, sometimes > > the program will seem to hang, and memory usage will balloon... > > sometimes it seems to hang without ballooning memory usage. Sometimes > > the program finishes successfully within seconds. But the error that is > > hurting us is an import error. If you check the output file (out.txt) > > after running the program, sometimes you will see something like the > > following: > > > > Traceback (innermost last): > > File "<string>", line 6, in ? > > ImportError: no module named n > > > > The module mentioned varies. I am told that on one of our high end > > multi-core machines (which I don't have access to; we are a > > geographically distributed team), this import error occurs reliably with > > only 3 or 4 threads. And that's what kills us. Our back-end may not > > need to spawn 500 threads, but 3-4 is routine. If you have any > > multi-core machines to test with, that would be ideal. > > > > I don't suppose fixing this is easy... but it will be interesting to at > > least see whether others can reproduce the error and acknowledge that it > > exists. Or maybe find a more reliable way to produce it. > > > > Andy > > > > Charlie Groves wrote: > > > It looks like the setOut and setErr problems in that message have been > > > fixed in the current code, but I'm not sure if it's fixed in the > > > alpha. Why are you using the alpha instead of the beta? I know I've > > > fixed a few other threading problems that were present in the alpha. > > > > > > In any case, there are a number of threading problems mentioned in > > > http://jython.org/patches/898709 that still exist as far as I know. > > > That information needs review and integration with the current code, > > > which is outside of the scope of the work I'm planning for the 2.2 > > > release. However, if you can provide more information about the > > > threading errors you're seeing or a way to reproduce them, that might > > > get some work on this moved into 2.2. I definitely want Jython to > > > have the ability to use multiple interpreters in multiple threads; > > > it's a question of how broken it is and how much work it would take to > > > fix it that determines when it'll get attention. > > > > > > Charlie > > > > > > On 2/20/07, Michael Chisholm <chi...@mi...> wrote: > > >> At http://sourceforge.net/mailarchive/message.php?msg_id=705531 is an > > >> old message stating that multiple instances of the Jython interpreter > > >> may not be usable from different threads, due to static data shared > > >> among the instances. We have a system which uses Jython from multiple > > >> threads (one instance per thread) and are seeing significant problems, > > >> which resolve themselves when we remove the multithreading. We are > > >> trying to understand the root cause of these issues. > > >> > > >> Are the issues mentioned in the above link still relevant for the 2.2 > > >> alpha version? (We still must use the alpha for reasons which I > > >> sincerely hope will resolve themselves once 2.2 final is out.) > > >> > > >> Andy > > >> > > >> > > >> > > >> ------------------------------------------------------------------------- > > >> > > >> Take Surveys. Earn Cash. Influence the Future of IT > > >> Join SourceForge.net's Techsay panel and you'll get the chance to > > >> share your > > >> opinions on IT & business topics through brief surveys-and earn cash > > >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > >> > > >> _______________________________________________ > > >> Jython-users mailing list > > >> Jyt...@li... > > >> https://lists.sourceforge.net/lists/listinfo/jython-users > > >> > > > > > > > > > import os > > import shutil > > import glob > > import sys > > import socket > > import time > > import math > > import urllib > > import popen2 > > import re > > import types > > from xml.sax.saxutils import escape > > import Test > > from java.util import List > > from java.io import File > > from javax.swing import JButton > > from java.awt import Dimension > > from java.util import Date > > from java.lang import Float > > from org.w3c.dom import Document > > > > #print "Running test" > > Test.n += 1 > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by DB2 Express > > Download DB2 Express C - the FREE version of DB2 express and take > > control of your XML. No limits. Just data. Click to get it now. > > http://sourceforge.net/powerbar/db2/ > > _______________________________________________ > > Jython-users mailing list > > Jyt...@li... > > https://lists.sourceforge.net/lists/listinfo/jython-users > > > > > > > > |