Re: [Jython-users] Using Jython in separate threads

[Oti <oh...@gm...>]

Hi,

I'd like to inform you that 2.2rc1 is out
(http://www.jython.org/Project/index.html), and that this bug is fixed
within. It was the parser not being threadsafe. My users cannot
reproduce the strange effects any more.

best wishes,
Oti.

On 6/12/07, Oti <oh...@gm...> wrote:
> Hi all,
>
> to have a single place of resources, I opened bug 1735864
> https://sourceforge.net/tracker/index.php?func=detail&aid=1735864&group_id=12867&atid=112867
> and will attach the current test program and the results tomorrow.
>
> best wishes,
> Oti.
>
> On 6/11/07, Oti <oh...@gm...> wrote:
> > Andy (or Michael)
> >
> > Hopefully you don't mind if I change to jython-dev on this thread.
> >
> > I can reproduce it on my at-work box. My environment is:
> > - Pentium 4 HT
> > - Ubuntu 6.10
> > (see the attached cpuinfo.txt)
> >
> > huo@bisonws0135:~$ java -version
> > java version "1.5.0_08"
> > Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> > Java HotSpot(TM) Client VM (build 1.5.0_08-b03, mixed mode, sharing)
> >
> >
> > From this it should be clear that we cannot blame windows.
> >
> > I think I can explain the 'seems to hang forever' effect:
> > If you get an error on the
> >   Test.n += 1
> > line of test.py, the public static variable n of the Test class is not
> > incremented. Your main thread will then yield (=pause) forever.
> > Introducing a hasAliveThreads() method solved the problem.
> > (see attached Test.java and test.py)
> > Strangely, the error you get in this case is a verbatim
> >   'no module named n'
> > error. This means that Test.n is not known at this point.
> >
> > I could not reproduce the 'memory seems to balloon' effect. Even with
> > 500 threads, memory only increases a little, while CPU is >= 90% for a
> > few seconds.
> >
> > The magic number of threads in my case is 12.
> > With 12 threads, I have >90% probability of errors,
> > with <= 11 threads it decreases rapidly.
> > But I also got errors with 5 threads, rarely though.
> >
> > That's the first impression, now I'll have to dive in deeper.
> > Hints are welcome !
> >
> > Best wishes,
> > Oti.
> >
> > On 5/22/07, Michael Chisholm <chi...@mi...> wrote:
> > > It's been a few months; other higher-priority tasks got in the way of
> > > investigating this further.  But it is again moving up the priority
> > > queue...
> > >
> > > Attached is the code we came up with the demonstrate the error.  We have
> > > been using XP and Jython 2.2b2 to show the error.  Unfortunately, it is
> > > rather difficult to reproduce on single-core computers.  This code
> > > simply creates a bunch of threads that instantiate their own
> > > PythonInterpreter object and run the same simple python script.  The
> > > script does a lot of importing, to try to produce an import error.
> > > Output goes to a file, since we found that writing to stdout was
> > > error-prone.  Sometimes the output just stops working.  E.g. with 10
> > > threads, where the script prints to stdout, you might see only 3 or 4
> > > prints.  But this is of secondary importance to us.
> > >
> > > The test program is set to create 500 threads (there's a variable at the
> > > top you can change), and that doesn't even reliably produce it.  If you
> > > are watching the Task Manager's process list when you run it, sometimes
> > > the program will seem to hang, and memory usage will balloon...
> > > sometimes it seems to hang without ballooning memory usage.  Sometimes
> > > the program finishes successfully within seconds.  But the error that is
> > > hurting us is an import error.  If you check the output file (out.txt)
> > > after running the program, sometimes you will see something like the
> > > following:
> > >
> > > Traceback (innermost last):
> > >   File "<string>", line 6, in ?
> > > ImportError: no module named n
> > >
> > > The module mentioned varies.  I am told that on one of our high end
> > > multi-core machines (which I don't have access to; we are a
> > > geographically distributed team), this import error occurs reliably with
> > > only 3 or 4 threads.  And that's what kills us.  Our back-end may not
> > > need to spawn 500 threads, but 3-4 is routine.  If you have any
> > > multi-core machines to test with, that would be ideal.
> > >
> > > I don't suppose fixing this is easy... but it will be interesting to at
> > > least see whether others can reproduce the error and acknowledge that it
> > > exists.  Or maybe find a more reliable way to produce it.
> > >
> > > Andy
> > >
> > > Charlie Groves wrote:
> > > > It looks like the setOut and setErr problems in that message have been
> > > > fixed in the current code, but I'm not sure if it's fixed in the
> > > > alpha.  Why are you using the alpha instead of the beta?  I know I've
> > > > fixed a few other threading problems that were present in the alpha.
> > > >
> > > > In any case, there are a number of threading problems mentioned in
> > > > http://jython.org/patches/898709  that still exist as far as I know.
> > > > That information needs review and integration with the current code,
> > > > which is outside of the scope of the work I'm planning for the 2.2
> > > > release.  However, if you can provide more information about the
> > > > threading errors you're seeing or a way to reproduce them, that might
> > > > get some work on this moved into 2.2.  I definitely want Jython to
> > > > have the ability to use multiple interpreters in multiple threads;
> > > > it's a question of how broken it is and how much work it would take to
> > > > fix it that determines when it'll get attention.
> > > >
> > > > Charlie
> > > >
> > > > On 2/20/07, Michael Chisholm <chi...@mi...> wrote:
> > > >> At http://sourceforge.net/mailarchive/message.php?msg_id=705531 is an
> > > >> old message stating that multiple instances of the Jython interpreter
> > > >> may not be usable from different threads, due to static data shared
> > > >> among the instances.  We have a system which uses Jython from multiple
> > > >> threads (one instance per thread) and are seeing significant problems,
> > > >> which resolve themselves when we remove the multithreading.  We are
> > > >> trying to understand the root cause of these issues.
> > > >>
> > > >> Are the issues mentioned in the above link still relevant for the 2.2
> > > >> alpha version?  (We still must use the alpha for reasons which I
> > > >> sincerely hope will resolve themselves once 2.2 final is out.)
> > > >>
> > > >> Andy
> > > >>
> > > >>
> > > >>
> > > >> -------------------------------------------------------------------------
> > > >>
> > > >> Take Surveys. Earn Cash. Influence the Future of IT
> > > >> Join SourceForge.net's Techsay panel and you'll get the chance to
> > > >> share your
> > > >> opinions on IT & business topics through brief surveys-and earn cash
> > > >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > > >>
> > > >> _______________________________________________
> > > >> Jython-users mailing list
> > > >> Jyt...@li...
> > > >> https://lists.sourceforge.net/lists/listinfo/jython-users
> > > >>
> > > >
> > >
> > >
> > > import os
> > > import shutil
> > > import glob
> > > import sys
> > > import socket
> > > import time
> > > import math
> > > import urllib
> > > import popen2
> > > import re
> > > import types
> > > from xml.sax.saxutils import escape
> > > import Test
> > > from java.util import List
> > > from java.io import File
> > > from javax.swing import JButton
> > > from java.awt import Dimension
> > > from java.util import Date
> > > from java.lang import Float
> > > from org.w3c.dom import Document
> > >
> > > #print "Running test"
> > > Test.n += 1
> > >
> > > -------------------------------------------------------------------------
> > > This SF.net email is sponsored by DB2 Express
> > > Download DB2 Express C - the FREE version of DB2 express and take
> > > control of your XML. No limits. Just data. Click to get it now.
> > > http://sourceforge.net/powerbar/db2/
> > > _______________________________________________
> > > Jython-users mailing list
> > > Jyt...@li...
> > > https://lists.sourceforge.net/lists/listinfo/jython-users
> > >
> > >
> > >
> >
> >
>