Re: [Jython-dev] [issue2609] strptime in Jython2.7.1 not working correctly in multithreaded environment

[Jim B. <jim...@py...>]

+jython-dev because of the seriousness of this issue

Jeff,

I increasingly believe this issue is almost certainly
http://bugs.jython.org/issue2487 as Ivan pointed out earlier in this
thread. Note that the synchronization in PyType.fromClass, which is
responsible for constructing this map, was causing deadlocks was removed in
2.7.1b3 and replaced with a busy wait in 2.7.1rc1.

I want to ask this question: *should we revisit our original decision to
remove this synchronization*? (In
https://hg.python.org/jython/rev/cffe5b824a21 and at least another followup
commit set). It is possible that we got this mixed up with the low memory
problem, and how it had a nasty action at a distance problem, see
http://bugs.jython.org/issue2536

At the very least this is a low cost thing to try out, although it may not
be conclusive.

I should point out that I spent too much time trying to revisit how this
publishes, and I never found a workable solution that didn't do the busy
waiting that Darjus proposed as an alternative. Clearly we don't care about
this code path being synchronized, only that it doesn't deadlock; it is not
going to be hot except in some pathological usage scenario (constant
reloading of Java classes through distinct class loaders, then used in some
sort of callback scenario is the only thing that I can think of; otherwise
this cost is only a startup cost).

- Jim


On Sun, Jul 30, 2017 at 3:35 AM, Jeff Allen <re...@bu...> wrote:

>
> Jeff Allen added the comment:
>
> Hmmm ... I'm not sure that actual concurrency (large numbers of parsers
> working at the same time) has much to do with this. I run my test as:
>
> > while ../inst/bin/jython parse3a.py ; do echo "ok"; done
>
> which lets me fiddle with the source as it runs. It can be quite a wait.
> The times I've caught this, the numbers were small. Here's one:
>
> REPEATS=2, THREADS=10,  LENGTH=2.
> barfoo
> foofoo
> foobar
> barfoo
> foofoo
> barfoo
> barfoo
> barfoo
> barfoo
> barfoo
> barfoo
> barfoo
> foofoo
> foobar
> foobar
> barfoofoofoo
>
> barfoo
> barfoo
>  *** Caught one! ***
> Traceback (most recent call last):
>   File "parse3a.py", line 25, in parse
>     if m.groups(): short_print(m.group(1))
> AttributeError: 'org.python.modules.sre.MatchObject' object has no
> attribute 'groups'
> > /home/jeff/Jython/iss2609/work/parse3a.py(25)parse()
> -> if m.groups(): short_print(m.group(1))
> (Pdb) p m
> <org.python.modules.sre.MatchObject object at 0x2>
> (Pdb) m.groups()
> ('foobar', '52536')
> (Pdb) p tests
> ['barfoo84894', 'barfoo38345E', 'foofoo8017', 'barfoo42953',
> 'foobar52536E', 'foobar18661', 'barfoo35695E', 'foofoo89916',
> 'barfoo94921', 'barfoo32379']
> (Pdb) p i
> 0
>
> Now, as you can see, the object m behaved in the thread as if it had no
> groups() method, but later in the debugger it is present, and it yields the
> right result. This happens in the first block of threads.
>
> My theory is that use of this object by the thread is running ahead of
> initialising the object or (quite likely) its class dictionary. I'll test
> that, see if I can get an incriminating snapshot.
>
> ----------
> assignee:  -> jeff.allen
> components: +Core -Library
> Added file: http://bugs.jython.org/file1594/parse3a.py
>
> _______________________________________
> Jython tracker <re...@bu...>
> <http://bugs.jython.org/issue2609>
> _______________________________________
>