On Tue, 2003-03-04 at 23:30, Edmund Lian wrote:
> I've got a periodic task that can be autostarted when WebKit starts, via a
> call to contextInitialize(), or it can be started manually via a web page
> being served by WebKit.
> When the task is autostarted on WebKit launch, WebKit sometimes slow down
> to a dead crawl, as if the task is consuming all the CPU cycles. When I
> turn off the autostart, and start the task manually from a web page,
> everything seems to be fine. I can't reproduce the problem consistently,
> but what I see so far suggests that there is some kind of time-sensitivity
> involved, meaning that if the task is kicked of at the wrong moment, it
> causes some wierd stall in WebKit.
Does the thread spin endlessly? This sounds like it _could_ be the same
problem I ran into about a month ago, which had to do with concurrent
I probably should have posted this to the list at that time, but
anyways... this is what I figured out:
1. Webware uses an import hook so that it can customize how servlets are
imported (see WebKit/ServletFactory.py) which ultimately relies on the
special __import__ function.
2. The python interpreter has a global lock which it uses to prevent
concurrent imports from happening, but
3. Calling __import__ does not make use the global lock (it can't, since
a user-defined __import__ could be purely python code), and furthermore
4. The lock is not exposed/available to python code, so custom import
hooks can't make use of it.
Conclusion: any multithreaded Python app which uses an import hook faces
the possibility of concurrent imports occuring.
Luckily there is light at the end of the tunnel. I also discovered that
a bug had been filed against python to expose the lock/unlock functions:
After a couple hours of hacking/testing, I produced a patch to expose
the global lock, and confirmed that by using the exposed it, I could no
longer reproduce the problem. I sent in the patch and whaddya know?
Guido himself merged it in. I even convinced him to apply it to the
2.2.x maintenance branch, so it should be in 2.2.3 when it comes out.
Note: I need to tweak the Webware source to check for the lock and use
it if available. We can do this already, so that the code will take
advantage of the lock when running on newer Python versions.
To mitigate the problem in the meantime, I'd suggest adding
from time import sleep
to your periodic task (to delay its execution by a few seconds), and see
if that helps.
In my experience, this bug is most likely to occur during or shortly
after an appserver restart. This makes sense, since the likelihood of
concurrent imports occuring go down as more and more of the modules get
loaded into RAM.
> Can anybody suggest a way for me to isolate the problem? I suppose that if
> it is a threading issue (in WebKit--I'm not using threads), then it's going
> to be impossible to pin down...?
When I started trying to figure this out, I found this useful page:
which describes how to use GDB to obtain a python traceback from a
spinning thread -- very useful! The resulting traceback was really
weird; it looked like some method call on an object ended up executing a
method of a completely unrelated object. That clued me in that
something really bad was happening.
Jason D. Hildebrand