That's a rant, it's off topic, and I'm probably posting on the wrong forum, but anyway.

I feel your pain, I've spent a good few hours this week debugging threaded programs. At one point I found that I was using the same telnet instance on two threads at the same time. No wonder things were messed up. But it took me two hours to find the root of the problem.

But the fact is that today you have to deal with parallelism in one form or another. It can be threads or multiple processes, it's unavoidable. And it will get worse with newer CPUs.

Your argument reminds me of a recent post by Guido van Rossum. If I remember it right it was one of the posts regarding Python 3k, someone asked if there were any plans to remove the GIL. Guido said that the someone wrote a patch to remove the GIL. It was very complex and the performance was poor, because of the number of extra checks. In the end it became impossible to keep the patch synchronized with the development on the main Python trunk. Given that and given Guido's opinion regarding threads, the GIL continues to be what it is - a single generic lock for Python instead of a more granular locking mechanism, which would be better for multithreaded programs (at the cost of complexity).

Also, note that Google AppEngine uses short lived processes instead of threads. It's a different philosophy, and it permeates everything in Google's design.

The hidden assumption is that it's better to scale an application with multiple processes than using threads. But performance may suffer, due to the cost of the synchronization that becomes necessary (note that locking is needed in both cases). Libraries such as SQLObject that keep data cached in memory are heavily affected. There's a potential performance advantage if you use threads with SQLObject because you keep only a single copy of the object in memory for all threads. The difference is noticeable specially for long running processes.

Back to the original question, wich asked about a how to for SQLObjects and threads. I believe the same question can be applied if you resort to multiple processes. What's the best way to do it, specially for long running applications?

On Fri, Apr 18, 2008 at 5:13 AM, Oleg Broytmann <phd@phd.pp.ru> wrote:
On Thu, Apr 17, 2008 at 04:03:08PM -0700, Fred C wrote:
> I was wandering if there is somewhere an sqlobject 101 for threading
> environment?

  My personal position is: I avoid threads like a plague.

  Other than that extreme position - SQLObject is mostly thread-safe,
there are people who successfully use it in multithreading TurboGears
  There is a controversial patch, its author claims without the patch
SQLObject hangs, and there are reports that SQLObject hangs with the patch.
The patch is now in SQLObject but will be removed in the next release
because the last bug report was the patch causes more harm then good. (See
why I avoid threads? Threads are evil by nature: everything in programming
is about separation of control and responsibility - methods, classes,
functions, modules, processes - and threads violate the law by allowing
a few processes to share the same process space and memory; that violation
leads to deadlocks, race conditions and subtle, hard to debug bugs.)

    Oleg Broytmann            http://phd.pp.ru/            phd@phd.pp.ru
          Programmers don't die, they just GOSUB without RETURN.

This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
sqlobject-discuss mailing list

Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@gmail.com
mail: carribeiro@yahoo.com