Jp Calderone [mailto:exarkun@...] wrote:
> Twisted does attempt to be single-threaded in any case where this is
> possible. Unfortunately, it isn't -always- possible, usually
> when using an
> extension module that presents a blocking interface. To deal
> with this,
> Twisted does make it possible to employ threads, in a manner
> consistent with
> the rest of the single-threaded, asynchronous API. The basic unit of
> "callback abstraction" in Twisted is a Deferred object, and
> these can be
> used with threads:
> from twisted.internet import threads
> deferred = threads.deferToThread(longRunningFunction, ...)
> deferred.addCallbacks(handleResult, handleErrors)
> handleResult will be called with whatever object longRunningFunction
> chooses to return, or handleErrors will be called if it raises an
> exceptions. Both functions will be invoked in the main thread.
Thanks for pointing this out. Does this make use of a thread pool (as
Webware does), or does it spawn a new thread each time? A thread pool ought
to be more efficient, although depending on the OS spawning new threads may
be relatively cheap.
Not that it would be hard to add a thread pool to Twisted if it doesn't
already have one.
> One doesn't see this very often in the Twisted code, or
> applications that
> use Twisted, because we try so hard to avoid thread usage.
Well, it seems that any application that makes use of SQL (and that's
probably the majority of web applications) will either have to use threads
or will have to live with each SQL call blocking the entire server, which
seems quite unacceptable to me. If you're going to have to use threads, why
not use an appserver designed for threading from the start?
Another approach is to avoid threads and use a database adapter specifically
designed for integration with Twisted's event loop (I don't know if such
things exist, but it seems that Twisted has a little of everything...). But
in that case you're back to "contorting" your code to fit the async model.
Looking at some of the servlets I'm using, with significant amounts of code
and potentially many SQL queries involved, I shudder to think at the
convolutions I'd have to put into place to make them fit a purely async
model. It just doesn't fit my brain and would make the code significantly
more complicated and less understandable and maintainable.
Have people actually used Twisted to write SQL-driven web applications? Did
they use threads? How was the experience overall?
I can definitely see the value of combining Webware's thread pool and
servlet model together with Twisted's efficient handling of many protocols.