[Mod-pubsub-developer] Twisted Internet -- worth investigating as an asyncore.py replacement?

[Adam R. <Ad...@Kn...>]

Regarding python_pubsub: the more we play with asyncore.py, the more we =
think it's worth investigating replacing asyncore with the Twisted =
framework from Twisted Matrix.  Kragen, is there a reason you didn't go =
with this library?  Was it even available in 2001?  Does it actually =
work now, in February 2003?

    http://twistedmatrix.com/

Twisted Matrix Laboratories is a distributed group of open-source =
developers working on Twisted, an event-driven networking framework =
written in Python and licensed under the LGPL. Twisted supports TCP, =
UDP, SSL/TLS, multicast, Unix sockets, a large number of protocols =
(including HTTP, NNTP, SSH, IRC, FTP, and others), and much more.

This would give us much cleaner multiprotocol handling.  It has some =
very useful abstractions that we don't have yet, such as a non-polling =
Reactor.

  -- Adam



P.S. -- They have a bunch of neat things, like an ORB, but the most =
interesting part is Twisted Internet:

    http://twistedmatrix.com/products/internet

Overview and Features

Twisted is an asynchronous networking framework. It provides the tools =
that you need to write client and server networking applications that =
don't require a large number of threads to scale.=20

Reactor-based Main Loop

Twisted Internet uses the Reactor Pattern for its event architecture. =
There are many pluggable implementations of the reactor for integrating =
with various toolkits and other event loops (more information can be =
found in the Reactor Basics document).=20

GUI Integration

GUI integration: Our Reactor design is "pluggable", allowing us to =
integrate various toolkits into Twisted. You can run servers or clients =
in a Twisted process and not have to worry about threading or =
conflicting event loops. What this means is that you can have a client =
or server running in a GUI, so implementing a user interface for your =
Twisted-based applications is easy! Currently Twisted integrates with =
PyGTK, WxPython, Tkinter, PyQT, and even a threaded Jython event loop.

Asynchronous Programming Utilities

Twisted provides a number of utilities for simplifying common =
asynchronous programming idioms. This includes tools for managing thread =
pools and support for 'deferred' operations which will execute in the =
future, and without blocking the main process.=20

Persistent Servers

Servers can be persistent; there need not be configuration files, only =
pickles of the state of the server as you run it last.

twistd - A Built-in Daemon

'twistd' is the Twisted Daemon. It is a simple tool designed in a UNIX =
and command-line friendly way; however it is portable to many =
environments (including Win32 and Jython). It can take care of many =
common tasks for you including:=20

1. Daemonization on UNIX platforms, with support for services on Windows =
NT/2000/XP in the future.=20
2. Logging.=20
3. Loading and execution of serialized Twisted application objects in =
any of the serialization formats supported by Twisted.=20
4. Saving a currently running Twisted application to a serialized object =
to be loaded by twistd in the future.=20

Wide Ranging Protocol Support

We are replacing various UNIX daemons as we go along; all of these =
services are designed to be friendly to other Python code and the =
Twisted framework. This includes the HTTP, IRC, POP3, SMTP, and Telnet =
implementations, which are fairly good examples of where we plan to take =
this. More information about the protocol support is available on =
Twisted Protocols.=20

Competition: Asyncore

Asyncore is a asynchronous networking library that is part of the Python =
standard library. It is a partial solution which mainly provides =
low-level networking support, leaving out many pieces of an =
industrial-strength asynchronous framework. Asyncore is also only being =
minimally maintained.=20

Here is a checklist of features for Twisted and Asyncore:=20

Feature                             Twisted Asyncore=20
TCP                                   Yes     Yes=20
UDP                                   Yes     Yes=20
SSL                                   Yes      No=20
Event scheduling                      Yes      No=20
Thread integration                    Yes      No=20
Win32 event-loop support              Yes      No=20
GUI integration                       Yes      No=20
Separation of Protocol and Transport  Yes      No=20
"echo" server implementation        5 lines 20 lines=20