[Mod-pubsub-developer] RE: Twisted Internet -- worth investigating as an asyncore.py replacement?

[Andrew A. <aa...@me...>]

Asyncore: good.
Twisted: better.
Nutter-butter: best.
A.


-----Original Message-----
From: Adam Rifkin [mailto:Ad...@Kn...]
Sent: Sunday, February 16, 2003 3:22 PM
To: mod...@li...
Subject: Twisted Internet -- worth investigating as an asyncore.py
replacement?


Regarding python_pubsub: the more we play with asyncore.py, the more we
think it's worth investigating replacing asyncore with the Twisted framework
from Twisted Matrix.  Kragen, is there a reason you didn't go with this
library?  Was it even available in 2001?  Does it actually work now, in
February 2003?

    http://twistedmatrix.com/

Twisted Matrix Laboratories is a distributed group of open-source developers
working on Twisted, an event-driven networking framework written in Python
and licensed under the LGPL. Twisted supports TCP, UDP, SSL/TLS, multicast,
Unix sockets, a large number of protocols (including HTTP, NNTP, SSH, IRC,
FTP, and others), and much more.

This would give us much cleaner multiprotocol handling.  It has some very
useful abstractions that we don't have yet, such as a non-polling Reactor.

  -- Adam



P.S. -- They have a bunch of neat things, like an ORB, but the most
interesting part is Twisted Internet:

    http://twistedmatrix.com/products/internet

Overview and Features

Twisted is an asynchronous networking framework. It provides the tools that
you need to write client and server networking applications that don't
require a large number of threads to scale.

Reactor-based Main Loop

Twisted Internet uses the Reactor Pattern for its event architecture. There
are many pluggable implementations of the reactor for integrating with
various toolkits and other event loops (more information can be found in the
Reactor Basics document).

GUI Integration

GUI integration: Our Reactor design is "pluggable", allowing us to integrate
various toolkits into Twisted. You can run servers or clients in a Twisted
process and not have to worry about threading or conflicting event loops.
What this means is that you can have a client or server running in a GUI, so
implementing a user interface for your Twisted-based applications is easy!
Currently Twisted integrates with PyGTK, WxPython, Tkinter, PyQT, and even a
threaded Jython event loop.

Asynchronous Programming Utilities

Twisted provides a number of utilities for simplifying common asynchronous
programming idioms. This includes tools for managing thread pools and
support for 'deferred' operations which will execute in the future, and
without blocking the main process.

Persistent Servers

Servers can be persistent; there need not be configuration files, only
pickles of the state of the server as you run it last.

twistd - A Built-in Daemon

'twistd' is the Twisted Daemon. It is a simple tool designed in a UNIX and
command-line friendly way; however it is portable to many environments
(including Win32 and Jython). It can take care of many common tasks for you
including:

1. Daemonization on UNIX platforms, with support for services on Windows
NT/2000/XP in the future.
2. Logging.
3. Loading and execution of serialized Twisted application objects in any of
the serialization formats supported by Twisted.
4. Saving a currently running Twisted application to a serialized object to
be loaded by twistd in the future.

Wide Ranging Protocol Support

We are replacing various UNIX daemons as we go along; all of these services
are designed to be friendly to other Python code and the Twisted framework.
This includes the HTTP, IRC, POP3, SMTP, and Telnet implementations, which
are fairly good examples of where we plan to take this. More information
about the protocol support is available on Twisted Protocols.

Competition: Asyncore

Asyncore is a asynchronous networking library that is part of the Python
standard library. It is a partial solution which mainly provides low-level
networking support, leaving out many pieces of an industrial-strength
asynchronous framework. Asyncore is also only being minimally maintained.

Here is a checklist of features for Twisted and Asyncore:

Feature                             Twisted Asyncore
TCP                                   Yes     Yes
UDP                                   Yes     Yes
SSL                                   Yes      No
Event scheduling                      Yes      No
Thread integration                    Yes      No
Win32 event-loop support              Yes      No
GUI integration                       Yes      No
Separation of Protocol and Transport  Yes      No
"echo" server implementation        5 lines 20 lines