mod-pubsub-developer

[Mod-pubsub-developer] port to twisted

[George S. <ge...@om...>]

Hi,

Joyce Park has been talking to me about porting mod_pubsub to use the
twisted event framework.  I've done some poking around and wanted to
share some assessments before I start work in earnest:

In your main event loop, you currently do this:

          while server.alive:
              asyncore.poll(scheduler.timeout())
              scheduler.run()

which basically polls with a timeout equal to the minimum time of any
scheduled event, handles those, and then does a linear search through
the entire list of scheduled events looking for ones which are
runnable.  Having to do a linear search though the scheduled event list
is relatively slow, prone to race conditions, and I can't think of a
good reason that it shouldn't be handled internal to the event system.

In moving to twisted, I would like to pull the scheduled events into
the twisted system itself, using the twisted reactors native support
for time-scheduled events.

Actual implementation aside, is their any reason I'm missing as to why
this is a bad idea?  I have a partial port running that incorporates
this functionality with twisted and it works well.

George

Re: [Mod-pubsub-developer] port to twisted

[Alex R. <al...@ne...>]

=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 26 March 2004 2:05 pm, George Schlossnagle wrote:
> Hi,
>
> Joyce Park has been talking to me about porting mod_pubsub to use
> the twisted event framework.  I've done some poking around and
> wanted to share some assessments before I start work in earnest:
>
> In your main event loop, you currently do this:
>
>           while server.alive:
>               asyncore.poll(scheduler.timeout())
>               scheduler.run()

Right. This was the primary discussion point around=20
poll/select/kqueue/epoll. We all realize that this is horrendously=20
inefficient. The Plan (TM), as I understand it, is to move this to a=20
Twisted reactor and then use the appropriate reactor for the system=20
at hand (kqueue, epoll, etc...).

Someone had mentioned libevent on Wed, and I'm considering writing a=20
reactor for it, which would ease the portability issues further.

[ snip ]
> Actual implementation aside, is their any reason I'm missing as to
> why this is a bad idea?=20

Nope, it's exactly the right approach.

Regards

=2D --=20
Alex Russell
al...@bu...   BD10 7AFC 87F6 63F9 1691 83FA 9884 3A15 AFC9 61B7
al...@ne... F687 1964 1EF6 453E 9BD0 5148 A15D 1D43 AB92 9A46
=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)

iD8DBQFAZLiuoV0dQ6uSmkYRAunUAKDI9jskaFwpZCmvIAp/xaFhN+OP0gCgsOS+
/z3jZEZfppqRU3PpIdsCufc=3D
=3DfAKv
=2D----END PGP SIGNATURE-----

Re: [Mod-pubsub-developer] port to twisted

[Joyce P. <tru...@ya...>]

--- Alex Russell <al...@ne...> wrote:
> The Plan (TM), as I understand it, is to move this to a 
> Twisted reactor and then use the appropriate reactor for the system 
> at hand (kqueue, epoll, etc...).
Just to be clear:  Friendster will be sponsoring part A but not part B.  We
only need to run on Linux.  If other people want to use libevent or otherwise
make it more portable, they're of course welcome to do so.  JP