=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Friday 26 March 2004 10:44 am, Joyce Park wrote:
> Hey guys,
>
> So we had a fabulous meeting Wednesday night, mostly focused on
> server improvements. My notes are appended. Old Mod-pubsubbers
> also had a chance to meet the fresh blood, Alex and Vishy and
> Henry, who are pushing the effort on behalf of Friendster.
It was really neat to meet everyone. I (for one) and very excited=20
about the possiblity of getting modpubsub to be scalable enough to=20
handle some of the loads we are discussing.
[snip]
> Performance
> --memory
> --long queues (delivery bug)
> --select()
So to outline this problem a little bit more for the list, the=20
traditional select() and poll() mechanisms available on POSIX systems=20
require a main-loop iteration over a list of open file descriptors to=20
determine if an action needs to happen for a given file descirptor.=20
When that list becomes very long, that traversal becomes increasingly=20
expensive, chewing up lots of CPU time that we (as app programmers)=20
would like to be using to actually _do_ something.
Given that clients of modpubsub protocol servers generally require=20
long-lived connections in order to acheive low-latency, modpubsub=20
essentially forces worst-case behavior of select() and poll().=20
=46ortunantly, OS manufacturers have recently come to our rescue with=20
kqueue/kevent (FreeBSD and OS X) and epoll (Linux 2.6). These=20
event-driven file descriptor change notification syscalls allow app=20
programmers to register callbacks which are called when something=20
happens, so apps can spend less time figuring out if they need to do=20
something, and more time actually _doing_ things. This asynchronous=20
approach is how we intend to get modpubsub servers supporting 10K=20
concurrent clients. This also dove-tails into using the Twisted async=20
framework for the Python server nicely.
> --old event initial route population has a long pause
> --HTTP 1.1
HTTP 1.1 will buy us chunking and reusable connections. These are both=20
Good Things (TM).
> --robust (lazy expiration)
> --topics never die
> --overhead for reaping
>
> Security
> --SSL
It was noted that in REALLY BIG installations, SSL is still expensive,=20
but might allow moderately sized sites/apps to allow HTTP 1.1 through=20
proxies.
> --authentication
> --security
Security requirements seem to be one of the largest areas for=20
improvement. It was decided that authentication and authorization=20
shouldn't be handled in the server deamon itself, but rather that the=20
server should expose sane "hooks" or proxy-points to allow an=20
external security monitor service to give the thumbs up or down to a=20
message. The discussion about Unix Domain Sockets sprung out of this=20
approach to security, sinc we'll need REALLY fast IPC to get this=20
working well.
> --do_method=3Dbatch loses errors, error handling unspecified
> --sequencing (events have to be delivered in order)
> --auth/auth policy callbacks or hooks
> --e.g. unix-domain-sockets
This point refers to a concept of talking the modpubsub HTTP protocol=20
over a UDS and that a security monitor could then insert itself into=20
the processing stream via this connection.
> --disable introspection
>
> Latency
>
> Scalability
> --10K connections/box
With fewer connections than this supported by a single server, the=20
communications overhead for replication and failover looks to become=20
overwhelming.
> Failover
> --topic-based expiration policy
Since topics are events, events can expire their sub-events, and=20
therefore topics can enforce sub-topic expiration. This seemed an=20
elegant way of dealing with te reaping problem, but we don't have any=20
experience with this in the wild. Time will tell if this design=20
works.
> --weak ref topic for route dest
> --lease-extension for expiring objects
Topic creation events for existing topics will re-set their=20
time-to-expire timestamps and then be dropped. This seems a good way=20
to allow applications to do topic "keepalive".
I'm sure others will have better clarifications.
Regards
=2D --=20
Alex Russell
al...@bu... BD10 7AFC 87F6 63F9 1691 83FA 9884 3A15 AFC9 61B7
al...@ne... F687 1964 1EF6 453E 9BD0 5148 A15D 1D43 AB92 9A46
=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)
iD8DBQFAZIrGoV0dQ6uSmkYRApRkAJ9+1Ls9Ztq7I18VtZweJKC6aQUFYACfetys
jYJPq7Pj20zX4drsSbKsD/0=3D
=3DG/6h
=2D----END PGP SIGNATURE-----
|
