Menu
▾
▴
mod-pubsub-developer — Developer list for mod_pubsub
You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(7) |
Dec
(45) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(16) |
Feb
(23) |
Mar
(62) |
Apr
(33) |
May
(35) |
Jun
(37) |
Jul
(45) |
Aug
(15) |
Sep
(22) |
Oct
(41) |
Nov
(23) |
Dec
(17) |
| 2004 |
Jan
(14) |
Feb
|
Mar
(55) |
Apr
(8) |
May
(1) |
Jun
(11) |
Jul
|
Aug
|
Sep
(20) |
Oct
(11) |
Nov
(10) |
Dec
(14) |
| 2005 |
Jan
(4) |
Feb
(2) |
Mar
(6) |
Apr
(26) |
May
(5) |
Jun
(11) |
Jul
(1) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2006 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
(18) |
Dec
(40) |
| 2007 |
Jan
(7) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Zachary G. <zgi...@pu...> - 2003-12-09 15:11:20
|
I've played with some of the demo apps on the mod-pubsub.org site and on a test server that we've setup here. When sniffing the connection, I find that from the mod-pubsub site javascript keepalive messages are sent once every 15 - 30 seconds, whereas the same app from our test server sends them about once per second. Is this configurable? Our test server is running apache 1.3 whereas I notice mod-pubsub.org has 2.0. Any hints will be much appreciated. Please cc this email address as I'm not subscribed to the mailing list. Zachary |
|
From: Chris O. <co...@dy...> - 2003-11-27 06:15:47
|
On Thu, 27 Nov 2003, Robert Leftwich wrote: > > At 09:01 AM 27/11/2003, Kragen Sitaker wrote: > > >On Thu, Nov 27, 2003 at 08:03:34AM +1100, Robert Leftwich wrote: > > > I'm using kn_content_filter to specify some server-side filtering and in > > > some cases the filter regex will only work if some additional flags are > > > specified when compiling it (e.g. DOTALL, MULTILINE mode, etc). To do this > > > I added a new kn_content_filter_flags element which can be used to specify > > > the required combination of flags (in integer form) which, if present, is > > > passed to the call to re.compile() in pubsub.py. > > > >Can't you just use "(?m:some$^regex)"? > > I wasn't aware that the Python regex engine supported the embedded modifer > syntax, but a bit of testing showed otherwise. Interestingly there is no > mention of that support in the Python doco. For the archives, the RE syntax page (4.2.1 Regular Expression Syntax, lib/re-syntax.html) does mention it, although it isn't very prominent. Search for "(?iLmsux)" to find it. /cco |
|
From: Joyce P. <tru...@ya...> - 2003-11-27 05:31:56
|
We have Python documentation? News to me! ;-) Oh, you mean PYTHON documentation... JP --- Robert Leftwich <ro...@le...> wrote: > > I wasn't aware that the Python regex engine supported the embedded modifer > syntax, but a bit of testing showed otherwise. Interestingly there is no > mention of that support in the Python doco. > > Thanks for the feedback. I can throw out my changes now and boy, do I just > love deleting code! |
|
From: Robert L. <ro...@le...> - 2003-11-27 04:29:48
|
At 09:01 AM 27/11/2003, Kragen Sitaker wrote: >On Thu, Nov 27, 2003 at 08:03:34AM +1100, Robert Leftwich wrote: > > I'm using kn_content_filter to specify some server-side filtering and in > > some cases the filter regex will only work if some additional flags are > > specified when compiling it (e.g. DOTALL, MULTILINE mode, etc). To do this > > I added a new kn_content_filter_flags element which can be used to specify > > the required combination of flags (in integer form) which, if present, is > > passed to the call to re.compile() in pubsub.py. > >Can't you just use "(?m:some$^regex)"? I wasn't aware that the Python regex engine supported the embedded modifer syntax, but a bit of testing showed otherwise. Interestingly there is no mention of that support in the Python doco. Thanks for the feedback. I can throw out my changes now and boy, do I just love deleting code! Robert |
|
From: Kragen S. <kra...@ai...> - 2003-11-26 22:02:34
|
On Thu, Nov 27, 2003 at 08:03:34AM +1100, Robert Leftwich wrote: > I'm using kn_content_filter to specify some server-side filtering and in > some cases the filter regex will only work if some additional flags are > specified when compiling it (e.g. DOTALL, MULTILINE mode, etc). To do this > I added a new kn_content_filter_flags element which can be used to specify > the required combination of flags (in integer form) which, if present, is > passed to the call to re.compile() in pubsub.py. Can't you just use "(?m:some$^regex)"? |
|
From: Robert L. <ro...@le...> - 2003-11-26 21:03:18
|
I'm using kn_content_filter to specify some server-side filtering and in some cases the filter regex will only work if some additional flags are specified when compiling it (e.g. DOTALL, MULTILINE mode, etc). To do this I added a new kn_content_filter_flags element which can be used to specify the required combination of flags (in integer form) which, if present, is passed to the call to re.compile() in pubsub.py. The reason for the email is to make sure that this is acceptable to all before committing it, in particular it might cause incompatibilities with the Perl server or for that matter the existing KN product as KN may have implemented it another way. Any comments would be welcome. Robert (attempting to fire up again after an extended hiatus...) |
|
From: Joyce P. <tru...@ya...> - 2003-11-24 07:06:33
|
Hi Andrew! Yes, it's all free for the taking from our Sourceforge site. We'd be thrilled if you used it! Do you understand how to set up the server and stuff? Joyce Park --- Andrew Johnson <adj...@ya...> wrote: > > I happened across your site, and I found your > co-browsing demo application. Is free to download > somewhere to try out on my web site? I hope to use it > for a multi-user conference (students) where the > individuals can surf together through presentations on > different web sites such that they don't have to type > URL addresses and inevitably get lost. It wouldn't be > public accessable to anyone outside the class, but all > other programs I have tried are no longer supported > (EZWebCar, iVisit, and another one I can't remember > the name of). Please help me.d-pubsub-developer |
|
From: Andrew J. <adj...@ya...> - 2003-11-24 06:22:02
|
Hello, I happened across your site, and I found your co-browsing demo application. Is free to download somewhere to try out on my web site? I hope to use it for a multi-user conference (students) where the individuals can surf together through presentations on different web sites such that they don't have to type URL addresses and inevitably get lost. It wouldn't be public accessable to anyone outside the class, but all other programs I have tried are no longer supported (EZWebCar, iVisit, and another one I can't remember the name of). Please help me. Andrew Johnson |
|
From: Asynch M. <asy...@ho...> - 2003-11-12 06:23:36
|
You don't need jscript - you can use the URL itself. Retrieving an image doesn't /require/ that the URL be a file based url. Spam mail does this via 'web-bugs'. Little graphics whose src URL has 'interesting' query parameters that identify the recipient, etc. You could also tail the web hit log for the image URL and post an event to a topic. There are no problems, only solutions... ----- Original Message ----- From: <ra...@ex...> To: <mod...@li...> Sent: Tuesday, November 11, 2003 9:01 AM Subject: [Mod-pubsub-developer] Tracking the meme > I was thinking about the new mps button, and viral marketing and > memes, and mod-pubsub, and about the new wiki page for sites that use > mod-pubsub, and stuff, and I was kind of stuttering on a comment jp > made about the inherent weakness of depending on people to register > things manually (useful in some contexts, a barrier in others, but that's > another topic)... > > ...and I thought, it sure would be nice if we knew where the mps-button > was getting to, and how often people were being exposed to it... > > What if we embedded a jscript in the button that did a quick pubsub 'ping' > to the mps server? Then, whenever a page that contains the mps-button is > opened, we'd get a ping. With this feedback, we could monitor how wide > and how fast the meme is spreading. That seems useful. > > Then I started wondering what kind of information should be contained > in this ping, if any. I think another mps-er named Mike was just > recently wondering how to discover what other event publications are > lurking just around the corner. Could event producers (electively, those > that want to be discoverable) use the ping to include some kind of > basic namespace/topic info about, say, the root topic space that their > feed publishes to? So that their feed could be dynamically discovered by > prospective consumers? > > I also am very interested in this issue of 'discoverability of feeds'... > > Thoughts? > > > ------------------------------------------------------- > This SF.Net email sponsored by: ApacheCon 2003, > 16-19 November in Las Vegas. Learn firsthand the latest > developments in Apache, PHP, Perl, XML, Java, MySQL, > WebDAV, and more! http://www.apachecon.com/ > _______________________________________________ > Mod-pubsub-developer mailing list > Mod...@li... > https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer > |
|
From: <ra...@ex...> - 2003-11-11 17:01:20
|
I was thinking about the new mps button, and viral marketing and memes, and mod-pubsub, and about the new wiki page for sites that use mod-pubsub, and stuff, and I was kind of stuttering on a comment jp made about the inherent weakness of depending on people to register things manually (useful in some contexts, a barrier in others, but that's another topic)... ...and I thought, it sure would be nice if we knew where the mps-button was getting to, and how often people were being exposed to it... What if we embedded a jscript in the button that did a quick pubsub 'ping' to the mps server? Then, whenever a page that contains the mps-button is opened, we'd get a ping. With this feedback, we could monitor how wide and how fast the meme is spreading. That seems useful. Then I started wondering what kind of information should be contained in this ping, if any. I think another mps-er named Mike was just recently wondering how to discover what other event publications are lurking just around the corner. Could event producers (electively, those that want to be discoverable) use the ping to include some kind of basic namespace/topic info about, say, the root topic space that their feed publishes to? So that their feed could be dynamically discovered by prospective consumers? I also am very interested in this issue of 'discoverability of feeds'... Thoughts? |
|
From: Joyce P. <tru...@ya...> - 2003-11-09 19:34:17
|
Hey all, can anyone help me get libkn working on Debian? It's barfing on creating the file "kn_config_h.in" during setup -- says it can't stat that file during the cp step, so I assume that means something earlier should have created it but didn't. The system: * Debian Woody * autoconf 2.53 * gcc 2.95 * g++ 2.95 * GNU make 3.79 * libtool 1.4.2 They're all in the default PATH. I'm sudoing, and all permissions are temporarily set to 775 all the way down the libkn directory structure. Anyone who's gotten this to work recently, lend me a clue! Thanks, JP |
|
From: Joyce P. <tru...@ya...> - 2003-11-08 18:18:02
|
There are other bits of Mod-pubsubbiness in the blog too -- like it pings when we create an entry, and also when someone makes a comment. We were planning to have it be kind of the centerpiece of Mojito... but now that's kind of on the back burner, at least for me. We could put the button under the Serendipity button at the bottom of the weblog. Let me try that. JP --- ra...@ex... wrote: --------------------------------- Excellent! Let's see that list grow. What do you think about putting it on the blog page? Given that 'blogchatter' and 'count connections' are driven by it...though I'm not sure exactly where in the layout is appropriate. |
|
From: <ra...@ex...> - 2003-11-08 02:42:39
|
Excellent! Let's see that list grow.<br> <br> What do you think about putting it on the blog page? Given that 'blogchatter' and 'count connections' are driven by it...though I'm not sure exactly where in the layout is appropriate.<BR><BR>randog<br> <BLOCKQUOTE style="PADDING-LEFT: 8px; MARGIN-LEFT: 8px; BORDER-LEFT: blue 2px solid"><BR>-----Original Message-----<BR>From: Joyce Park [mailto:tru...@ya...]<BR>Sent: Wednesday, November 05, 2003 10:14 PM<BR>To: Asynch Messaging; Rand Anderson;<BR>mod...@li...<BR>Subject: Re: [Mod-pubsub-developer] 'Built with Mod-PubSub' Logo/Button<BR><BR><BR>Genius, Mr. Anderson! I was inspired to put up a page on the wiki:<BR><BR>http://www.mod-pubsub.org/moinmoin/BuiltWithModPubsubButtons<BR><BR>And also one for projects built with Mod-pubsub:<BR><BR>http://www.mod-pubsub.org/moinmoin/BuiltWithModPubsub<BR><BR>Yay! JP<BR><BR><BR>--- Asynch Messaging <asy...@ho...> wrote:<BR>> MessageVery cool indeed!<BR>> <BR>> ----- Original Message ----- <BR>> From: Rand Anderson <BR>> To: mod...@li... <BR>> Sent: Wednesday, November 05, 2003 9:00 PM<BR>> Subject: [Mod-pubsub-developer] 'Built with Mod-PubSub' Logo/Button<BR>> <BR>> <BR>> MPS-er's,<BR>> So I was thinking about mod-pubsub evangelism, and I thought we should<BR>put<BR>> together a button that has the logo on it and says something like 'Built<BR>with<BR>> mod_pubsub', then place it prominently on the mod-pubsub homepage and<BR>> encourage folks to copy it and embed it in their mod_pubsub'ed pages and<BR>> products. Link the button to www.mod-pubsub.org.<BR>> <BR>> Or alternatively, instead of a 'built with' type thing, it could be<BR>> something like the XML orange pill, or something whipped up with the<BR>> button-maker app, like this:<BR>> <BR>> Or maybe it should use mod_pubsub official colors (are those colors on<BR>the<BR>> home-page the official ones? what kind of green is that? ;) :<BR>> <BR>> Now that I think about it, these buttons are more right (than the 'built<BR>> with' button style) with the soul of mod_pubsub anyway, what-with their<BR>> heritage in web protocol standards.<BR>> <BR>> Anyway, might help to propagate the meme.<BR>> <BR>> Whaddya'all think?<BR><BR><BR><BR>-------------------------------------------------------<BR>This SF.net email is sponsored by: SF.net Giveback Program.<BR>Does SourceForge.net help you be more productive? Does it<BR>help you create better code? SHARE THE LOVE, and help us help<BR>YOU! Click Here: http://sourceforge.net/donate/<BR>_______________________________________________<BR>Mod-pubsub-developer mailing list<BR>Mod...@li...<BR>https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer </BLOCKQUOTE> |
|
From: Joyce P. <tru...@ya...> - 2003-11-07 19:55:27
|
So I don't personally have a lot of faith in human-updated lists, but feel free to start something on the wiki Mike. :-) JP --- Asynch Messaging <asy...@ho...> wrote: > Well, I wasn't thinking of topic discovery, more like the type of > information published to topics. For example, I'd like to know that there is > a feed from Moreover and then find out what topic it uses. > > A Wiki page may be a good starting point to register event sources? |
|
From: Asynch M. <asy...@ho...> - 2003-11-07 06:03:21
|
Well, I wasn't thinking of topic discovery, more like the type of information published to topics. For example, I'd like to know that there is a feed from Moreover and then find out what topic it uses. A Wiki page may be a good starting point to register event sources? ----- Original Message ----- From: "Adam Rifkin" <Ad...@Kn...> To: "Joyce Park" <tru...@ya...>; "Asynch Messaging" <asy...@ho...> Cc: <mod...@li...> Sent: Thursday, November 06, 2003 7:30 PM Subject: RE: [Mod-pubsub-developer] Event Sources Mike wrote: > Is there a catalog of current event sources that are mod-pubsub compatible? > I'm wondering if there is an event stream on the next topic over that's > busily jammin away, with me none the wiser... Joyce replied: > Ah, very good question -- and unfortunately, not one we have a great answer for > right now. :-( We know of the Blogchatter feed and the RSS feed from Syndic8. > I also throw up an event every time we post on the blog, and every time > someone comments on a post. Much of the rest seems to be people trying out the > various little sample apps. > How can we better keep track? JP I don't really have a good answer here, either. Basically, introspect the server to see what's available on it. Joyce's analysis is pretty much what I know, too. Unfortunately, we don't have many introspection tools yet; what we do have is: mod_pubsub/webdev_toolkit/topicsubtool.php mod_pubsub/kn_apps/introspect/ mod_pubsub/kn_tools/list_events.plx mod_pubsub/kn_tools/list_routes.plx mod_pubsub/kn_tools/list_subtopics.plx Note that there are some feeds that we don't regularly run on mod-pubsub.org for streams like RSS from Moreover, stocks, and forex. These are in mod_pubsub/kn_sense/ What did you have in mind, Mike? Adam ------------------------------------------------------- This SF.net email is sponsored by: SF.net Giveback Program. Does SourceForge.net help you be more productive? Does it help you create better code? SHARE THE LOVE, and help us help YOU! Click Here: http://sourceforge.net/donate/ _______________________________________________ Mod-pubsub-developer mailing list Mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer |
|
From: Joyce P. <tru...@ya...> - 2003-11-07 03:37:57
|
Introspection is definitely our bete noire -- it's a REST thing, I guess. You know what we should do: have a tool that rank-orders the various topics by how frequently they get published to. That would tell us what we want to know, I think. Is there something in the server that keeps track of that stuff? JP --- Adam Rifkin <Ad...@Kn...> wrote: > Mike wrote: > > Is there a catalog of current event sources that are mod-pubsub compatible? > > I'm wondering if there is an event stream on the next topic over that's > > busily jammin away, with me none the wiser... > > Joyce replied: > > Ah, very good question -- and unfortunately, not one we have a great answer > for > > right now. :-( We know of the Blogchatter feed and the RSS feed from > Syndic8. > > I also throw up an event every time we post on the blog, and every time > > someone comments on a post. Much of the rest seems to be people trying out > the > > various little sample apps. > > How can we better keep track? JP > > I don't really have a good answer here, either. Basically, introspect the > server > to see what's available on it. Joyce's analysis is pretty much what I know, > too. > > Unfortunately, we don't have many introspection tools yet; what we do have > is: > > mod_pubsub/webdev_toolkit/topicsubtool.php > mod_pubsub/kn_apps/introspect/ > mod_pubsub/kn_tools/list_events.plx > mod_pubsub/kn_tools/list_routes.plx > mod_pubsub/kn_tools/list_subtopics.plx > > Note that there are some feeds that we don't regularly run on mod-pubsub.org > for > streams like RSS from Moreover, stocks, and forex. These are in > > mod_pubsub/kn_sense/ > > What did you have in mind, Mike? |
|
From: Adam R. <Ad...@Kn...> - 2003-11-07 03:32:21
|
Mike wrote: > Is there a catalog of current event sources that are mod-pubsub = compatible? > I'm wondering if there is an event stream on the next topic over = that's > busily jammin away, with me none the wiser... Joyce replied: > Ah, very good question -- and unfortunately, not one we have a great = answer for > right now. :-( We know of the Blogchatter feed and the RSS feed from = Syndic8. > I also throw up an event every time we post on the blog, and every = time > someone comments on a post. Much of the rest seems to be people = trying out the > various little sample apps. > How can we better keep track? JP I don't really have a good answer here, either. Basically, introspect = the server to see what's available on it. Joyce's analysis is pretty much what I = know, too. Unfortunately, we don't have many introspection tools yet; what we do = have is: mod_pubsub/webdev_toolkit/topicsubtool.php mod_pubsub/kn_apps/introspect/ mod_pubsub/kn_tools/list_events.plx mod_pubsub/kn_tools/list_routes.plx mod_pubsub/kn_tools/list_subtopics.plx Note that there are some feeds that we don't regularly run on = mod-pubsub.org for streams like RSS from Moreover, stocks, and forex. These are in mod_pubsub/kn_sense/ What did you have in mind, Mike? Adam |
|
From: Joyce P. <tru...@ya...> - 2003-11-06 07:41:07
|
OK, I know I shouldn't be working on this now... but I've recently fallen in love with Mozile. http://mozile.mozdev.org/ It allows you to edit the text of an XHTML page directly in Mozilla, using the DOM of course. Of special interest to us, it has a transport driver API that allows you to select the method you want to use to write out your changes permanently: directly to a file, via http POST, or webdav. So we could, of course, send website changes to the mod-pubsub server. I'm not sure if inserting mod-pubsub into the mix here adds much value... but it couldn't hurt. :-) JP |
|
From: Joyce P. <tru...@ya...> - 2003-11-06 06:31:24
|
Ah, very good question -- and unfortunately, not one we have a great answer for right now. :-( We know of the Blogchatter feed and the RSS feed from Syndic8. I also throw up an event every time we post on the blog, and every time someone comments on a post. Much of the rest seems to be people trying out the various little sample apps. How can we better keep track? JP --- Asynch Messaging <asy...@ho...> wrote: > MessageIs there a catalog of current event sources that are mod-pubsub > compatible? > I'm wondering if there is an event stream on the next topic over that's > busily jammin away, with me none the wiser... |
|
From: Joyce P. <tru...@ya...> - 2003-11-06 06:13:51
|
Genius, Mr. Anderson! I was inspired to put up a page on the wiki: http://www.mod-pubsub.org/moinmoin/BuiltWithModPubsubButtons And also one for projects built with Mod-pubsub: http://www.mod-pubsub.org/moinmoin/BuiltWithModPubsub Yay! JP --- Asynch Messaging <asy...@ho...> wrote: > MessageVery cool indeed! > > ----- Original Message ----- > From: Rand Anderson > To: mod...@li... > Sent: Wednesday, November 05, 2003 9:00 PM > Subject: [Mod-pubsub-developer] 'Built with Mod-PubSub' Logo/Button > > > MPS-er's, > So I was thinking about mod-pubsub evangelism, and I thought we should put > together a button that has the logo on it and says something like 'Built with > mod_pubsub', then place it prominently on the mod-pubsub homepage and > encourage folks to copy it and embed it in their mod_pubsub'ed pages and > products. Link the button to www.mod-pubsub.org. > > Or alternatively, instead of a 'built with' type thing, it could be > something like the XML orange pill, or something whipped up with the > button-maker app, like this: > > Or maybe it should use mod_pubsub official colors (are those colors on the > home-page the official ones? what kind of green is that? ;) : > > Now that I think about it, these buttons are more right (than the 'built > with' button style) with the soul of mod_pubsub anyway, what-with their > heritage in web protocol standards. > > Anyway, might help to propagate the meme. > > Whaddya'all think? |
|
From: Asynch M. <asy...@ho...> - 2003-11-06 06:04:02
|
MessageIs there a catalog of current event sources that are mod-pubsub = compatible? I'm wondering if there is an event stream on the next topic over that's = busily jammin away, with me none the wiser... |
|
From: Adam R. <Ad...@Kn...> - 2003-11-01 03:38:12
|
I noticed a corrupted file in the 0.993 tarball so I re-released the = tarball (with datestamp 10-31) for download at: = http://sourceforge.net/project/showfiles.php?group_id=3D66293&release_id=3D= 192963 For those of you who love trivia, we're very close to our 500th = mod_pubsub download and our 25,000th web page view. Congrats, folks! Also, Joyce made a lot of really good additions to the Wiki today: http://www.mod-pubsub.org/moinmoin/RecentChanges Including adding Kragen's post: = http://www.mod-pubsub.org/moinmoin/PythonServerWithOnePerlClientAndManyEv= ents And new pages for other considerations that will evolve over time: http://www.mod-pubsub.org/moinmoin/PerformanceConsiderations http://www.mod-pubsub.org/moinmoin/DesignConsiderations http://www.mod-pubsub.org/moinmoin/TheOneTrueThinClient Also, I really like Joyce's recent post to the blog, "Is simplicity = sexy?": http://www.mod-pubsub.org/blog/archives/344_Is_simplicity_sexy.html Included below for those (like me) too lazy to click the link. :) mod-pubsub.org is now regularly maintaining over 20 simultaneous = connections, Adam A bunch of us recently enjoyed a very stimulating (unfortunately = offline, or at least I can't find it in the mailing list archive) = discussion, led by Robert O. of New Zealand, about whether there is = something off-puttingly... too simple about mod-pubsub. That is an = amazingly deep and perceptive question with a lot of ramifications, and = one that we still don't have an adequate public answer for. You would think that something both powerful and easy to use would be = immediately perceived by all as a good thing. But... well, let me put it = this way. If you've ever watched Rohit give a talk, you've doubtless = noticed the audience dividing into those who look at him as if he has = the answer to all their questions about the Internet, and those who are = more or less squirming in their seats at the utter slobbiness of it all = (well, and the third group -- those who just have no idea what's he's = going on about but have been literally stupefied by the fluency of his = discourse). Invariably the skeptics zone right in on what they imagine = to be the weakness of mod-pubsub: lack of guarantees. It's quite amusing = to watch the total self-confidence with which Rohit blows off such = concerns, leaving his critics spluttering "But, but, but" in his wake. I = suspect for mod-pubsub developers, what looks to others like poor = engineering values and lack of foresight actually appears to be the = genius design bit. Of late I can't help coming to the conclusion that a whole bunch of good = programmers are having an awfully hard time coming to grips with the = Internet itself -- as opposed to nice neat tightly-coupled = object-oriented programming on a private LAN. On the Net, you don't = really know who you're dealing with, where they're coming from, what = they want, or how they plan to use the resources you give them. Many = engineers are loath to begin designing and building anything until = questions like these can be answered completely; and if they're forced = to do so, they can drive themselves crazy and burn a lot of resources = trying to cover every conceivable ambiguity. Instead of building within = the Internet paradigm, they fight everything good about the environment = -- particularly the ability to move quickly to share resources with = others. So yes, I think in some ways the apparent simplicity of mod-pubsub works = against us rather than for us. A lot of the people who are currently = interested in what we do seem to be architecturally-minded chaps, and = for a lot of people architecture means building Taj Mahals instead of, = say, Japanese detached palaces. Perhaps those who love marble mausoleums = won't be able to find anything beautiful in a little heap of sticks and = paper and bamboo. But the whole point of the detached palace might be = how organically and artfully it fits into its environment, how quickly = it can be put up with local products, and how a structure made of the = most fragile materials can exist for centuries. It's a different = aesthetic, and perhaps one that doesn't come naturally to the = (over)engineering mentality -- but it's actually a pretty complete = statement on Rohit and Adam's part of what it means to build software = for the Internet, or possibly to build the Internet itself. I don't = think it's too much to ask the world to accept or reject us on those = terms, instead of on the entirely specious grounds that we didn't build = something else.=20 |
|
From: Adam R. <Ad...@Kn...> - 2003-10-26 17:34:51
|
Thanks Joyce, I edited my Wiki page to say "inappropriate": http://www.mod-pubsub.org/moinmoin/AdamRifkin I also started http://www.mod-pubsub.org/moinmoin/ModPubsubMailingList which says: There are two mailing lists for ModPubsub and you can subscribe to them = from their Sourceforge interface: = http://sourceforge.net/mail/?group_id=3D66293 ...=20 mod-pubsub-developer is the more active list (averages 1-2 messages/day) = and covers any issues a developer of mod-pubsub or a developer with = mod-pubsub would encounter. You can use this list to share experiences = and ask questions.=20 mod-pubsub-general is the less technical, less active list (averages 1-2 = messages/week) and covers any issues not covered in = mod-pubsub-developer.=20 Wikis are fun, :) Adam -----Original Message----- From: Joyce Park [mailto:tru...@ya...] Sent: Sat 10/25/2003 3:23 PM To: mod...@li... Cc:=09 Subject: [Mod-pubsub-developer] About us -> wiki OK guys, as promised (and only a month late!) I have added an "About Us" = link from the blog that goes to the wiki page called "ModPubsubPeople". If = you haven't spent the last month fixing up your wiki page, you're really = letting the whole team down. :-P BTW Adam, I think I meant to say "totally _in_appropriate settings like = the mod-pubsub mailing list. :-) JP ------------------------------------------------------- This SF.net email is sponsored by: The SF.net Donation Program. Do you like what SourceForge.net is doing for the Open Source Community? Make a contribution, and help us add new features and functionality. Click here: http://sourceforge.net/donate/ _______________________________________________ Mod-pubsub-developer mailing list Mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer |
|
From: Adam R. <Ad...@Kn...> - 2003-10-25 23:37:05
|
Kragen, thanks for the writeup! Sounds like you've grappled with many interesting issues!! I could be wrong, but I believe you're the only one doing an app like = this. As for when the next PubSub User Group meeting is -- when are you = available? We'll coordinate one around you... :) Adam -----Original Message----- From: Kragen Sitaker [mailto:kra...@ai...] Sent: Wednesday, October 22, 2003 3:27 PM To: mod...@li... Subject: [Mod-pubsub-developer] using mod_pubsub in the middle of a production app So we've been slowly increasing our use of mod_pubsub in our product, and I thought I'd share my experience with all of you. First, the good news: we're mostly pretty happy with the improvements in performance and maintainability that are coming from restructuring most of our IPC to use mod_pubsub. It's easy to integrate. A note about YAML: we tried using YAML to serialize Perl data = structures, but it was amazingly slow. A small hash of 1000 entries took 0.1 = seconds to serialize and deserialize. Now we're using Data::Dumper to = serialize, and Perl eval to deserialize, which is about 100 times as fast. We're using the Perl microserver. One of the first problems we hit was that the microserver's 'subscribe' call wasn't persistent enough for us. Normally we start pubsub.py (the python_pubsub server) at the same time as all the other daemons. Sometimes it takes longer to initialize than the other daemons, so they start trying to subscribe too early, and get 'connection refused'. So we added a 'subscribe_blocking' method that retries every few seconds for a couple of minutes. Another one is that the microserver's PubSub::Tunnel::parse_events function is deeply recursive, if you're processing a large batch of events, and at least Perl 5.8 seems to use a lot of memory in this case. (Probably because each stack frame has its own local copy of the = remaining data in the network buffer.) We fixed this by breaking it apart into parse_event, which returns zero or one events, and parse_events, which is a loop around parse_event. A minor problem is that PubSub::Client::handle_events doesn't have any = way to block until there are events, so we settle for calling it frequently. It's pretty efficient when there aren't any events. One of the reasons for wanting to run things in an event-driven fashion is to decouple the different parts of your software. For example, our software monitors network nodes and periodically collects some data from them, which several programs analyze as they get it. It's nice for the data collector to have a separate set of resources, and not share failure modes, with the data analysis programs. So the data collector posts its data to a topic as it reads it, and the data analyzer = subscribes to the topic and analyzes data as it can, which is often slower than the collector can collect it. This causes some big problems with pubsub.py. As soon as an event is posted, it appends an encoded copy of the event to the outbound network buffers of all the subscribing tunnels, then tries to flush them. When the client sucking events down the tunnel isn't keeping up with the event stream, then eventually the network buffers associated with the client's socket and the socket on pubsub.py's side fill up, and pubsub.py starts getting EAGAIN errors. It handles these temporary failures "correctly" in that it keeps the data buffered and retries later when it can, but this creates a secondary problem. The outgoing data buffer can get very large; I've observed over 100MB. Events in the outgoing data buffer don't expire, and don't get updated by more recent events; this happens by design, although I can't remember why. But eventually the unbounded growth of this buffer starts paging everything else out to disk. It's even worse than that, actually, because the buffer is just a Python string. So when we append to it, we create a new string, copy the old contents into it, and append the new data. This happens frequently enough that it becomes a performance bottleneck. A further problem is that falling behind like this means that you're always processing outdated data. This is a problem for a real-time network monitoring system, where late answers are often wrong answers. We've adopted the solution of closing connections that get too much outgoing data buffered for them, by the simple expedient of raising an error in Connection.send if there's more than 50M of data already in the outbuffer before we append to it (by calling our superclass's send). (This could, of course, cause problems with large events, but as = mentioned before, we don't handle those very well anyway.) Our subscribers see = the EOF on the tunnel and die, and then get restarted externally. The error gets logged in pubsub.err.log and also in the individual client's log. This solution prevents the slow client from rendering the entire machine unusably slow, but it still produces late answers. So we have adopted a client-side solution to improve matters further. Instead of analyzing each event as it arrives, we simply deserialize the events and store them in a hash (indexed by network node ID), and periodically analyze = the entire hash --- the most recent data for each network node. Events = still pile up while we're analyzing, but the analyze time is bounded by the number of network nodes, so we keep up if we analyze rarely enough. So the question remains: how do we analyze rarely enough? We initially tried calling PubSub::Client::handle_events, then analyzing, and that worked up to a point. It would read all the available events, then analyze them. The trouble was, we could read the events into the hash fast enough that we could empty out our 128K of socket buffers and return from get_events before pubsub.py had a chance to get out of its select() and send us the other 4MB of data it had queued for us. So now we publish an event to a 'wake-up' topic, and analyze our hash of events when we receive that event back. We don't publish that event again until we've gotten it back, so we know that when we receive it, we've received all the events that got sent to us during our previous analysis phase. This seems to work reasonably well, but needs some kind of throttle to keep us from spending all of our time sending and handling the 'wake-up' event. Right now, we sleep 0.3 seconds, which puts us in the uncomfortable position of adjusting a polling interval to overcome problems in our event system. Perhaps instead we should wait for some other event to show up. I'm curious who else is doing stuff like this, what problems you've run into, and what solutions you've come up with. And when is the next mod-pubsub user's group? ------------------------------------------------------- This SF.net email is sponsored by OSDN developer relations Here's your chance to show off your extensive product knowledge We want to know what you know. Tell us and you have a chance to win $100 http://www.zoomerang.com/survey.zgi?HRPT1X3RYQNC5V4MLNSV3E54 _______________________________________________ Mod-pubsub-developer mailing list Mod...@li... https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer |
|
From: Joyce P. <tru...@ya...> - 2003-10-25 22:26:04
|
OK guys, as promised (and only a month late!) I have added an "About Us" link from the blog that goes to the wiki page called "ModPubsubPeople". If you haven't spent the last month fixing up your wiki page, you're really letting the whole team down. :-P BTW Adam, I think I meant to say "totally _in_appropriate settings like the mod-pubsub mailing list. :-) JP |
10 messages has been excluded from this view by a project administrator.
