mod-pubsub-developer — Developer list for mod_pubsub

[Mod-pubsub-developer] Re: pubsub log4j appender

[Adam R. <Ad...@Kn...>]

Fred wrote:
> Here's some code for a pubsub log4j appender, which means you can=20
> configure your logging to issue events.

Fred, this looks great!  Wanna check it in?

> This code calls the kn_publish command, so it's limited to machines
> with the C library installed.  You can easily change it to use
> KnowNow's Java library, but I didn't since it's not in the mod_pubsub
> package.

How useful would a Java PubSub client library be?

Alternatively, how hard would it be to write a JNI wrapper for
c_pubsub and/or compile python_pubsub/pubsublib.py using Jython
into Java bytecodes?

   Adam

[Mod-pubsub-developer] mod_pubsub release 0.7 tarball is available

[Adam R. <Ad...@Kn...>]

Slowly but surely, the system evolves a little each week...
mod_pubsub v0.7 is now available at

=20
https://sourceforge.net/project/showfiles.php?group_id=3D66293&release_id=
=3D
144907

Changes include:
  1. Bug fixes, additions, and reorganization to cxx_pubsub.
  2. Cleanup of c_pubsub.  (Thanks Fred!)
  3. New kn_apps: connect4, reversi, topics, tree.
  4. New kn_tools: perftest_throughput.java
  5. New kn_sense: apache_logfile.sh
  6. New python_pubsub: bridge.py

Coming soon, I'm adding in Greg Burd's elisp and ruby pubsub client
libraries...
and there was much rejoicing...

   :) Adam

[Mod-pubsub-developer] RE: C MS and js_compress.sh breaks

[Adam R. <Ad...@Kn...>]

Fred wrote:
> I did a bunch of cleanup on the C Microserver last night, mostly=20
> fixing messed up indentation and cosmetic hooey like that, but also=20
> some autoconf/build stuff such as not forcing the native compiler on=20
> Solaris (you should be able to set CC to whatever compiler you want
now).

Thanks Fred!!

> Holler if I broke something.

Not that I saw... But...

> Adam, if you still can't build on Linux, send me the error.
> It should work.

It still doesn't work for me on Linux...

adam@oyster:~/sandbox/mod_pubsub/c_pubsub/libkn$ ./configure=20
loading cache ./config.cache
checking whether make sets ${MAKE}... yes
checking host system type... configure: error: can not guess host type;
you must specify one
adam@oyster:~/sandbox/mod_pubsub/c_pubsub/libkn$

Can I buy a clue?  ;)

> Subject: js_compress.sh breaks
> You shouldn't reply on a non-standard program like nl, or should at=20
> least document that it's needed in INSTALL, or notice it's missing and

> symlink pubsub.js to pubsub_raw.js.

Ok, from now on when we build a tarball we will run make first to
include
the compressed pubsub.js with the release.

The man page claims nl is from the gnu textutils.  What system doesn't
ship with it?
It should be easy to install on any system it's not a standard part of,
right?

   Adam

[Mod-pubsub-developer] pubsub log4j appender

[<wsa...@ap...>]

FYI-

   Here's some code for a pubsub log4j appender, which means you can 
configure your logging to issue events.  This code calls the kn_publish 
command, so it's limited to machines with the C library installed.  You 
can easily change it to use KnowNow's Java library, but I didn't since 
it's not in the mod_pubsub package.

	-wsv


//
// PubSubAppender.java
//
// This class uses the kn_publish command provided by the C pubsub 
library.
// Ideally, it would use a Java pubsub library, but we don't have one 
yet.
//

import org.apache.log4j.*;
import org.apache.log4j.spi.*;
import org.apache.log4j.helpers.LogLog;

public class PubSubAppender extends AppenderSkeleton {

     public String getPubCommand      () { return pubCommand     ; }
     public String getPubSubServerURI () { return pubSubServerURI; }
     public String topic              () { return topic          ; }

     public void setPubCommand      (String value) { pubCommand      = 
value; }
     public void setPubSubServerURI (String value) { pubSubServerURI = 
value; }
     public void setTopic           (String value) { topic           = 
value; }

     private String pubCommand      = null;
     private String pubSubServerURI = null;
     private String topic           = null;

     public boolean requiresLayout () { return false; }

     public boolean ready () {
         if (! ready) {
             ready = true;
             if (pubCommand == null) {
                 LogLog.error("unable to locate publish command; can't 
send HTTP events.");
                 ready = false;
             }
             if (pubSubServerURI == null) {
                 LogLog.error("no server URI; can't send HTTP events.");
                 ready = false;
             }
             if (topic == null) {
                 LogLog.error("no topic; can't send HTTP events.");
                 ready = false;
             }
         }
         return ready;
     }
     private boolean ready = false;

     public void append (LoggingEvent event) {
         if (! ready()) return;

         String level            = event.getLevel().toString();
         String loggerName       = event.getLoggerName();
         String message          = event.getRenderedMessage();
         String startTime        = String.valueOf(event.getStartTime());
         String threadName       = event.getThreadName();
         String className        = "";
         String fileName         = "";
         String lineNumber       = "";
         String methodName       = "";
         String formattedMessage = (this.layout == null) ? "" : 
this.layout.format(event);

         LocationInfo locationInfo = event.getLocationInformation();

         if (locationInfo != null) {
             className  = locationInfo.getClassName();
             fileName   = locationInfo.getFileName();
             lineNumber = locationInfo.getLineNumber();
             methodName = locationInfo.getMethodName();
         }

         String command[] = new String[]{
             pubCommand,
             "-H", "level"            , level           ,
             "-H", "class"            , className       ,
             "-H", "file"             , fileName        ,
             "-H", "line"             , lineNumber      ,
             "-H", "method"           , methodName      ,
             "-H", "logger"           , loggerName      ,
             "-H", "message"          , message         ,
             "-H", "start time"       , startTime       ,
             "-H", "thread"           , threadName      ,
             "-H", "formatted message", formattedMessage,
             pubSubServerURI, topic, "/dev/null"
         };

         try {
             if (Runtime.getRuntime().exec(command).waitFor() != 0) {
                 LogLog.warn("unable to log event: " + 
event.getMessage());
             }
         } catch (Exception e) {
             LogLog.warn ("exception while logging event: " + 
event.getMessage());
             LogLog.error("exception while logging: " + e);
         }
     }

     public void close () {}
}

[Mod-pubsub-developer] js_compress.sh breaks

[<wsa...@ap...>]

   You shouldn't reply on a non-standard program like nl, or should at 
least document that it's needed in INSTALL, or notice it's missing and 
symlink pubsub.js to pubsub_raw.js.

	-wsv


[silent-bob:WebServer/Documents/mod_pubsub] wsanchez% sudo make 
HTTPD_USER=www
bash kn_tools/js_compress.sh < kn_apps/kn_lib/pubsub_raw.js > 
kn_apps/kn_lib/pubsub.js
sed: 12: "
         s/\\\//___quot ...": undefined label ';'
kn_tools/js_compress.sh: nl: command not found
[ x"www" != x ]  # Define HTTPD_USER as the user that runs your web 
server, e.g. make HTTPD_USER=httpd (or nobody, or www-data)
mkdir kn_events
if ! chown www kn_events; then rmdir kn_events; false; fi

[Mod-pubsub-developer] C MS

[<wsa...@ap...>]

   I did a bunch of cleanup on the C Microserver last night, mostly 
fixing messed up indentation and cosmetic hooey like that, but also 
some autoconf/build stuff such as not forcing the native compiler on 
Solaris (you should be able to set CC to whatever compiler you want 
now).

   Holler if I broke something.

   Adam, if you still can't build on Linux, send me the error.  It 
should work.

	-wsv

[Mod-pubsub-developer] New cvs check-ins: topics-listing app, Apache logfile sensor, throughput perf test tool, bridge.py

[Adam R. <Ad...@Kn...>]

Four new cvs check-ins:

1. kn_apps/topics

   A browser JavaScript app that lists the topics on a PubSub server.
   I updated the kn_apps/index.html to include this new utility.

   See also the command-line utility kn_tools/list_subtopics.plx.

2. kn_sense/apache_logfile.sh

   Fred's Apache logfile sensor, written in the Bourne shell.
   This sensor monitors an Apache log file (common or combined) and
   generates events as log entries are written to it.

   Note that this sensor requires the libkn PubSub C Library
   available in the c_pubsub/libkn directory.

   Also note that since I haven't been able to figure out how to
   get this c_pubsub/libkn library to compile on Linux, I still
   have not gotten this sensor working yet, either.

   But I assure you, it's very fine code.  *wink*

3. kn_tools/perftest_throughput.java

   Adam's command-line utility that does not require any PubSub
   client library to run.  All it needs is a Java runtime.

   Spawns x threads, each of which publish y events,
   and tells you the total elapsed time in ms.

   Slowly but surely we're putting together a useful set of tools.

4. python_pubsub/bridge.py

   Andrew's bridge demonstration.

   We've often needed the capability to "bridge" between pubsub
   servers.  That is, to have two pubsub servers reflect the same
   topics and events on those topics.  Often, we want this to be the
   case for only a subset of the topic space of these two pubsub
   servers.  Often, we want to do transformation on the way.

   You could do this using routes between the pubsub servers -- use
   do_max_age with an inter-pubsub-server route.  Alternatively,
   bridge.py subscribes to various pubsub servers (as configured in
   the variables at the top) and attempts to synchronize topic
   hierarchies and events.

   FIXME: This should be generalized to bridge between URLs -- that
   is, it should work intra-pubsub-server as well as
   inter-pubsub-server.  For example, so that you can mimic the
   subtopic space of /what/nasdaq onto /what/realtime/app.


Also, I just got a bunch of updates to the PubSub C++ Library
for Windows/ActiveX/.NET connectivity.  I need to do some diffs,
After which I'll check them into the cxx_pubsub/ directory.

Enjoy,
  Adam

[Mod-pubsub-developer] Instant messaging with pushlets: presence-by-polling.

[Adam R. <Ad...@Kn...>]

A few short notes:

  1. People are using Pushlets to try to build instant messaging =
environments.

  2. The suggestion from the email thread below is to use a polling =
mechanism
(via JS timers) to check to see if the Pushlet client is still connected =
to
the Pushlet server.  Presence-by-polling may not scale, but it's =
probably a
serviceable solution for what they're using it for.

  -- Adam





  function checkClient() {
    var frame1 =3D chkPushletLocation();
    var errorString =3D "Screen "+subject;
    if(frame1=3D=3Dfalse){
      errorString =3D errorString+" Pushlet has failed. ";
      pushletFrame.pushReceiverFrame.location.href =3D
pushletFrame.pushletURL;
    }

    //Send info to servlet to send email to the administrator.
    setTimeout('checkClient();', 5000);
  }

  function chkPushletLocation() {
    if (pushletFrame.pushletIsLive) {
=20
window.pushletFrame.framesetController.document.body.style.backgroundColo=
r=3D"
white";
      return true;
    }
    else {  // sets background colour of frameset controller to red so =
you
know it's trying to resuscitate the pushlet
=20
window.pushletFrame.framesetController.document.body.style.backgroundColo=
r=3D"
red";
      return false;
    }
  }





-----Original Message-----
From: pu...@ya... [mailto:pu...@ya...]
Sent: Tuesday, March 04, 2003 9:05 AM
To: pu...@ya...
Subject: [pushlet] Digest Number 192

To Post a message, send it to:   pu...@eG...
To Unsubscribe, send a blank message to: pus...@eG...
------------------------------------------------------------------------

There are 5 messages in this issue.

Topics in this digest:

      1. Stopping the server
           From: "walruszz <fi...@ti...>" <fi...@ti...>
      2. RE: Stopping the server
           From: "Briggs, Ben" <ben...@ed...>
      3. RE: Stopping the server
           From: "Briggs, Ben" <ben...@ed...>
      4. Re: Stopping the server
           From: "walruszz <fi...@ti...>" <fi...@ti...>
      5. RE: Re: Stopping the server
           From: "Briggs, Ben" <ben...@ed...>


________________________________________________________________________
________________________________________________________________________

Message: 1
   Date: Tue, 04 Mar 2003 10:50:50 -0000
   From: "walruszz <fi...@ti...>" <fi...@ti...>
Subject: Stopping the server

Hi!
I'm using this framework to develop an istant messaging environment,=20
and it's working pretty fine.
I use apache/tomcat 4.0.1.
Here's the little problem: when I want to stop and republish the=20
server, it waits until all the client browser pages are closed, or=20
moved to other locations.
How can I stop the server if someone (possibly far) has the page=20
open? Is there a way to force it?

Thank you very much!

Filippo Mescoli




________________________________________________________________________
________________________________________________________________________

Message: 2
   Date: Tue, 4 Mar 2003 11:16:45 -0000
   From: "Briggs, Ben" <ben...@ed...>
Subject: RE: Stopping the server

Buon giorno Filippo,
=20
I haven't noticed this problem; perhaps WebLogic has different behaviour
from Tomcat in this regard. When I stop the server to republish it, it
automatically kills the pushlet servlets and the connections to the =
browser
close.
=20
Perhaps someone will suggest a way to close connections that already =
exists.
If not, you might change things a little, and see if it's good enough to
contribute to Just's project. Here's a suggestion for a change:
=20
To force it to close, you could cause the PushletSubscriber to break out =
of
its infinite loop. So instead of it doing "while(true)", it could do a
"while(finished=3D=3Dfalse)" and if it ever receives an event with the =
subject
"killAllSubscribers" or something, then it would set finished=3Dtrue. =
Just an
idea.
=20
Alternatively, you could do it on the client side, and when the clients
received the "killAllSubscribers" event, then they could disconnect by
changing the location.href to a benign location. However, I wouldn't
recommend this route since JavaScript is so browser-dependent.
=20
It's worth noting that some colleagues and I have written some =
JavaScript
that ties in with the Pushlet framework and detects when the connection =
has
been lost between the client and the server (due to the server being
restarted, or a cable coming unplugged or anything). It then =
automatically
reopens the connection to the servlet as soon as the HTTP connection can =
be
reestablished. I'd be happy to share this with the project.
=20
Cheers,
Ben
=20
=20

-----Original Message-----
From: walruszz <fi...@ti...> [mailto:fi...@ti...]
Sent: 04 March 2003 10:51
To: pu...@ya...
Subject: [pushlet] Stopping the server


Hi!
I'm using this framework to develop an istant messaging environment,=20
and it's working pretty fine.
I use apache/tomcat 4.0.1.
Here's the little problem: when I want to stop and republish the=20
server, it waits until all the client browser pages are closed, or=20
moved to other locations.
How can I stop the server if someone (possibly far) has the page=20
open? Is there a way to force it?

Thank you very much!

Filippo Mescoli

________________________________________________________________________
________________________________________________________________________

Message: 3
   Date: Tue, 4 Mar 2003 15:02:56 -0000
   From: "Briggs, Ben" <ben...@ed...>
Subject: RE: Stopping the server

Greg,

It's not been written for cross-browser compatibility yet, but it works =
in
IE6 (on Weblogic 7).

The primary modification is the <b>ONLOAD=3D"pushletStopped()"</b> added =
to
the frame loading the pushlet servlet. If the connection dies, the frame
stops loading and in a sense receives the end of the page. This triggers =
the
browser's onLoad event handler, and then some javascript -- in this case
pushletStopped() -- can handle what to do if it's lost the connection.

The "frameset controller" frame polls the pushlet frame to see if its =
still
alive. Mine does this every 5 seconds, but it doesn't have to be this =
often.
I've changed the architecture a bit and would need to go back to the
original pushlet framework to see what needs doing to make these ideas =
fit
in with the Pushlet project. But maybe these code snippets will be =
enough
for you to understand the basic premise.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
in pushlet.html:

<SCRIPT LANGUAGE=3D"JavaScript">
  self.document.writeln('<FRAMESET BORDER=3D1 ROWS=3D"30,*">');
  self.document.writeln('<FRAME SRC=3D"frameset-controller.html"
NAME=3D"framesetController"  BORDER=3D0  SCROLLING=3Dno>');
  self.document.writeln('<FRAME =
SRC=3D"/PubDisp/servlet/pushlet?subject=3D' +
subject + '&bgColor=3D' + bgColor + '" NAME=3D"pushReceiverFrame" =
BORDER=3D0
SCROLLING=3Dno ONLOAD=3D"pushletStopped()">');
  self.document.writeln('</FRAMESET>');
  statusFrameReady=3Dtrue;
</SCRIPT>

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
in pushlet.html:

  function pushletStopped() {
    if (pushletIsLive =3D=3D true) { // inform the admin if the pushlet =
just
died
      // call a servlet that sends an email to the administrator
    }
    pushletIsLive =3D false;
  }

  setTimeout('parent.checkClient();', 5000);

  function restartPushlet() {
//    alert('restarting pushlet');
    pushReceiverFrame.src =3D
"/PubDisp/servlet/pushlet?subject=3D"+subject+"&bgColor=3D"+bgColor;
  }

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
in client.js:

  function checkClient() {
    var frame1 =3D chkPushletLocation();
    var errorString =3D "Screen "+subject;
    if(frame1=3D=3Dfalse){
      errorString =3D errorString+" Pushlet has failed. ";
      pushletFrame.pushReceiverFrame.location.href =3D
pushletFrame.pushletURL;
    }

    //Send info to servlet to send email to the administrator.
    setTimeout('checkClient();', 5000);
  }

  function chkPushletLocation() {
    if (pushletFrame.pushletIsLive) {
=20
window.pushletFrame.framesetController.document.body.style.backgroundColo=
r=3D"
white";
      return true;
    }
    else {  // sets background colour of frameset controller to red so =
you
know it's trying to resuscitate the pushlet
=20
window.pushletFrame.framesetController.document.body.style.backgroundColo=
r=3D"
red";
      return false;
    }
  }


Cheers,
Ben

=20


-----Original Message-----
From: Greg Bohmann [mailto:gcb...@ya...]
Sent: 04 March 2003 14:38
To: Briggs, Ben
Subject: RE: [pushlet] Stopping the server


Ben -

You mentioned that you and some other individuals have already
developed JavaScript code that can reestablish a lost HTTP connection.=20
I am interested in seeing the code and am sure others in the pushlet
community are as well.

The increased availability gained through this modification would
benefit all applications utilizing the pushlet framework whether in a
single server or clustered setting.

Thanks,
Greg

--- "Briggs, Ben" <ben...@ed...> wrote:
> Buon giorno Filippo,
> =20
> I haven't noticed this problem; perhaps WebLogic has different
> behaviour
> from Tomcat in this regard. When I stop the server to republish it,
> it
> automatically kills the pushlet servlets and the connections to the
> browser
> close.
> =20
> Perhaps someone will suggest a way to close connections that already
> exists.
> If not, you might change things a little, and see if it's good enough
> to
> contribute to Just's project. Here's a suggestion for a change:
> =20
> To force it to close, you could cause the PushletSubscriber to break
> out of
> its infinite loop. So instead of it doing "while(true)", it could do
> a
> "while(finished=3D=3Dfalse)" and if it ever receives an event with the
> subject
> "killAllSubscribers" or something, then it would set finished=3Dtrue.
> Just an
> idea.
> =20
> Alternatively, you could do it on the client side, and when the
> clients
> received the "killAllSubscribers" event, then they could disconnect
> by
> changing the location.href to a benign location. However, I wouldn't
> recommend this route since JavaScript is so browser-dependent.
> =20
> It's worth noting that some colleagues and I have written some
> JavaScript
> that ties in with the Pushlet framework and detects when the
> connection has
> been lost between the client and the server (due to the server being
> restarted, or a cable coming unplugged or anything). It then
> automatically
> reopens the connection to the servlet as soon as the HTTP connection
> can be
> reestablished. I'd be happy to share this with the project.
> =20
> Cheers,
> Ben


________________________________________________________________________
________________________________________________________________________

Message: 4
   Date: Tue, 04 Mar 2003 15:38:21 -0000
   From: "walruszz <fi...@ti...>" <fi...@ti...>
Subject: Re: Stopping the server

--- In pu...@ya..., "Briggs, Ben" <ben.briggs@e...> wrote:
> Here's a suggestion for a change:
> =20
> To force it to close, you could cause the PushletSubscriber to=20
break out of
> its infinite loop. So instead of it doing "while(true)", it could=20
do a
> "while(finished=3D=3Dfalse)" and if it ever receives an event with the =

subject
> "killAllSubscribers" or something, then it would set finished=3Dtrue.=20
Just an
> idea.

Thank you very much Ben,
so I have to send a message to the servlet, prior to stop the server?
I'll try this one first.
Then I will look for an automatic mechanism to force it to step out=20
from the loop when I try to stop the server.

Bye!

Filippo Mescoli



________________________________________________________________________
________________________________________________________________________

Message: 5
   Date: Tue, 4 Mar 2003 15:57:31 -0000
   From: "Briggs, Ben" <ben...@ed...>
Subject: RE: Re: Stopping the server

Filippo,
=20
You might be able to write an exit servlet that runs when you deactivate =
the
web application. You can then configure this in your web.xml file to =
listen
for this event.
=20
I'm not sure if this would work because it might be a case of the =
chicken
and the egg. If this event isn't fired until after the pushlets are =
closed,
then you can't expect these methods to cause the pushlets to close. You
could then make a request to this servlet, such as
http://localhost:8080/pushlet/ExitServlet
<http://localhost:8080/pushlet/ExitServlet> . But if you don't require
authentication, then anyone can kill all your pushlets.
=20
=20
Something like this
=20
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
in ExitServlet.java
=20
package ###path###;
=20
import javax.servlet.*;
import javax.servlet.http.*;
import java.io.IOException;
import java.io.PrintWriter;
=20
import ###path###.Publisher;

public class ExitServlet extends HttpServlet implements
ServletContextListener {
  static final private String CONTENT_TYPE =3D "text/html";
  //Initialize global variables
  public void init() throws ServletException {
  }
=20
  public void contextInitialized(ServletContextEvent e) {
=20
System.out.println("#####################################################=
###
#");
    System.out.println("Pushlet application loading...");
=20
System.out.println("#####################################################=
###
#");
  }
=20
  public void contextDestroyed(ServletContextEvent e) {
=20
System.out.println("#####################################################=
###
#");
    System.out.println("Telling all pushlets to close...");
    shutDownPushlets();
=20
System.out.println("#####################################################=
###
#");
    System.out.println("Pushlet application shutting down...");
=20
System.out.println("#####################################################=
###
#");
  }
=20
  private void shutDownPushlets() {
    Publisher.getInstance().publish(/* event with subject
"killAllSubscribers" */);
  }
=20
 public void doGet(HttpServletRequest request, HttpServletResponse =
response)
throws ServletException, IOException {
    response.setContentType(CONTENT_TYPE);
    PrintWriter out =3D response.getWriter();
    out.println("Shutting down pushlets");
    shutDownPushlets();
  }
  =20

}
=20
=20
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=20
in web.xml
=20
  <listener>
    <listener-class>###path###.ExitServlet</listener-class>
  </listener>

=20
Cheers,
Ben

=20

-----Original Message-----
From: walruszz <fi...@ti...> [mailto:fi...@ti...]
Sent: 04 March 2003 15:38
To: pu...@ya...
Subject: [pushlet] Re: Stopping the server


--- In pu...@ya..., "Briggs, Ben" <ben.briggs@e...> wrote:
> Here's a suggestion for a change:
> =20
> To force it to close, you could cause the PushletSubscriber to=20
break out of
> its infinite loop. So instead of it doing "while(true)", it could=20
do a
> "while(finished=3D=3Dfalse)" and if it ever receives an event with the =

subject
> "killAllSubscribers" or something, then it would set finished=3Dtrue.=20
Just an
> idea.

Thank you very much Ben,
so I have to send a message to the servlet, prior to stop the server?
I'll try this one first.
Then I will look for an automatic mechanism to force it to step out=20
from the loop when I try to stop the server.

Bye!

Filippo Mescoli




[Non-text portions of this message have been removed]



________________________________________________________________________
________________________________________________________________________

[Mod-pubsub-developer] RE: PubSub User Group (PUG) Meeting minutes for February.

[Adam R. <Ad...@Kn...>]

Oh yeah, I forgot the minutes.  Actually, more like hours.

I think, in total, we argued for about two hours about why kn_journals
should be removed from the system.  And Rohit mooched food from
everyone.

   ;) Adam




-----Original Message-----
From: Adam Rifkin=20
Sent: Friday, February 28, 2003 4:01 PM
To: 'mod...@li...'
Subject: PubSub User Group (PUG) Meeting minutes for February.


New cvs change to the Python PubSub server:
  - fixed kn_expires support

Mod_pubsub statistics, as of 2/28/2003:
  # of developers: 36
  # of client libraries: 5 (C, Windows C++, JavaScript, Perl, Python)
  # of kn_apps: 40 (new this month: sitewatch)
  # of CVS adds: 464
  # of CVS commits: 124
  # of downloads: 118
  # of pageviews: 4273
  # of mailing list list emails: 98
  SF project rank: 6003 out of 57,380

I provide no trend analysis because we have no trends.
We're doing zero marketing except for our own (minimal) word-of-mouth.
Vive la http://mod-pubsub.sourceforge.net/

As for the future...

Potential things to add to mod_pubsub distribution:
  - command-line performance testing tools
  - Fred's Apache web server logfile sensor (+ update sitewatch to use
it?)
  - Joyce's soon-to-be PHP client library
  - updates to the Windows C++ client library (including PocketPC
support!)
  - Greg's elisp client library
  - Greg's ruby client library
  - Jeff's Excel client library
  - update Perl client library to support trailing "/"s
  - Java client library (or, try to compile pubsublib.py as Java byte
codes that run on a JVM)
  - JavaScript client library that's not browser-based for use with
Konfabulator runtime
  - Flash client library
  - SOAP client library
  - make pubsub.cgi an actual Apache module, mod_pubsub
  - kn_apps for Connect 4 game and Reversi game
  - kn_apps for demonstrating email in and out of a PubSub server
  - explore Windows Scripting Host uses
  - explore use with mod_dav for DAV notifications

And, because I'm on a Python kick...

Potential Python things to add to mod_pubsub distribution:
  - Python command-line tools to list events, routes, and subtopics
(also, recursively)
  - cleaner.py utility to remove expired events from a PubSub server
  - off-host route support (and kn_content_transform support) in
pubsub.py and pubsublib.py
    [this may involve a switch to using standard asyncore rather than
our custom one]
  - username/password authentication and SSL support in pubsub.py and
pubsublib.py
  - Python client library test suite ported from the JavaScript PubSub
library suite
  - Python sample applications for ping and chat
  - update scheduler.py to do exception handling
  - try to compile pubsub.py as a Windows exe
  - explore use of Twisted Internet
  - bridge.py to form server federations
  - explore the merging of pubsub.py and pubsublib.py

Next month's PubSub User Group meeting will be earlier (Tue March 25 @
7pm) somewhere Further north (say, somewhere in Redwood City).  See you
then,

   Adam

[Mod-pubsub-developer] PubSub User Group (PUG) Meeting minutes for February.

[Adam R. <Ad...@Kn...>]

New cvs change to the Python PubSub server:
  - fixed kn_expires support

Mod_pubsub statistics, as of 2/28/2003:
  # of developers: 36
  # of client libraries: 5 (C, Windows C++, JavaScript, Perl, Python)
  # of kn_apps: 40 (new this month: sitewatch)
  # of CVS adds: 464
  # of CVS commits: 124
  # of downloads: 118
  # of pageviews: 4273
  # of mailing list list emails: 98
  SF project rank: 6003 out of 57,380

I provide no trend analysis because we have no trends.
We're doing zero marketing except for our own (minimal)
word-of-mouth.  Vive la http://mod-pubsub.sourceforge.net/

As for the future...

Potential things to add to mod_pubsub distribution:
  - command-line performance testing tools
  - Fred's Apache web server logfile sensor (+ update sitewatch to use
it?)
  - Joyce's soon-to-be PHP client library
  - updates to the Windows C++ client library (including PocketPC
support!)
  - Greg's elisp client library
  - Greg's ruby client library
  - Jeff's Excel client library
  - update Perl client library to support trailing "/"s
  - Java client library (or, try to compile pubsublib.py as Java byte
codes that run on a JVM)
  - JavaScript client library that's not browser-based for use with
Konfabulator runtime
  - Flash client library
  - SOAP client library
  - make pubsub.cgi an actual Apache module, mod_pubsub
  - kn_apps for Connect 4 game and Reversi game
  - kn_apps for demonstrating email in and out of a PubSub server
  - explore Windows Scripting Host uses
  - explore use with mod_dav for DAV notifications

And, because I'm on a Python kick...

Potential Python things to add to mod_pubsub distribution:
  - Python command-line tools to list events, routes, and subtopics
(also, recursively)
  - cleaner.py utility to remove expired events from a PubSub server
  - off-host route support (and kn_content_transform support) in
pubsub.py and pubsublib.py
    [this may involve a switch to using standard asyncore rather than
our custom one]
  - username/password authentication and SSL support in pubsub.py and
pubsublib.py
  - Python client library test suite ported from the JavaScript PubSub
library suite
  - Python sample applications for ping and chat
  - update scheduler.py to do exception handling
  - try to compile pubsub.py as a Windows exe
  - explore use of Twisted Internet
  - bridge.py to form server federations
  - explore the merging of pubsub.py and pubsublib.py

Next month's PubSub User Group meeting will be earlier (Tue March 25 @
7pm) somewhere
Further north (say, somewhere in Redwood City).  See you then,

   Adam

[Mod-pubsub-developer] mod_pubsub v0.6 tarball now available on Sourceforge...

[Adam R. <Ad...@Kn...>]

Slowly but surely the mod_pubsub distribution is making improvements.

mod_pubsub v0.6 tarball is now available for download:

   =
https://sourceforge.net/project/showfiles.php?group_id=3D66293&release_id=
=3D139122

New additions in version 0,6:

   1. Added python_pubsub client library.
   2. Fixed bug in pubsub.cgi that allows relative pathnames.
   3. Added some command-line kn_tools: list_events.plx, =
list_subtopics.plx, and list_routes.plx.
   4. Modified sitewatch application to use new python_pubsub client.

This includes some cvs check-ins I did today: a top-level index.html =
that gives an overview of all the goodies in the distribution, and some =
command-line utilities in kn_tools:

kn_tools/list_events.plx
kn_tools/list_subtopics.plx
kn_tools/list_routes.plx

usage is similar for all three utilities: give them two arguments, a =
router uri and a topic name. example output:

$ cd kn_tools
$ perl list_subtopics.plx http://127.0.0.1:8000/kn/ /what
chat [Fri Feb 21 15:53:30 2003]
$ perl list_events.plx http://127.0.0.1:8000/kn/ /what/chat
391f7a28-2ff7-11d7-8dd2-6b61626f6f6d [Fri Feb 21 15:54:49 2003]
392100dc-2ff7-11d7-8dd2-6b61626f6f6d [Fri Feb 21 15:54:49 2003]
3a52ca12-2ff7-11d7-8dd2-6b61626f6f6d [Fri Feb 21 15:54:51 2003]
3a5450b2-2ff7-11d7-8dd2-6b61626f6f6d [Fri Feb 21 15:54:51 2003]
3af799e8-2ff7-11d7-8dd2-6b61626f6f6d [Fri Feb 21 15:54:52 2003]
3af921a0-2ff7-11d7-8dd2-6b61626f6f6d [Fri Feb 21 15:54:52 2003]
$ perl list_routes.plx http://127.0.0.1:8000/kn/ /what/chat
http://127.0.0.1:8000/kn/who/anonymous/s/981587186063738/kn_journal [Fri =
Feb 21 15:54:00 2003]
http://127.0.0.1:8000/kn/who/anonymous/s/280702445264073/kn_journal [Fri =
Feb 21 15:55:10 2003]
$

NOTE: you must include a trailing '/' at the end of the router URI, as =
in the examples above.

   Adam


P.S. -- We're on for a PubSub User Group meeting at Left at =
Albuquerque's in downtown Palo Alto, on Tuesday February 25 at 8pm.  =
Next month, we'll do one further north, like in Redwood City.

[Mod-pubsub-developer] PUG meeting on Tuesday, Feb 25. Also, a few more cvs check-ins tonight.

[Adam R. <Ad...@Kn...>]

We want to start a pattern of Pubsub User Group (PUG) meetings the last =
Tuesday of every month.  Unfortunately, this Tuesday I have an event I =
need to attend from 5pm-8pm at Blue Chalk Cafe in downtown Palo Alto.  =
How do you folks feel about an 8pm-10pm PUG meeting somewhere in =
downtown Palo Alto on Tuesday?  I'm thinking the Left-at-Albuquerue in =
honor of our New Mexican PUGgers... Thoughts?


Also, a couple of cvs check-ins tonight:

   1. Moved wakeup.py into the python_pubsub directory, so it can be =
used by apps that use pubsublib.py .
   2. Removed python_pubsub helper libraries from the kn_apps/sitewatch =
directory, and fixed sitewatch to use python_pubsub.

If you're using pubsub.cgi, you will need to update to the latest =
version that fixes a longstanding bug, if you want to use the =
just-checked-in pubsublib.py client.  A good reason to do a "cvs update =
-Pd" on your checkout.  I'll make a new tarball for download soon, but =
first I want to put in an index.html at the top level.  (There are also =
several cxx_pubsub changes that need to be committed when we get a round =
tuit.)

Top to-do-list of issues with pubsublib.py client library and pubsub.py =
server library:

   1. pubsublib.py test suite based on the JavaScript PubSub Library =
test suite.
   2. ping.py sample app.
   3. chat.py sample app.
  *4. Modify pubsub.py to have off-host routes, using pubsublib.py's =
HTTPUserAgent.
  *5. kn_content_transform (related to off-host routes).
  *6. SSL support in pubsub.py and pubsublib.py.
  *7. Username/password authentication in pubsub.py and pubsublib.py.
   8. cleaner.py to remove expired events from a pubsub server =
(pubsub.py or pubsub.cgi).
   9. bridge.py to form server federations.
  10. Merge pubsub.py and pubsublib.py to create a prototype "munchkin =
clerver".  (A clever munchkin??)

Starred items are ones needed for "feature completeness" of the Python =
PubSub client and server... items 9 and 10 really excite me but I have =
to eat my broccoli first...

   Adam

[Mod-pubsub-developer] New check-ins in the python_pubsub directory.

[Adam R. <Ad...@Kn...>]

Just did a CVS commit of asynchttp.py (from another sourceforge =
project), plus the newly-written-by-us scheduler.py and pubsublib.py, =
into the mod_pubsub/python_pubsub directory.  As a result, python_pubsub =
now contains a working pubsub standalone server and embeddable client =
library.

Also, earlier this evening checked in a patch to pubsub.cgi that fixed =
posting absolute url for topic with the notify method.  Added a new test =
to cgi-bin/pubsub_test.cgi to check for this.

For those folks with CVS, you can do an update; I'll put together =
another tarball of the whole distribution later this week.

   -- Adam



P.S. -- Below is the preamble to pubsublib.py .

    pubsublib.py -- PubSub Python Client Library
    for use with pubsub.py & mod_pubsub

    This is a simple Python client library for use in event-driven
    programs.  It can publish, subscribe, and unsubscribe in (more
    or less) the same way the PubSub JavaScript Library does.

    Phil Harris' kn_python is available at =
http://cvs.developer.knownow.com/ .

    Two ways in which pubsublib.py differs from kn_python:

       1. The interface is URL based.
       2. All operations are in a single thread and nonblocking,
          using asyncore.

    Version 1.0 -- February 18, 2003.  Initial implementation.
    Works fine on Debian GNU Linux 3.0 with Python 2.1.3.

    Known Issues:

       1. No unit test suite.  We plan to port the JavaScript PubSub
          Library unit test suite to test this implementation.

       2. Lots of FIXME and TODO things left -- see source code =
comments.        =20

[Mod-pubsub-developer] RE: Twisted Internet -- worth investigating as an asyncore.py replacement?

[Andrew A. <aa...@me...>]

Asyncore: good.
Twisted: better.
Nutter-butter: best.
A.


-----Original Message-----
From: Adam Rifkin [mailto:Ad...@Kn...]
Sent: Sunday, February 16, 2003 3:22 PM
To: mod...@li...
Subject: Twisted Internet -- worth investigating as an asyncore.py
replacement?


Regarding python_pubsub: the more we play with asyncore.py, the more we
think it's worth investigating replacing asyncore with the Twisted framework
from Twisted Matrix.  Kragen, is there a reason you didn't go with this
library?  Was it even available in 2001?  Does it actually work now, in
February 2003?

    http://twistedmatrix.com/

Twisted Matrix Laboratories is a distributed group of open-source developers
working on Twisted, an event-driven networking framework written in Python
and licensed under the LGPL. Twisted supports TCP, UDP, SSL/TLS, multicast,
Unix sockets, a large number of protocols (including HTTP, NNTP, SSH, IRC,
FTP, and others), and much more.

This would give us much cleaner multiprotocol handling.  It has some very
useful abstractions that we don't have yet, such as a non-polling Reactor.

  -- Adam



P.S. -- They have a bunch of neat things, like an ORB, but the most
interesting part is Twisted Internet:

    http://twistedmatrix.com/products/internet

Overview and Features

Twisted is an asynchronous networking framework. It provides the tools that
you need to write client and server networking applications that don't
require a large number of threads to scale.

Reactor-based Main Loop

Twisted Internet uses the Reactor Pattern for its event architecture. There
are many pluggable implementations of the reactor for integrating with
various toolkits and other event loops (more information can be found in the
Reactor Basics document).

GUI Integration

GUI integration: Our Reactor design is "pluggable", allowing us to integrate
various toolkits into Twisted. You can run servers or clients in a Twisted
process and not have to worry about threading or conflicting event loops.
What this means is that you can have a client or server running in a GUI, so
implementing a user interface for your Twisted-based applications is easy!
Currently Twisted integrates with PyGTK, WxPython, Tkinter, PyQT, and even a
threaded Jython event loop.

Asynchronous Programming Utilities

Twisted provides a number of utilities for simplifying common asynchronous
programming idioms. This includes tools for managing thread pools and
support for 'deferred' operations which will execute in the future, and
without blocking the main process.

Persistent Servers

Servers can be persistent; there need not be configuration files, only
pickles of the state of the server as you run it last.

twistd - A Built-in Daemon

'twistd' is the Twisted Daemon. It is a simple tool designed in a UNIX and
command-line friendly way; however it is portable to many environments
(including Win32 and Jython). It can take care of many common tasks for you
including:

1. Daemonization on UNIX platforms, with support for services on Windows
NT/2000/XP in the future.
2. Logging.
3. Loading and execution of serialized Twisted application objects in any of
the serialization formats supported by Twisted.
4. Saving a currently running Twisted application to a serialized object to
be loaded by twistd in the future.

Wide Ranging Protocol Support

We are replacing various UNIX daemons as we go along; all of these services
are designed to be friendly to other Python code and the Twisted framework.
This includes the HTTP, IRC, POP3, SMTP, and Telnet implementations, which
are fairly good examples of where we plan to take this. More information
about the protocol support is available on Twisted Protocols.

Competition: Asyncore

Asyncore is a asynchronous networking library that is part of the Python
standard library. It is a partial solution which mainly provides low-level
networking support, leaving out many pieces of an industrial-strength
asynchronous framework. Asyncore is also only being minimally maintained.

Here is a checklist of features for Twisted and Asyncore:

Feature                             Twisted Asyncore
TCP                                   Yes     Yes
UDP                                   Yes     Yes
SSL                                   Yes      No
Event scheduling                      Yes      No
Thread integration                    Yes      No
Win32 event-loop support              Yes      No
GUI integration                       Yes      No
Separation of Protocol and Transport  Yes      No
"echo" server implementation        5 lines 20 lines

Re: [Mod-pubsub-developer] We need a PubSub.widget for Konfabulator (OS/X)

[Gregory B. <gre...@ya...>]

Scott,

	You hit the nail on the head.  A core ECMAScript microserver sans any 
browser-specific integration at the bottom turtle (trying to sound like 
Rohit here).  Then layers for things like browsers, etc wrap around 
that.

so, like how soon?  ;-)

-greg

On Sunday, February 16, 2003, at 06:22  PM, Scott Andrew LePera wrote:

> Yeah, I thought the same thing when I saw the Konfabulator stuff 
> (although I'm not very familiar with the environment; it's JavaScript 
> runtime, with hooks into OSX? Like HTA/XUL?)
>
> Anyway, it would be kinda cool to have a core ECMAScript microserver 
> sans any browser-specific integration. A large part of of the KN JS 
> connector is dedicated to handling interaction with the browser 
> windows. We have the beginnings of a plug-in transport API, but it's 
> still tightly bound to the window model (IOW, any alternate transport 
> has to mimic window properties and methods so as not to break 
> anything).
>
> So, if we had a core that provided a more generic transport interface, 
> it could be more easily ported to similar-yet-different environments 
> like Flash ActionScript, Konfabulator and browsers with differing 
> object models.
>
> -- Scott
>
> Gregory Burd wrote:
>> http://www.konfabulator.com/
>> http://kmirror.deskmod.com/info/
>> Something that takes part of the JavaScript microserver and gives me 
>> the ability to pub/sub in a widget.
>> I guess this begs the question, why not split the JavaScript 
>> microserver into two pieces, the part that is requires only a 
>> JavaScript runtime and the part that requires a JavaScript runtime 
>> within a browser (all the fancy browser stuff).  Or maybe its easier 
>> to start from scratch and write a pubsub.js "library" for use in 
>> JavaScript based projects.
>> With that in hand people could create fun and creative widgets for 
>> OS/X desktops.  Things that do much the same thing as what we've done 
>> for HTML/Browsers, but without that crazy browser in the middle.
>> thoughts?
>> -greg
>> PS:  How long until I can use Safari with the JavaScript microserver?

Re: [Mod-pubsub-developer] We need a PubSub.widget for Konfabulator (OS/X)

[Scott A. L. <sc...@sc...>]

Yeah, I thought the same thing when I saw the Konfabulator stuff 
(although I'm not very familiar with the environment; it's JavaScript 
runtime, with hooks into OSX? Like HTA/XUL?)

Anyway, it would be kinda cool to have a core ECMAScript microserver 
sans any browser-specific integration. A large part of of the KN JS 
connector is dedicated to handling interaction with the browser windows. 
We have the beginnings of a plug-in transport API, but it's still 
tightly bound to the window model (IOW, any alternate transport has to 
mimic window properties and methods so as not to break anything).

So, if we had a core that provided a more generic transport interface, 
it could be more easily ported to similar-yet-different environments 
like Flash ActionScript, Konfabulator and browsers with differing object 
models.

-- Scott

Gregory Burd wrote:
> http://www.konfabulator.com/
> http://kmirror.deskmod.com/info/
> 
> Something that takes part of the JavaScript microserver and gives me the 
> ability to pub/sub in a widget.
> 
> I guess this begs the question, why not split the JavaScript microserver 
> into two pieces, the part that is requires only a JavaScript runtime and 
> the part that requires a JavaScript runtime within a browser (all the 
> fancy browser stuff).  Or maybe its easier to start from scratch and 
> write a pubsub.js "library" for use in JavaScript based projects.
> 
> With that in hand people could create fun and creative widgets for OS/X 
> desktops.  Things that do much the same thing as what we've done for 
> HTML/Browsers, but without that crazy browser in the middle.
> 
> thoughts?
> 
> -greg
> 
> PS:  How long until I can use Safari with the JavaScript microserver?
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Mod-pubsub-developer mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
> 
> 

Re: [Mod-pubsub-developer] Twisted Internet -- worth investigating as an asyncore.py replacement?

[Chris O. <co...@dy...>]

Twisted wasn't visible in 2001, at leat not to me.  They have come a long
way (and it sounds like I need to look a their code again).  They have
good buzz, and they do provide a lot of useful services, but I have to
admit that the comparison is a little off-putting to me...  My objection
is that they're comparing apples to oranges; asyncore was never intended
to provide all of the services they list, and calling it 'minimally
maintained' is making a failing out of a virtue - it works just fine, so
it hasn't needed much work, and recently got some polish from Guido, if
memory serves.

That said, the Twisted guys are smart, and the code looks pretty good. I'm
going to take another look, but I wouldn't be surprised if I use it
instead of asyncore (or asynchat.py, which is what I tend to use instead
of raw asyncore).

    /cco

[Mod-pubsub-developer] Twisted Internet -- worth investigating as an asyncore.py replacement?

[Adam R. <Ad...@Kn...>]

Regarding python_pubsub: the more we play with asyncore.py, the more we =
think it's worth investigating replacing asyncore with the Twisted =
framework from Twisted Matrix.  Kragen, is there a reason you didn't go =
with this library?  Was it even available in 2001?  Does it actually =
work now, in February 2003?

    http://twistedmatrix.com/

Twisted Matrix Laboratories is a distributed group of open-source =
developers working on Twisted, an event-driven networking framework =
written in Python and licensed under the LGPL. Twisted supports TCP, =
UDP, SSL/TLS, multicast, Unix sockets, a large number of protocols =
(including HTTP, NNTP, SSH, IRC, FTP, and others), and much more.

This would give us much cleaner multiprotocol handling.  It has some =
very useful abstractions that we don't have yet, such as a non-polling =
Reactor.

  -- Adam



P.S. -- They have a bunch of neat things, like an ORB, but the most =
interesting part is Twisted Internet:

    http://twistedmatrix.com/products/internet

Overview and Features

Twisted is an asynchronous networking framework. It provides the tools =
that you need to write client and server networking applications that =
don't require a large number of threads to scale.=20

Reactor-based Main Loop

Twisted Internet uses the Reactor Pattern for its event architecture. =
There are many pluggable implementations of the reactor for integrating =
with various toolkits and other event loops (more information can be =
found in the Reactor Basics document).=20

GUI Integration

GUI integration: Our Reactor design is "pluggable", allowing us to =
integrate various toolkits into Twisted. You can run servers or clients =
in a Twisted process and not have to worry about threading or =
conflicting event loops. What this means is that you can have a client =
or server running in a GUI, so implementing a user interface for your =
Twisted-based applications is easy! Currently Twisted integrates with =
PyGTK, WxPython, Tkinter, PyQT, and even a threaded Jython event loop.

Asynchronous Programming Utilities

Twisted provides a number of utilities for simplifying common =
asynchronous programming idioms. This includes tools for managing thread =
pools and support for 'deferred' operations which will execute in the =
future, and without blocking the main process.=20

Persistent Servers

Servers can be persistent; there need not be configuration files, only =
pickles of the state of the server as you run it last.

twistd - A Built-in Daemon

'twistd' is the Twisted Daemon. It is a simple tool designed in a UNIX =
and command-line friendly way; however it is portable to many =
environments (including Win32 and Jython). It can take care of many =
common tasks for you including:=20

1. Daemonization on UNIX platforms, with support for services on Windows =
NT/2000/XP in the future.=20
2. Logging.=20
3. Loading and execution of serialized Twisted application objects in =
any of the serialization formats supported by Twisted.=20
4. Saving a currently running Twisted application to a serialized object =
to be loaded by twistd in the future.=20

Wide Ranging Protocol Support

We are replacing various UNIX daemons as we go along; all of these =
services are designed to be friendly to other Python code and the =
Twisted framework. This includes the HTTP, IRC, POP3, SMTP, and Telnet =
implementations, which are fairly good examples of where we plan to take =
this. More information about the protocol support is available on =
Twisted Protocols.=20

Competition: Asyncore

Asyncore is a asynchronous networking library that is part of the Python =
standard library. It is a partial solution which mainly provides =
low-level networking support, leaving out many pieces of an =
industrial-strength asynchronous framework. Asyncore is also only being =
minimally maintained.=20

Here is a checklist of features for Twisted and Asyncore:=20

Feature                             Twisted Asyncore=20
TCP                                   Yes     Yes=20
UDP                                   Yes     Yes=20
SSL                                   Yes      No=20
Event scheduling                      Yes      No=20
Thread integration                    Yes      No=20
Win32 event-loop support              Yes      No=20
GUI integration                       Yes      No=20
Separation of Protocol and Transport  Yes      No=20
"echo" server implementation        5 lines 20 lines=20

[Mod-pubsub-developer] We need a PubSub.widget for Konfabulator (OS/X)

[Gregory B. <gre...@ya...>]

http://www.konfabulator.com/
http://kmirror.deskmod.com/info/

Something that takes part of the JavaScript microserver and gives me 
the ability to pub/sub in a widget.

I guess this begs the question, why not split the JavaScript 
microserver into two pieces, the part that is requires only a 
JavaScript runtime and the part that requires a JavaScript runtime 
within a browser (all the fancy browser stuff).  Or maybe its easier to 
start from scratch and write a pubsub.js "library" for use in 
JavaScript based projects.

With that in hand people could create fun and creative widgets for OS/X 
desktops.  Things that do much the same thing as what we've done for 
HTML/Browsers, but without that crazy browser in the middle.

thoughts?

-greg

PS:  How long until I can use Safari with the JavaScript microserver?

[Mod-pubsub-developer] RE: pubsub.py now works with JavaScript applications (such as those in kn_apps)

[Adam R. <Ad...@Kn...>]

Additional comments...

Actually, running pubsub_test.cgi from the command line has two other =
server-side unit tests that don't work:
   4. content transform: FAILED (Event payload wrong: Kragen Sittler at =
./pubsub_test.cgi line 916.)
   5. evil characters in route-to URL: FAILED (bad result routing on =
http://.../kn/what/apps/pubsub_test.cgi/1045279259_11347_1/evil_character=
s_in_route_to_URL/1 -> =
http://.../kn/what/apps/pubsub_test.cgi/1045279259_11347_1/evil_character=
s_in_route_to_URL/2/../flurb: HTTP/1.0 500 Internal Server Error

Diagnosis for #4 is that off-host routes aren't working properly.

Diagnosis for #5 is that pubsub.py does not suffer from the "event =
naming bug" that
makes this test necessary for security in pubsub.cgi -- therefore, not =
really a bug here.

So to fix #1 and #4 below, off-host routes are needed in pubsub.py.  =
Probably the
fastest way to get there is to (finally) start building the client =
library, pubsublib.py :
      1. Write a tunnel packet parser.
      2. Write a simple response format decoder.
      3. Implement asynchronous http post, which includes request queue.
      4. Implement publish.
      5. Implement subscribe, which includes dispatcher.
      6. Implement unsubscribe, which is a recast of subscribe.
      7. Implement connect and disconnect, to robustify subscribe.

Other interesting items once that is done:
      1. Python Tkinter Ping client.
      2. Python Tkinter Chat client.
      3. Off-host routes in pubsub.py.
      4. Modify sitewatch app to use pubsublib.py.
         Note: can also modify it to use ping2's performance and/or =
/kn_statistics.
      5. Make a reaper.py which removes expired events.
         Note: this client can work for both pubsub.py and pubsub.cgi.

I'm also thinking about checking some performance testing tools into the =
distribution.

That's all for now.  Much to think about.

Elvis is everywhere,
  Adam



-----Original Message-----
From: Adam Rifkin=20
Sent: Friday, February 14, 2003 7:12 PM
To: 'mod...@li...'
Subject: pubsub.py now works with JavaScript applications (such as those
in kn_apps)


I checked some changes into CVS (but haven't yet made a new distribution =
tarball -- will do that next week) that fix pubsub.py so that it now =
properly vends live JavaScript applications.

This means that you can run pubsub.py as a standalone PubSub server, =
rather than do all the shenanigans of getting pubsub.cgi and mod_perl =
working with your Apache server.

As a result, all of the unit tests vended from kn_apps/test now work =
with pubsub.py (except the call to cgi-bin/pubsub_test.cgi since =
pubsub.py still has no cgi support).

Running pubsub_test.cgi from the command line, I see that all of the 55 =
server-side unit test work except:
   1. kn_time_t on forwarded HTTP posts: FAILED (Didn't get a connection =
at ./pubsub_test.cgi line 1293.
   2. replay of events: FAILED =
(http://.../kn/what/apps/pubsub_test.cgi/1045277635_11338_1/replay_of_eve=
nts/2/kn_journal: Didn't get expected event, =
kn_id=3D1045277683_11338_293 at ./pubsub_test.cgi line 1457)
   3. replay of events with warp factor: FAILED (Got bad =
do_method=3Dreplay result HTTP/1.0 400 (Bad Request) Method Not =
Supported

I'm guessing #1 means off host routes are unimplemented, and #2 and #3 =
mean do_method=3Dreplay is unimplemented.  Otherwise, looks like it's =
feature-complete with pubsub.py ...

In fact, even vending cookie_auth.js seems to work for lightweight =
authentication.

Changes made to the cvs check-in:
   1. Modified testlist.html to use mod_pubsub terminology.
   2. Modified pubsub.py so that do_method's have the right knroot.
   3. Modified pubsub.py so that urlroot is removed from the beginning =
of a topic.

So for those of y'all who want an all-events-in-memory, standalone =
PubSub server, look into using pubsub.py .  (Also note that it has 34 =
items on its "TODO" list before we believe it to be feature-complete =
with pubsub.cgi ... :)

Elvis has entered the building,
   Adam

[Mod-pubsub-developer] pubsub.py now works with JavaScript applications (such as those in kn_apps)

[Adam R. <Ad...@Kn...>]

I checked some changes into CVS (but haven't yet made a new distribution =
tarball -- will do that next week) that fix pubsub.py so that it now =
properly vends live JavaScript applications.

This means that you can run pubsub.py as a standalone PubSub server, =
rather than do all the shenanigans of getting pubsub.cgi and mod_perl =
working with your Apache server.

As a result, all of the unit tests vended from kn_apps/test now work =
with pubsub.py (except the call to cgi-bin/pubsub_test.cgi since =
pubsub.py still has no cgi support).

Running pubsub_test.cgi from the command line, I see that all of the 55 =
server-side unit test work except:
   1. kn_time_t on forwarded HTTP posts: FAILED (Didn't get a connection =
at ./pubsub_test.cgi line 1293.
   2. replay of events: FAILED =
(http://.../kn/what/apps/pubsub_test.cgi/1045277635_11338_1/replay_of_eve=
nts/2/kn_journal: Didn't get expected event, =
kn_id=3D1045277683_11338_293 at ./pubsub_test.cgi line 1457)
   3. replay of events with warp factor: FAILED (Got bad =
do_method=3Dreplay result HTTP/1.0 400 (Bad Request) Method Not =
Supported

I'm guessing #1 means off host routes are unimplemented, and #2 and #3 =
mean do_method=3Dreplay is unimplemented.  Otherwise, looks like it's =
feature-complete with pubsub.py ...

In fact, even vending cookie_auth.js seems to work for lightweight =
authentication.

Changes made to the cvs check-in:
   1. Modified testlist.html to use mod_pubsub terminology.
   2. Modified pubsub.py so that do_method's have the right knroot.
   3. Modified pubsub.py so that urlroot is removed from the beginning =
of a topic.

So for those of y'all who want an all-events-in-memory, standalone =
PubSub server, look into using pubsub.py .  (Also note that it has 34 =
items on its "TODO" list before we believe it to be feature-complete =
with pubsub.cgi ... :)

Elvis has entered the building,
   Adam

[Mod-pubsub-developer] How to keep kn_events small. Plus, bug in mod_pubsub 0.5 distribution.

[Adam R. <Ad...@Kn...>]

Hi Yoz,

Thanks for finding the bug in the documentation; I have checked in a new =
INSTALL that walks people through running the Makefile to create =
pubsub.js and kn_events.  This will be bundled with the next tarball =
when we get around to making the next tarball.

Thanks for playing -- I think that's what we're all doing at =
mod-pubsub-developer!  Please do let us know if you're doing anything =
interesting.

As for questions, please ask the list so in case I'm busy someone else =
can answer.

As for the question of how to keep kn_events small, I have three =
techniques:

   1. Publish events with kn_expires header set to a low value.  The =
disadvantage to this technique is that the Perl pubsub server uses "lazy =
evaluation" in determining when to remove things, so it may not remove =
things as quickly as desired.

   2. Use the Python pubsub server instead.  Unfortunately, we don't =
have it working with the JavaScript applications yet, so this technique =
might take a while.

   3. Manually delete files or directories within kn_events.  This can =
be automated.  This is the technique I most often use.

Keep asking questions, please... it's the only way we're gonna make the =
system better...

Thanks,
   Adam


-----Original Message-----
From: Yoz Grahame [mailto:yo...@yo...]
Sent: Wednesday, February 12, 2003 7:28 PM
To: Adam Rifkin
Cc: kr...@po...; Ben Sittler
Subject: Re: Bug in mod_pubsub 0.5 distribution

Adam Rifkin wrote:
> Hi Yoz,
>=20
> Thanks for taking the time to check out the distribution.
>=20
> Alternatively, you can run the Makefile at the top level and it will
> generate a compressed pubsub.js for you -- much smaller for faster
> delivery over the Internet.
>=20
> Come to think of it, you'll need to run the Makefile to create a
> kn_events directory anyway.

Ah - I didn't know that. I did it by hand.
Actually, the INSTALL file doesn't mention the Makefile.

> If you tell me what your Sourceforge account is, I'll add you to the
> "developers" group.

Don't have one! It's okay, I'm mainly playing, though I have an idea for =

a chat client that PS would be good for. I'll let you know if it gets=20
anywhere interesting (though I'll probably be plaguing you with dev=20
questions before then)

Actually, here's my first dev question: what's the best way of stopping=20
my kn_events dir from getting massive? Do the files in it get physically =

removed in accordance with the expiry dates on the messages?

-- Yoz

[Mod-pubsub-developer] Pushlets don't allow multiple subscriptions.

[Adam R. <Ad...@Kn...>]

It sounds like, from the conversation I'm forwarding below, that with =
Pushlets you can't subscribe to two topics at once.  (On the other hand, =
they do have hierarchical subscriptions, meaning you can subscribe to =
all the topics in a subtree.)

To do two subscriptions, Pushlets require you to make two client-side =
connections; I guess this has significant problems scaling.

  Adam


-----Original Message-----
From: pu...@ya... [mailto:pu...@ya...]
Sent: Thursday, February 13, 2003 8:40 AM
To: pu...@ya...
Subject: [pushlet] Digest Number 190


To Post a message, send it to:   pu...@eG...
To Unsubscribe, send a blank message to: pus...@eG...
------------------------------------------------------------------------

There are 2 messages in this issue.

Topics in this digest:

      1. Re: Subscribing to more than 1 subject?
           From: Raymond Pau <rp...@ya...>
      2. Re: Subscribing to more than 1 subject?
           From: Just van den Broecke <ju...@ju...>


________________________________________________________________________
________________________________________________________________________

Message: 1
   Date: Wed, 12 Feb 2003 17:32:08 -0800 (PST)
   From: Raymond Pau <rp...@ya...>
Subject: Re: Subscribing to more than 1 subject?

Ji Just,

Just curious, what happen if I need to subscribe 2
separate unrelated subjects?

/activity
/iostatus

Do I have to resort to subscribing to '/' and
filtering off what I don't want?

TIA.

Raymond Pau

--- Just van den Broecke <ju...@ju...> wrote:
> Hi Raymond,
>=20
> Yes in your case you only need to subscribe to
> '/activity'
>=20
> Subscribing to a subject automatically subscribes a
> client to all child=20
> subjects. For example subscribing to '/' subscribes
> to _all_ subjects in=20
> the system (see also the pushlet examples page on
> http://www.pushlets.com).
>=20
> best,
>=20
> Just
>=20
> Raymond Pau  wrote:
> > Hi there,
> >=20
> > It is possible to subscribe to more than 1 subject
> for
> > each connection?
> >=20
> > I have the following tree structure:
> > /activity
> > /activity/alarm
> > /activity/timeclock
> >=20
> > I would like to have a jsp page that will update
> > automatically when an event occurs in anyone of
> the
> > above subjects but not for others.
> >=20
> > Regards,
> >=20
> > Raymond Pau
> >=20
> >=20
> >=20
> > To Post a message, send it to: =20
> pu...@eG...
> > To Unsubscribe, send a blank message to:
> pus...@eG...=20
> >=20
> > Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/

__________________________________________________
Do you Yahoo!?
Yahoo! Shopping - Send Flowers for Valentine's Day
http://shopping.yahoo.com


________________________________________________________________________
________________________________________________________________________

Message: 2
   Date: Thu, 13 Feb 2003 10:01:04 +0000
   From: Just van den Broecke <ju...@ju...>
Subject: Re: Subscribing to more than 1 subject?

Hi Raymond,

Yes (in your case you have to filter).It is currently not possible to=20
subscribe to multiple subjects (although you may try on the client side=20
with two connections, each in a separate hidden frame and forwarding to=20
the same event catcher in the main page).
best,
Just

Raymond Pau wrote:
> Ji Just,
>=20
> Just curious, what happen if I need to subscribe 2
> separate unrelated subjects?
>=20
> /activity
> /iostatus
>=20
> Do I have to resort to subscribing to '/' and
> filtering off what I don't want?
>=20
> TIA.
>=20
> Raymond Pau
>=20
> --- Just van den Broecke <ju...@ju...> wrote:
>=20
>>Hi Raymond,
>>
>>Yes in your case you only need to subscribe to
>>'/activity'
>>
>>Subscribing to a subject automatically subscribes a
>>client to all child=20
>>subjects. For example subscribing to '/' subscribes
>>to _all_ subjects in=20
>>the system (see also the pushlet examples page on
>>http://www.pushlets.com).
>>
>>best,
>>
>>Just
>>
>>Raymond Pau  wrote:
>>
>>>Hi there,
>>>
>>>It is possible to subscribe to more than 1 subject
>>
>>for
>>
>>>each connection?
>>>
>>>I have the following tree structure:
>>>/activity
>>>/activity/alarm
>>>/activity/timeclock
>>>
>>>I would like to have a jsp page that will update
>>>automatically when an event occurs in anyone of
>>
>>the
>>
>>>above subjects but not for others.
>>>
>>>Regards,
>>>
>>>Raymond Pau
>>>
>>>
>>>
>>>To Post a message, send it to: =20
>>
>>pu...@eG...
>>
>>>To Unsubscribe, send a blank message to:
>>
>>pus...@eG...=20
>>
>>>Your use of Yahoo! Groups is subject to
>>
>>http://docs.yahoo.com/info/terms/
>=20
>=20
> __________________________________________________
> Do you Yahoo!?
> Yahoo! Shopping - Send Flowers for Valentine's Day
> http://shopping.yahoo.com
>=20
> To Post a message, send it to:   pu...@eG...
> To Unsubscribe, send a blank message to: =
pus...@eG...=20
>=20
> Your use of Yahoo! Groups is subject to =
http://docs.yahoo.com/info/terms/=20
>=20
>=20
>=20
>=20


--=20

--Just

Just van den Broecke          ju...@ju...
Just Objects B.V.             tel. +31 65 4268627
The Netherlands               http://www.justobjects.nl








________________________________________________________________________
________________________________________________________________________



Your use of Yahoo! Groups is subject to =
http://docs.yahoo.com/info/terms/=20

[Mod-pubsub-developer] New release (0.5). Plus, /kn, sitewatch, pubsub.py

[Adam R. <Ad...@Kn...>]

I just uploaded the 0.5 tarball to:
   =
https://sourceforge.net/project/showfiles.php?group_id=3D66293&release_id=
=3D139123

Changes:

   1. Made /kn style work with all of the kn_apps
   2. Added "sitewatch" application in kn_apps, including sensor
   3. Some smaller modifications


Also, I wrote:
> There's no KNN anymore.  We should get modpubsub.com up and
> running soon, to serve as the *new* KN Network.  (Yes, with a weblog,
> too.  Joyce just loves it when I volunteer her for things. :)

Mike responded:
> Would you like to use xmlrouter.net? I also have topiczero.com &
> some others...

Nifty!  I don't see a reason why we shouldn't run multiple sites.
Perhaps even bridge them... thoughts?

Also, I wrote:
> For example, do_method=3Dwhoami was added to pubsub.py ... of course,
> when running pubsub.py I see that [pubsub.py] does not, at present,
> work.  Will need to go through it carefully to determine why.

Kragen responded:
> Last time I tried pubsub.py, it didn't work with any apps because the
> apps expect to find the event router at "../../cgi-bin/pubsub.cgi",
> while pubsub.py puts it at "/kn", the same place KnowNow's product
> does.  I've begun making the necessary changes to the apps and the
> Perl server to use "/kn" instead, but haven't finished.

By "haven't finished", you mean that apps can't find the auxiliary cgi's
and docs when they are set up to view /kn ?

Yesterday we modified all the kn_apps to use the /kn convention instead
of the ../../cgi-bin/pubsub.cgi convention, and they seem to work fine
with the (ever)"unfinished" Perl server, so I checked the changes into =
CVS.

I also checked in a new app called "sitewatch" which basically is just
a web server monitoring app.  The UI is neat (lifted from a previous =
dhtml
app we wrote internally for Xander's traffic monitor), but the sensor
is the reason we embarked on a new app.

The goal for the sitewatch app was to look into what it would take to
write an all-asynchronous Python networking app.  I found another =
project
on Sourceforge that hadn't been touched for years called asynchttp, =
which
provides some nice facilities for making asynchronous HTTP/1.1 calls.
Nice little library.

We wrote some other helper Classes -- asyncgeturl (asynchronous URL
GET), scheduler (asynchronous scheduler), and wakeup (a wakeup timer) --
and put them together to create sitewatch_sensor.py .  All of these
programs, along with the presentation logic in index.html and the
asynchttp.py code [it's Python-licensed], have been checked into CVS
in the mod_pubsub/kn_apps/sitewatch directory.  Feel free to check it =
out.

(Side note: sitewatch could also offer the beginning of a "webmaster
dashboard", especially if one were to add Fred's Apache log sensor
and feed it into one of Scott's ActivePanel front ends (available
in kn_apps/jscomponents ), as a second part of sitewatch.)

Now that we have a little experience building asynchronous networked
event loops in Python, what's next?  Well, we have a scheduler, an
async http library, and a wakeup timer, so the steps to building a
Python pubsub client library are:

      1. Implement a tunnel packet parser.
      2. Implement a simple response format decoder.
      3. Implement asynchronous http post, which includes request queue.
      4. Implement publish.
      5. Implement subscribe, which includes dispatcher.
      6. Implement unsubscribe, which is a recast of subscribe.
      7. Implement connect and disconnect, to robustify subscribe.

If we do it right, rather than struggle with asynchronous networking
Python primitives, we can focus on building apps using publish,
subscribe, and unsubscribe.  For example, the "sitewatch" app would
have been much easier to write if we had that.  (Then we could have
sitewatch "subscribe" to PubSub servers instead of pinging them with
a noop, to determine their status by publishing to the same topic
sitewatch would be subscribing to...)

The goal is a simple Python client for use in event-driven programs
that can publish and subscribe in (more or less) the same way the
PubSub JavaScript Library does.

Phil Harris' kn_python was written 18 months ago, and has much of the
right interface -- it's available at http://cvs.developer.knownow.com/ .

Two ways in which a newly written Python client library would differ
from kn_python:

       1. The interface is URL based.=20
       2. All operations are in a single thread and nonblocking,
          using the already-modified asyncore in the python_pubsub
          distribution.

Kragen also wrote:
> IIRC, pubsub_test.cgi showed that pubsub.py more or less worked,
> except for some features without which nearly all apps should still
> work (off-host routes, content-transform, content-filter, and event
> replay, IIRC), but it doesn't test the all-important JavaScript tunnel
> format at all.  It appears to me that the sending code for that format
> is in &PubSub::Topic::send, and the receiving code is split between
> the JavaScript parser in a browser and _kn_sendCallback in
> pubsub_raw.js.  Inspection of the code doesn't suggest that the
> formats have changed much --- except for nameU and valueU --- since I
> wrote pubsub.py in early 2001.  I'll try it out soon, but not tonight.

Ah, this explains why I thought it wasn't working -- I can't get it to
vend the (dhtml) kn_apps, so I figured pubsub.py was broken.

You looked in the right places for the JavaScript tunnel format.  Those
are, AFAIK, the reference implementation.

Ben says that actually, nameU and valueU are optional for ASCII data,
so yeah, formats have not changed really for more than two years.
This is actually a good thing; rather than things having gotten worse,
they stayed the same.  "Better" is a nice-to-have.  :)

   Adam

Re: [Mod-pubsub-developer] Another demo app needed

[<kr...@po...>]

Adam Rifkin writes:
> For example, do_method=whoami was added to pubsub.py ... of course,
> when running pubsub.py I see that [pubsub.py] does not, at present,
> work.  Will need to go through it carefully to determine why.

Last time I tried pubsub.py, it didn't work with any apps because the
apps expect to find the event router at "../../cgi-bin/pubsub.cgi",
while pubsub.py puts it at "/kn", the same place KnowNow's product
does.  I've begun making the necessary changes to the apps and the
Perl server to use "/kn" instead, but haven't finished.

IIRC, pubsub_test.cgi showed that pubsub.py more or less worked,
except for some features without which nearly all apps should still
work (off-host routes, content-transform, content-filter, and event
replay, IIRC), but it doesn't test the all-important JavaScript tunnel
format at all.  It appears to me that the sending code for that format
is in &PubSub::Topic::send, and the receiving code is split between
the JavaScript parser in a browser and _kn_sendCallback in
pubsub_raw.js.  Inspection of the code doesn't suggest that the
formats have changed much --- except for nameU and valueU --- since I
wrote pubsub.py in early 2001.  I'll try it out soon, but not tonight.

-- 
<kr...@po...>       Kragen Sitaker     <http://www.pobox.com/~kragen/>
Edsger Wybe Dijkstra died in August of 2002.  The world has lost a great
man.  See http://advogato.org/person/raph/diary.html?start=252 and
http://www.kode-fu.com/geek/2002_08_04_archive.shtml for details.