mod-pubsub-developer

[Mod-pubsub-developer] Non-blocking publishing in Python

[<mod...@li...>]

Hello:

I'm trying to use the apps/publish.py module as a starting point for writing my
own simple publishing lib to work with repubsub. The script runs fine on its
own, but whenever I try to embed it in other code, I seem to get blocking
issues and the CPU load spikes.

Inside the apps/publish.py script, I looked at the following method calls:

  asyncore.poll()
  scheduler.timeout()
  scheduler.run()

Now Python's asyncore does not have documentation on the poll function, although
in my interpreter, I do see a reference for poll. The two scheduler function
calls both pass in their implementations. If I comment out that loop, my
messages don't get published. So what's going on here? 

Ideally, I'd like to be able to use Twisted's Deferred mechanism to handle the
blocking issue, but I'm unsure the best way to approach it without heavily
modifying the code base.

Any help is greatly appreciated.

Thanks,
Brian

-- 
  ====
brian lee dae yung
biomimetic art and research
  mux space . com

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

Re: [Mod-pubsub-developer] Non-blocking publishing in Python

[<mod...@li...>]

Haven't heard anything back from the list regarding this question. If it is
because the question was too bone-headed, then there is no remedy. Otherwise,
the following information might be useful to someone else getting started.

There are two python client libraries: pubsublib.py and libkn/libkn.py.
According to the source, pubsublib.py is a non-blocking implementation, while
libkn.py is blocking. The examples in apps are built off of pubsublib, so they
should be non-blocking.

This seems to be true when running as a standalone script, but what about when I
want to embed? Is the logging package being used to help the debugging process
or are people using pdb?

Also, I read somewhere that the client is supposed to be using Twisted in the
near future. Is this still the case, and if so, what's the progress on it?

Thanks,
Brian


Quoting mod...@li...:

> Hello:
> 
> I'm trying to use the apps/publish.py module as a starting point for writing
> my
> own simple publishing lib to work with repubsub. The script runs fine on its
> own, but whenever I try to embed it in other code, I seem to get blocking
> issues and the CPU load spikes.
> 
> Inside the apps/publish.py script, I looked at the following method calls:
> 
>   asyncore.poll()
>   scheduler.timeout()
>   scheduler.run()
> 
> Now Python's asyncore does not have documentation on the poll function,
> although
> in my interpreter, I do see a reference for poll. The two scheduler function
> calls both pass in their implementations. If I comment out that loop, my
> messages don't get published. So what's going on here? 
> 
> Ideally, I'd like to be able to use Twisted's Deferred mechanism to handle
> the
> blocking issue, but I'm unsure the best way to approach it without heavily
> modifying the code base.
> 
> Any help is greatly appreciated.
> 
> Thanks,
> Brian
> 
> -- 
>   ====
> brian lee dae yung
> biomimetic art and research
>   mux space . com
> 
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
> 
> 
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Mod-pubsub-developer mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
> 


-- 
  ====
brian lee dae yung
biomimetic art and research
  mux space . com

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

Re: [Mod-pubsub-developer] Non-blocking publishing in Python

[<mod...@li...>]

Hi Brian,

The question was not bone-headed at all. It's just that sometimes our
Python guy goes into  { while 1: sleep } mode. ;)

Both of those client libraries were experimental prototypes, as is
probably clear from reading the source. Plus, I was kind of learning
Python as we wrote them.

I'm Cc'ing Ben in case he knows the answer to your embedding question.

I do know that no one is currently working to port the client library
to Twisted; right now all the energy and effort is going into
rewriting the server using Twisted, and hopefully we can learn from
that to do a client rewrite sometime later. If you're feeling
ambitious, feel free to take a crack at doing it with Twisted's
Deferred mechanism...

   -- Adam



On Apr 5, 2005 4:13 AM,
mod...@li...
<mod...@li...> wrote:
> Haven't heard anything back from the list regarding this question. If it is
> because the question was too bone-headed, then there is no remedy. Otherwise,
> the following information might be useful to someone else getting started.
> 
> There are two python client libraries: pubsublib.py and libkn/libkn.py.
> According to the source, pubsublib.py is a non-blocking implementation, while
> libkn.py is blocking. The examples in apps are built off of pubsublib, so they
> should be non-blocking.
> 
> This seems to be true when running as a standalone script, but what about when I
> want to embed? Is the logging package being used to help the debugging process
> or are people using pdb?
> 
> Also, I read somewhere that the client is supposed to be using Twisted in the
> near future. Is this still the case, and if so, what's the progress on it?
> 
> Thanks,
> Brian
> 
> 
> Quoting mod...@li...:
> 
> > Hello:
> >
> > I'm trying to use the apps/publish.py module as a starting point for writing
> > my
> > own simple publishing lib to work with repubsub. The script runs fine on its
> > own, but whenever I try to embed it in other code, I seem to get blocking
> > issues and the CPU load spikes.
> >
> > Inside the apps/publish.py script, I looked at the following method calls:
> >
> >   asyncore.poll()
> >   scheduler.timeout()
> >   scheduler.run()
> >
> > Now Python's asyncore does not have documentation on the poll function,
> > although
> > in my interpreter, I do see a reference for poll. The two scheduler function
> > calls both pass in their implementations. If I comment out that loop, my
> > messages don't get published. So what's going on here?
> >
> > Ideally, I'd like to be able to use Twisted's Deferred mechanism to handle
> > the
> > blocking issue, but I'm unsure the best way to approach it without heavily
> > modifying the code base.
> >
> > Any help is greatly appreciated.
> >
> > Thanks,
> > Brian
> >
> > --
> >   ====
> > brian lee dae yung
> > biomimetic art and research
> >   mux space . com
> >
> > ----------------------------------------------------------------
> > This message was sent using IMP, the Internet Messaging Program.
> >
> >
> >
> > -------------------------------------------------------
> > SF email is sponsored by - The IT Product Guide
> > Read honest & candid reviews on hundreds of IT Products from real users.
> > Discover which products truly live up to the hype. Start reading now.
> > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> > _______________________________________________
> > Mod-pubsub-developer mailing list
> > Mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
> >
> 
> --
>   ====
> brian lee dae yung
> biomimetic art and research
>   mux space . com
> 
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Mod-pubsub-developer mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
>

Re: [Mod-pubsub-developer] Non-blocking publishing in Python

[<mod...@li...>]

Adam:

> The question was not bone-headed at all. It's just that sometimes our
> Python guy goes into  { while 1: sleep } mode. ;)

In that case, I'll continue to poll() the list. Hopefully the results aren't too
asynchronous!


> Both of those client libraries were experimental prototypes, as is
> probably clear from reading the source. Plus, I was kind of learning
> Python as we wrote them.
> 
> I'm Cc'ing Ben in case he knows the answer to your embedding question.

So is anyone using the Python clients for any interesting applications? I've
evaluated some other asynchronous messaging technologies, and mod/re-pubsub
looks the most promising w.r.t. Python and Internet-wide messaging. It would be
great if anyone could share some insight into the embedding question, or how to
harness the non-blocking behavior of the pubsublib.


> I do know that no one is currently working to port the client library
> to Twisted; right now all the energy and effort is going into
> rewriting the server using Twisted, and hopefully we can learn from
> that to do a client rewrite sometime later. If you're feeling
> ambitious, feel free to take a crack at doing it with Twisted's
> Deferred mechanism...

How is the progress on the server side? Is the current version stable enough to
be making changes that would be compatible with the current server development?
I have some experience with Twisted, so I wouldn't mind taking a look, but I
would like to get a sense of the project road map so my work isn't wasted.

>    -- Adam

Brian


PS Are you Adam Rifkin or another Adam?


> On Apr 5, 2005 4:13 AM,
> mod...@li...
> <mod...@li...> wrote:
> > Haven't heard anything back from the list regarding this question. If it
> is
> > because the question was too bone-headed, then there is no remedy.
> Otherwise,
> > the following information might be useful to someone else getting started.
> > 
> > There are two python client libraries: pubsublib.py and libkn/libkn.py.
> > According to the source, pubsublib.py is a non-blocking implementation,
> while
> > libkn.py is blocking. The examples in apps are built off of pubsublib, so
> they
> > should be non-blocking.
> > 
> > This seems to be true when running as a standalone script, but what about
> when I
> > want to embed? Is the logging package being used to help the debugging
> process
> > or are people using pdb?
> > 
> > Also, I read somewhere that the client is supposed to be using Twisted in
> the
> > near future. Is this still the case, and if so, what's the progress on it?
> > 
> > Thanks,
> > Brian
> > 
> > 
> > Quoting mod...@li...:
> > 
> > > Hello:
> > >
> > > I'm trying to use the apps/publish.py module as a starting point for
> writing
> > > my
> > > own simple publishing lib to work with repubsub. The script runs fine on
> its
> > > own, but whenever I try to embed it in other code, I seem to get
> blocking
> > > issues and the CPU load spikes.
> > >
> > > Inside the apps/publish.py script, I looked at the following method
> calls:
> > >
> > >   asyncore.poll()
> > >   scheduler.timeout()
> > >   scheduler.run()
> > >
> > > Now Python's asyncore does not have documentation on the poll function,
> > > although
> > > in my interpreter, I do see a reference for poll. The two scheduler
> function
> > > calls both pass in their implementations. If I comment out that loop, my
> > > messages don't get published. So what's going on here?
> > >
> > > Ideally, I'd like to be able to use Twisted's Deferred mechanism to
> handle
> > > the
> > > blocking issue, but I'm unsure the best way to approach it without
> heavily
> > > modifying the code base.
> > >
> > > Any help is greatly appreciated.
> > >
> > > Thanks,
> > > Brian
> > >
> > > --
> > >   ====
> > > brian lee dae yung
> > > biomimetic art and research
> > >   mux space . com
> > >
> > > ----------------------------------------------------------------
> > > This message was sent using IMP, the Internet Messaging Program.
> > >
> > >
> > >
> > > -------------------------------------------------------
> > > SF email is sponsored by - The IT Product Guide
> > > Read honest & candid reviews on hundreds of IT Products from real users.
> > > Discover which products truly live up to the hype. Start reading now.
> > > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> > > _______________________________________________
> > > Mod-pubsub-developer mailing list
> > > Mod...@li...
> > > https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
> > >
> > 
> > --
> >   ====
> > brian lee dae yung
> > biomimetic art and research
> >   mux space . com
> > 
> > ----------------------------------------------------------------
> > This message was sent using IMP, the Internet Messaging Program.
> > 
> > -------------------------------------------------------
> > SF email is sponsored by - The IT Product Guide
> > Read honest & candid reviews on hundreds of IT Products from real users.
> > Discover which products truly live up to the hype. Start reading now.
> > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> > _______________________________________________
> > Mod-pubsub-developer mailing list
> > Mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
> >
> 
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Mod-pubsub-developer mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
> 


-- 
  ====
brian lee dae yung
biomimetic art and research
  mux space . com

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

Re: [Mod-pubsub-developer] Non-blocking publishing in Python

[<mod...@li...>]

In the interest of strengthening everyone's knowledge base, what did you 
compare against and what were your concerns?  I've noticed that 
xmlBlaster and Activ MQ both offer HTTP oriented methodologies for 
asynchronous notification.  But I am not sure how robust they are, nor 
how native/central to the design they are, nor how well they work in 
javascript contexts.

A.

mod...@li... wrote:

>Adam:
>
>  
>
>>The question was not bone-headed at all. It's just that sometimes our
>>Python guy goes into  { while 1: sleep } mode. ;)
>>    
>>
>
>In that case, I'll continue to poll() the list. Hopefully the results aren't too
>asynchronous!
>
>
>  
>
>>Both of those client libraries were experimental prototypes, as is
>>probably clear from reading the source. Plus, I was kind of learning
>>Python as we wrote them.
>>
>>I'm Cc'ing Ben in case he knows the answer to your embedding question.
>>    
>>
>
>So is anyone using the Python clients for any interesting applications? I've
>evaluated some other asynchronous messaging technologies, and mod/re-pubsub
>looks the most promising w.r.t. Python and Internet-wide messaging. It would be
>great if anyone could share some insight into the embedding question, or how to
>harness the non-blocking behavior of the pubsublib.
>
>
>  
>
>>I do know that no one is currently working to port the client library
>>to Twisted; right now all the energy and effort is going into
>>rewriting the server using Twisted, and hopefully we can learn from
>>that to do a client rewrite sometime later. If you're feeling
>>ambitious, feel free to take a crack at doing it with Twisted's
>>Deferred mechanism...
>>    
>>
>
>How is the progress on the server side? Is the current version stable enough to
>be making changes that would be compatible with the current server development?
>I have some experience with Twisted, so I wouldn't mind taking a look, but I
>would like to get a sense of the project road map so my work isn't wasted.
>
>  
>
>>   -- Adam
>>    
>>
>
>Brian
>
>
>PS Are you Adam Rifkin or another Adam?
>
>
>  
>
>>On Apr 5, 2005 4:13 AM,
>>mod...@li...
>><mod...@li...> wrote:
>>    
>>
>>>Haven't heard anything back from the list regarding this question. If it
>>>      
>>>
>>is
>>    
>>
>>>because the question was too bone-headed, then there is no remedy.
>>>      
>>>
>>Otherwise,
>>    
>>
>>>the following information might be useful to someone else getting started.
>>>
>>>There are two python client libraries: pubsublib.py and libkn/libkn.py.
>>>According to the source, pubsublib.py is a non-blocking implementation,
>>>      
>>>
>>while
>>    
>>
>>>libkn.py is blocking. The examples in apps are built off of pubsublib, so
>>>      
>>>
>>they
>>    
>>
>>>should be non-blocking.
>>>
>>>This seems to be true when running as a standalone script, but what about
>>>      
>>>
>>when I
>>    
>>
>>>want to embed? Is the logging package being used to help the debugging
>>>      
>>>
>>process
>>    
>>
>>>or are people using pdb?
>>>
>>>Also, I read somewhere that the client is supposed to be using Twisted in
>>>      
>>>
>>the
>>    
>>
>>>near future. Is this still the case, and if so, what's the progress on it?
>>>
>>>Thanks,
>>>Brian
>>>
>>>
>>>Quoting mod...@li...:
>>>
>>>      
>>>
>>>>Hello:
>>>>
>>>>I'm trying to use the apps/publish.py module as a starting point for
>>>>        
>>>>
>>writing
>>    
>>
>>>>my
>>>>own simple publishing lib to work with repubsub. The script runs fine on
>>>>        
>>>>
>>its
>>    
>>
>>>>own, but whenever I try to embed it in other code, I seem to get
>>>>        
>>>>
>>blocking
>>    
>>
>>>>issues and the CPU load spikes.
>>>>
>>>>Inside the apps/publish.py script, I looked at the following method
>>>>        
>>>>
>>calls:
>>    
>>
>>>>  asyncore.poll()
>>>>  scheduler.timeout()
>>>>  scheduler.run()
>>>>
>>>>Now Python's asyncore does not have documentation on the poll function,
>>>>although
>>>>in my interpreter, I do see a reference for poll. The two scheduler
>>>>        
>>>>
>>function
>>    
>>
>>>>calls both pass in their implementations. If I comment out that loop, my
>>>>messages don't get published. So what's going on here?
>>>>
>>>>Ideally, I'd like to be able to use Twisted's Deferred mechanism to
>>>>        
>>>>
>>handle
>>    
>>
>>>>the
>>>>blocking issue, but I'm unsure the best way to approach it without
>>>>        
>>>>
>>heavily
>>    
>>
>>>>modifying the code base.
>>>>
>>>>Any help is greatly appreciated.
>>>>
>>>>Thanks,
>>>>Brian
>>>>
>>>>--
>>>>  ====
>>>>brian lee dae yung
>>>>biomimetic art and research
>>>>  mux space . com
>>>>
>>>>----------------------------------------------------------------
>>>>This message was sent using IMP, the Internet Messaging Program.
>>>>
>>>>
>>>>
>>>>-------------------------------------------------------
>>>>SF email is sponsored by - The IT Product Guide
>>>>Read honest & candid reviews on hundreds of IT Products from real users.
>>>>Discover which products truly live up to the hype. Start reading now.
>>>>http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
>>>>_______________________________________________
>>>>Mod-pubsub-developer mailing list
>>>>Mod...@li...
>>>>https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
>>>>
>>>>        
>>>>
>>>--
>>>  ====
>>>brian lee dae yung
>>>biomimetic art and research
>>>  mux space . com
>>>
>>>----------------------------------------------------------------
>>>This message was sent using IMP, the Internet Messaging Program.
>>>
>>>-------------------------------------------------------
>>>SF email is sponsored by - The IT Product Guide
>>>Read honest & candid reviews on hundreds of IT Products from real users.
>>>Discover which products truly live up to the hype. Start reading now.
>>>http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
>>>_______________________________________________
>>>Mod-pubsub-developer mailing list
>>>Mod...@li...
>>>https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
>>>
>>>      
>>>
>>-------------------------------------------------------
>>SF email is sponsored by - The IT Product Guide
>>Read honest & candid reviews on hundreds of IT Products from real users.
>>Discover which products truly live up to the hype. Start reading now.
>>http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
>>_______________________________________________
>>Mod-pubsub-developer mailing list
>>Mod...@li...
>>https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
>>
>>    
>>
>
>
>  
>

Re: [Mod-pubsub-developer] Non-blocking publishing in Python

[<mod...@li...>]

Adam:

My basic criteria was finding a message-based system that was being actively
developed/supported with Python bindings. I looked at a number of open source
JMS implementations, most of which had no Python bindings. Activ MQ and Mom4J
were the two that had support for Python, but Activ MQ seems too
Windows-centric for serious use on Linux. Mom4J looked more promising, but
their implementation was incomplete and their user community was like a ghost
town. There's really no point in investing time in a dead project (unless it
has killer technology). When I discovered mod_pubsub and the REST-based systems
(been on my radar for a long time, but never delved into it) I was happy to
find out that they overcome the confinement limitations of JMS. At this point,
I'm ignorant of the security ramifications that this presents, which I'd like
to find out more. FWIW, Mom4J provided hooks for authentication, but no actual
implementation was present.

I'll probably be looking at some other REST systems, and I'll let you know the
outcome.

Brian


Quoting mod...@li...:

> In the interest of strengthening everyone's knowledge base, what did you 
> compare against and what were your concerns?  I've noticed that 
> xmlBlaster and Activ MQ both offer HTTP oriented methodologies for 
> asynchronous notification.  But I am not sure how robust they are, nor 
> how native/central to the design they are, nor how well they work in 
> javascript contexts.
> 
> A.
> 
> mod...@li... wrote:
> 
> >Adam:
> >
> >  
> >
> >>The question was not bone-headed at all. It's just that sometimes our
> >>Python guy goes into  { while 1: sleep } mode. ;)
> >>    
> >>
> >
> >In that case, I'll continue to poll() the list. Hopefully the results aren't
> too
> >asynchronous!
> >
> >
> >  
> >
> >>Both of those client libraries were experimental prototypes, as is
> >>probably clear from reading the source. Plus, I was kind of learning
> >>Python as we wrote them.
> >>
> >>I'm Cc'ing Ben in case he knows the answer to your embedding question.
> >>    
> >>
> >
> >So is anyone using the Python clients for any interesting applications?
> I've
> >evaluated some other asynchronous messaging technologies, and mod/re-pubsub
> >looks the most promising w.r.t. Python and Internet-wide messaging. It would
> be
> >great if anyone could share some insight into the embedding question, or how
> to
> >harness the non-blocking behavior of the pubsublib.
> >
> >
> >  
> >
> >>I do know that no one is currently working to port the client library
> >>to Twisted; right now all the energy and effort is going into
> >>rewriting the server using Twisted, and hopefully we can learn from
> >>that to do a client rewrite sometime later. If you're feeling
> >>ambitious, feel free to take a crack at doing it with Twisted's
> >>Deferred mechanism...
> >>    
> >>
> >
> >How is the progress on the server side? Is the current version stable enough
> to
> >be making changes that would be compatible with the current server
> development?
> >I have some experience with Twisted, so I wouldn't mind taking a look, but
> I
> >would like to get a sense of the project road map so my work isn't wasted.
> >
> >  
> >
> >>   -- Adam
> >>    
> >>
> >
> >Brian
> >
> >
> >PS Are you Adam Rifkin or another Adam?
> >
> >
> >  
> >
> >>On Apr 5, 2005 4:13 AM,
> >>mod...@li...
> >><mod...@li...> wrote:
> >>    
> >>
> >>>Haven't heard anything back from the list regarding this question. If it
> >>>      
> >>>
> >>is
> >>    
> >>
> >>>because the question was too bone-headed, then there is no remedy.
> >>>      
> >>>
> >>Otherwise,
> >>    
> >>
> >>>the following information might be useful to someone else getting
> started.
> >>>
> >>>There are two python client libraries: pubsublib.py and libkn/libkn.py.
> >>>According to the source, pubsublib.py is a non-blocking implementation,
> >>>      
> >>>
> >>while
> >>    
> >>
> >>>libkn.py is blocking. The examples in apps are built off of pubsublib, so
> >>>      
> >>>
> >>they
> >>    
> >>
> >>>should be non-blocking.
> >>>
> >>>This seems to be true when running as a standalone script, but what about
> >>>      
> >>>
> >>when I
> >>    
> >>
> >>>want to embed? Is the logging package being used to help the debugging
> >>>      
> >>>
> >>process
> >>    
> >>
> >>>or are people using pdb?
> >>>
> >>>Also, I read somewhere that the client is supposed to be using Twisted in
> >>>      
> >>>
> >>the
> >>    
> >>
> >>>near future. Is this still the case, and if so, what's the progress on
> it?
> >>>
> >>>Thanks,
> >>>Brian
> >>>
> >>>
> >>>Quoting mod...@li...:
> >>>
> >>>      
> >>>
> >>>>Hello:
> >>>>
> >>>>I'm trying to use the apps/publish.py module as a starting point for
> >>>>        
> >>>>
> >>writing
> >>    
> >>
> >>>>my
> >>>>own simple publishing lib to work with repubsub. The script runs fine on
> >>>>        
> >>>>
> >>its
> >>    
> >>
> >>>>own, but whenever I try to embed it in other code, I seem to get
> >>>>        
> >>>>
> >>blocking
> >>    
> >>
> >>>>issues and the CPU load spikes.
> >>>>
> >>>>Inside the apps/publish.py script, I looked at the following method
> >>>>        
> >>>>
> >>calls:
> >>    
> >>
> >>>>  asyncore.poll()
> >>>>  scheduler.timeout()
> >>>>  scheduler.run()
> >>>>
> >>>>Now Python's asyncore does not have documentation on the poll function,
> >>>>although
> >>>>in my interpreter, I do see a reference for poll. The two scheduler
> >>>>        
> >>>>
> >>function
> >>    
> >>
> >>>>calls both pass in their implementations. If I comment out that loop, my
> >>>>messages don't get published. So what's going on here?
> >>>>
> >>>>Ideally, I'd like to be able to use Twisted's Deferred mechanism to
> >>>>        
> >>>>
> >>handle
> >>    
> >>
> >>>>the
> >>>>blocking issue, but I'm unsure the best way to approach it without
> >>>>        
> >>>>
> >>heavily
> >>    
> >>
> >>>>modifying the code base.
> >>>>
> >>>>Any help is greatly appreciated.
> >>>>
> >>>>Thanks,
> >>>>Brian
> >>>>
> >>>>--
> >>>>  ====
> >>>>brian lee dae yung
> >>>>biomimetic art and research
> >>>>  mux space . com
> >>>>
> >>>>----------------------------------------------------------------
> >>>>This message was sent using IMP, the Internet Messaging Program.
> >>>>
> >>>>
> >>>>
> >>>>-------------------------------------------------------
> >>>>SF email is sponsored by - The IT Product Guide
> >>>>Read honest & candid reviews on hundreds of IT Products from real users.
> >>>>Discover which products truly live up to the hype. Start reading now.
> >>>>http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> >>>>_______________________________________________
> >>>>Mod-pubsub-developer mailing list
> >>>>Mod...@li...
> >>>>https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
> >>>>
> >>>>        
> >>>>
> >>>--
> >>>  ====
> >>>brian lee dae yung
> >>>biomimetic art and research
> >>>  mux space . com
> >>>
> >>>----------------------------------------------------------------
> >>>This message was sent using IMP, the Internet Messaging Program.
> >>>
> >>>-------------------------------------------------------
> >>>SF email is sponsored by - The IT Product Guide
> >>>Read honest & candid reviews on hundreds of IT Products from real users.
> >>>Discover which products truly live up to the hype. Start reading now.
> >>>http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> >>>_______________________________________________
> >>>Mod-pubsub-developer mailing list
> >>>Mod...@li...
> >>>https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
> >>>
> >>>      
> >>>
> >>-------------------------------------------------------
> >>SF email is sponsored by - The IT Product Guide
> >>Read honest & candid reviews on hundreds of IT Products from real users.
> >>Discover which products truly live up to the hype. Start reading now.
> >>http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> >>_______________________________________________
> >>Mod-pubsub-developer mailing list
> >>Mod...@li...
> >>https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
> >>
> >>    
> >>
> >
> >
> >  
> >
> 
> 
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Mod-pubsub-developer mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer
> 


-- 
  ====
brian lee dae yung
biomimetic art and research
  mux space . com

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

Re: [Mod-pubsub-developer] Non-blocking publishing in Python

[<mod...@li...>]

A. wrote:

> > In the interest of strengthening everyone's knowledge base, what did you
> > compare against and what were your concerns?  I've noticed that
> > xmlBlaster and Activ MQ both offer HTTP oriented methodologies for
> > asynchronous notification.  But I am not sure how robust they are, nor
> > how native/central to the design they are, nor how well they work in
> > javascript contexts.

xmlBlaster and Activ MQ's HTTP for asynchronous notification either
require polling or a proxy that can accept connections across the
firewall. The beauty (and scariness) of mod-pubsub's approach is the
use of streaming HTTP to keep connections persistent so no polling and
no proxy needed.

I'm certain neither of those projects work well in JavaScript
contexts. It's not what they were designed for, and it's the reason we
didn't use them in the first place.


Brian wrote:

> My basic criteria was finding a message-based system that was being actively
> developed/supported with Python bindings. I looked at a number of open source
> JMS implementations, most of which had no Python bindings. Activ MQ and Mom4J
> were the two that had support for Python, but Activ MQ seems too
> Windows-centric for serious use on Linux. Mom4J looked more promising, but
> their implementation was incomplete and their user community was like a ghost
> town. There's really no point in investing time in a dead project (unless it
> has killer technology). When I discovered mod_pubsub and the REST-based systems
> (been on my radar for a long time, but never delved into it) I was happy to
> find out that they overcome the confinement limitations of JMS. At this point,
> I'm ignorant of the security ramifications that this presents, which I'd like
> to find out more. FWIW, Mom4J provided hooks for authentication, but no actual
> implementation was present.
> 
> I'll probably be looking at some other REST systems, and I'll let you know the
> outcome.

Thanks, much appreciated.

By the way, I'm not sure anyone is currently using the Python client
library in a project... though I'm sure one day someone will...

   Adam

Re: [Mod-pubsub-developer] Non-blocking publishing in Python

[<mod...@li...>]

Adam wrote:
> 
> By the way, I'm not sure anyone is currently using the Python client
> library in a project... though I'm sure one day someone will...
> 

I use the Python client in a small project to send out modification 
notifications from a Subversion-based xml document repository. Both js and java 
clients are subscribed to these messages and it's been in production for well 
over a year (probably closer to two) without missing a beat.

Robert

Re: [Mod-pubsub-developer] Non-blocking publishing in Python

[<mod...@li...>]

A:

> xmlBlaster and Activ MQ's HTTP for asynchronous notification either
> require polling or a proxy that can accept connections across the
> firewall. The beauty (and scariness) of mod-pubsub's approach is the
> use of streaming HTTP to keep connections persistent so no polling and
> no proxy needed.


At what level of granularity is polling not required? The reason I ask is that
the Python pubsublib client very much uses polling:

    while 1:
        asyncore.poll(scheduler.timeout())
        # print "\n\n asyncore.poll \n"
        scheduler.run()

Going back to the original question I had, this is the core of the problem since
the while loop is very much blocking and is processor intensive. From the
original Medusa documentation on asyncore, I offer the following quote:

"Now that you know what select() does, you're ready for the final piece of the
puzzle: the main polling loop. This is nothing more than a simple while loop
that continually calls select() with a timeout (I usually use a 30-second
timeout). Such a program will use virtually no CPU if your server is idle; it
spends most of its time letting the operating system do the waiting for it.
This is much more efficient than a busy-wait loop."

Hence the solution was to do just this: replace the while loop with the loop()
command which is much more CPU-friendly.

  asyncore.loop()

Anyone interested in embedding the client in a larger python project will likely
need to do this. Ultimately, this is a more efficient implementation and should
probably be merged into the code base. Do you want a patch?

Brian

-- 
  ====
brian lee dae yung
biomimetic art and research
  mux space . com

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

Re: [Mod-pubsub-developer] Non-blocking publishing in Python

[<mod...@li...>]

Adam:

Is your information current?  I was under the impression that a 
streaming design was now in place for Active MQ and/or xmlBlaster's HTTP 
capabilities.  But that's probably wrong, I am vaguely remembering a 
feeling I got while skimming some docs a while back.

:-)

Either/both of these projects support dynamic topic creation and some of 
the core requirements to achieve REST client access characteristics.

I think mod-pubsub should consider layering itself onto these two quite 
popular, and quite active, MOM projecs.  Think "REST pubsub module" for 
MOM instead of for Apache.  The client-access characteristics that would 
emerge could act as a motivator for the MOM community to adopt REST 
architectural principles.  The current mod-pubsub server could/should 
interact well with these modules, allowing it to become a "subsidiary" 
or "proxy" event notification server, giving developers the choice of 
using the lightweight mod-pubsub solution in contexts where the 
full-blown MOM solution is inappropriate, yet connecting the two, or 
switching to the MOM solution, without ever changing APIs.

Alright, I'm just blowing hot air, but it was fun.

A.

mod...@li... wrote:

>A. wrote:
>
>  
>
>>>In the interest of strengthening everyone's knowledge base, what did you
>>>compare against and what were your concerns?  I've noticed that
>>>xmlBlaster and Activ MQ both offer HTTP oriented methodologies for
>>>asynchronous notification.  But I am not sure how robust they are, nor
>>>how native/central to the design they are, nor how well they work in
>>>javascript contexts.
>>>      
>>>
>
>  
>