pydispatcher-devel — Python Dispatcher Developers

[Pydispatcher-devel] bug-like phenomenon

[Antony K. <ant...@ya...>]

Hi!

In an application I am working on, I'm getting an
exception in dispatcher._removeReceiver, which occurs,
apparently, after the main script exits. When it is
called, dispatcher.sendersBack is None! I suspect that
this is happening as part of Python's shutdown
procedures which also affect the module's interiors.
Anyway, I made a patch that returns immediately from
_removeReceiver in case sendersBack is None, but I
don't know what is causing it and whether this
phenomenon could manifest itself in different ways.
I'd be happy to help further if I can.

Antony Kummel

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Pydispatcher-devel] Wording of simple_sample

[Stephen W. <go...@co...>]

Mike C. Fletcher wrote:
> Stephen Waterbury wrote:
> ...
>>So anyway, what I find perplexing is that the prints in
>>simple_sample.py all say "Sending [signal] *to* first object"
>>(emphasis mine) ... but according to everything else I see
>>there, the first object (which I assume refers to
>>ourObjects[0]) is the *sender*, not the receiver (the
>>receivers being the callables), so I would have expected
>>the prints to say "Sending [signal] *from* first object" ...
>>am I wrong?
> ...
> It really should have read "from", I just mis-typed :) .  I've fixed it
> locally, will see about getting it uploaded in fixed form when I get a
> chance.

Great -- I'm glad my sanity was not threatened (at least in
that one area! ;).

BTW, I just started using PyDispatcher this week, and I must
say IT ROCKS!!  :)  My first use of it is to dispatch
a "state changed" signal to components of my GUI app
(the receiver is a function 'onStateChange' that checks the 'state'
dictionary -- a module-level instance that all the components
import), so now all my gui windows that have a little
connector icon on the bottom right to show whether the
app is "online" or not are all synchronized.  (I was
actually low enough to steal mozilla's little connector
icons, too ;).  I'm easily amused!  :)  But it was very
cool how *easy* it was to implement with PyDispatcher ...
I really must write a testimonial or something!

Cheers,
Steve

Re: [Pydispatcher-devel] Wording of simple_sample

[Mike C. F. <mcf...@ro...>]

Stephen Waterbury wrote:
...

> So anyway, what I find perplexing is that the prints in
> simple_sample.py all say "Sending [signal] *to* first object"
> (emphasis mine) ... but according to everything else I see
> there, the first object (which I assume refers to
> ourObjects[0]) is the *sender*, not the receiver (the
> receivers being the callables), so I would have expected
> the prints to say "Sending [signal] *from* first object" ...
> am I wrong?

...

It really should have read "from", I just mis-typed :) .  I've fixed it
locally, will see about getting it uploaded in fixed form when I get a
chance.

Thanks,
Mike

-- 
________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://www.vrplumber.com
  http://blog.vrplumber.com

[Pydispatcher-devel] Wording of simple_sample

[Stephen W. <go...@co...>]

Hi gang,

I'm just now getting around to adding dispatcher to my app,
and I've been staring at the simple_sample.py for a while.
It runs fine, but I'm just confused by the wording (I found
this confusing a while ago, and figured maybe if I came back
to it later it would make sense to me, but it still doesn't).

So anyway, what I find perplexing is that the prints in
simple_sample.py all say "Sending [signal] *to* first object"
(emphasis mine) ... but according to everything else I see
there, the first object (which I assume refers to
ourObjects[0]) is the *sender*, not the receiver (the
receivers being the callables), so I would have expected
the prints to say "Sending [signal] *from* first object" ...
am I wrong?

Thanks for any enlightenment!

- Steve

Re: [Pydispatcher-devel] MANIFEST.in

[Patrick K. O'B. <po...@or...>]

Artem Baguinski wrote:
> hi
> 
> i'm new to python and all, so i don't know how come, but i needed to
> make some changed to the MANIFEST.in file to be able to build rpm
> package of PyDispatcher. may be the format of the template have changed
> since you've written it?
> 
> hope the attached patch can be usefull

Thank you very much for the patch, Artem.  Hopefully Mike will get a
chance to look at it to see what the issue is.  And, welcome to Python!

-- 
Patrick K. O'Brien
Orbtech    http://www.orbtech.com
Schevo     http://www.schevo.org
Pypersyst  http://www.pypersyst.org

[Pydispatcher-devel] MANIFEST.in

[Artem B. <ar...@v2...>]

hi

i'm new to python and all, so i don't know how come, but i needed to 
make some changed to the MANIFEST.in file to be able to build rpm 
package of PyDispatcher. may be the format of the template have changed 
since you've written it?

hope the attached patch can be usefull

-- 
Artem Baguinski:                         http://www.artm.org/
V2_Lab:                                  http://lab.v2.nl/
V2_ Organisation for the Unstable Media: http://www.v2.nl/

Re: [Pydispatcher-devel] 'dispatch' naming clash

[Matthew S. <gld...@us...>]

Stephen Waterbury wrote:
> IMHO, unless a package is
> maintained as a separate distribution, it should install
> itself within the namespace of its parent package.

This is why it totally, absolutely, positively, posilutely, absitively,
sucks that relative imports weren't implemented by Python 2.4
(...wishing I had more time, and more mad C skills...)

We were going to do precisely what you suggested to include dependencies
within Schevo, but then realized that said dependencies would be doing
quite a bit of absolute importing of themselves.

That's when we decided to hack around it, because we didn't want to
patch a bunch of import statements in the several large moving targets
that are Schevo's dependencies ;)

-- 
Matthew R. Scott

Re: [Pydispatcher-devel] 'dispatch' naming clash

[Stephen W. <go...@co...>]

Stephen Waterbury wrote:
> almost disappointed ... something in me is always secretly
> gleeful when deep voodoo goes awry.  ;)

Of course, I meant "when deep voodoo that my stuff doesn't
depend on goes awry"!  Or, as Mel Brooks once pointed out,
"Tragedy is when I get a hangnail; comedy is when someone
else falls down a manhole."  ;)

Steve

Re: [Pydispatcher-devel] 'dispatch' naming clash

[Stephen W. <go...@co...>]

Mike C. Fletcher wrote:
> That said, as long as PEAK's module doesn't overwrite any of the modules 
> in the PyDispatcher package (and as long as importing it's __init__.py 
> doesn't completely kark up the object model in some inventive and evil 
> way (this being PEAK, after all)) ...

Heh ... very well have you articulated the source of my unease.
But from Matthew's and Pat's replies, I gather that it didn't
kark up this particular subspace of the object model ... I'm
almost disappointed ... something in me is always secretly
gleeful when deep voodoo goes awry.  ;)

Matthew Scott wrote:
> Stephen Waterbury wrote:
>> ... Phillip Eby's PEAK package has a
>> sub-package called 'dispatch', which unfortunately (and,
>> IMO, rather arrogantly ;) PEAK installs into the
>> top-level site-packages namespace.  ...
> 
> I think it does this because PEAK collects a lot of other packages.

Welll ... <grumble grumble> ... yes PEAK does that, but I don't
see the PEAK dispatch package being released separately, as (say)
PyProtocols (justifiably) was.  IMHO, unless a package is
maintained as a separate distribution, it should install
itself within the namespace of its parent package.  (I also use
ZODB, and it's needless pollution always bugs me, but they have
managed not to step on anybody's else's names yet, AFAICT.)
 
> PJE's predicate dispatch package seems to be a "sister" package to the 
> 'protocols' package in PyProtocols -- so this problem would exist for 
> those using just PyProtocols and not just those using the entirety of PEAK.

True ... all the more reason ...

> Pat O'Brien and I ran into this namespace clash in developing Schevo ....
> We ended up solving the problem by a) including most Python dependencies 
> within Schevo in a manner that doesn't clash with site-packages, and b) 
> renaming the PyDispatch package from 'dispatch' to 'pydispatch' within 
> those included dependencies.

I don't mind this solution, but it shouldn't be necessary if PEAK were
playing nice.

Patrick K. O'Brien wrote:
> ....  The problem with PEAK began
> before Philip was aware of this project ....  The
> PEAK dispatcher is a different beast altogether, iirc.  (And let's face
> it, dispatch is a pretty generic term.  I wasn't the first and only
> person to create something with that name.)

True enough, but AFAIK, Mike and you were the first to release
a Python package with a module by that name, so I think your
claim to it in the Python namespace is legitimate.

> For now, let's give Mr. Eby the benefit of the doubt and assume that we
> can work out an amicable solution.  If that fails, Mike can feed him to
> the lions at PyCon while I sit ringside cheering on the lions.  ;-)

Rawr!  We should advertise the event -- heck, it might double
the PyCon registrations!   (Headline:  "Python Community
Metes Out Severe Penalties For Namespace Pollution!" ...
gimme that ol' time religion.  :)

> In all seriousness, perhaps a nice, polite message to PJE would be a
> good way to start.  I'd do it myself except I'm dog-tired right now.

Agreed.  My next step was going to be politely pointing out the
situation in a message to the PEAK list and seeing what response
came forth.  If you beat me to it, I'll chime in to register my vote
in favor of a more ecological approach to the Python contributed
package namespace, anyway.

Thanks for all the input, everyone.

Cheers,
Steve

Re: [Pydispatcher-devel] 'dispatch' naming clash

[Patrick K. O'B. <po...@or...>]

Drew Perttula wrote:
>>You may be aware that Phillip Eby's PEAK package has a
>>sub-package called 'dispatch', which unfortunately (and,
>>IMO, rather arrogantly ;) PEAK installs into the
>>top-level site-packages namespace.  So, since I already
> 
> Possibly this is related to [1], where PJE goes into some detail about the
> "important" differences between PyDispatcher and his (second?) rewrite.
> 
> I don't know exactly what his needs are, but if even if they do need a
> variation on PyDispatcher, I would certainly have asked this group first
> about offering that feature on the real thing, myself. 
> 
> [1] http://cvs.osafoundation.org/viewcvs.cgi/internal/Spike/src/spike/events.txt?rev=1.6&content-type=text/vnd.viewcvs-markup

No, I don't think the two are related.  The problem with PEAK began
before Philip was aware of this project, and is unrelated to his OSAF
Spike.  It's only his Spike dispatcher that is similar to this one.  The
PEAK dispatcher is a different beast altogether, iirc.  (And let's face
it, dispatch is a pretty generic term.  I wasn't the first and only
person to create something with that name.)

We are actually using pydispatcher and parts of PEAK in Schevo.  To
solve the conflict we had to do some local renaming tricks in our
Subversion repository.  When Matthew Scott has some free time he'll
provide more details.

For now, let's give Mr. Eby the benefit of the doubt and assume that we
can work out an amicable solution.  If that fails, Mike can feed him to
the lions at PyCon while I sit ringside cheering on the lions.  ;-)

In all seriousness, perhaps a nice, polite message to PJE would be a
good way to start.  I'd do it myself except I'm dog-tired right now.

-- 
Patrick K. O'Brien
Orbtech    http://www.orbtech.com
Schevo     http://www.schevo.org
Pypersyst  http://www.pypersyst.org

Re: [Pydispatcher-devel] 'dispatch' naming clash

[Matthew S. <gld...@us...>]

Stephen Waterbury wrote:
> Mike,
> 
> You may be aware that Phillip Eby's PEAK package has a
> sub-package called 'dispatch', which unfortunately (and,
> IMO, rather arrogantly ;) PEAK installs into the
> top-level site-packages namespace.  So, since I already
> had PEAK installed when I installed PyDispatcher, I
> did an 'import dispatch' and 'help(dispatch)' and was
> completely astonished at the result until I realized
> it was coming from PEAK's dispatch package.


I think it does this because PEAK collects a lot of other packages. 
PJE's predicate dispatch package seems to be a "sister" package to the 
'protocols' package in PyProtocols -- so this problem would exist for 
those using just PyProtocols and not just those using the entirety of PEAK.


> Anyway, I realize that may or may not have practical
> consequences, but it does make me uncomfortable.  I need
> both PEAK and PyDispatch in my application, so I was
> wondering if anyone had tried that and, if so, whether
> they had experienced anything untoward.

Pat O'Brien and I ran into this namespace clash in developing Schevo, 
because we too use both PyDispatcher and PyProtocols' predicate-dispatch 
generic functions.

We ended up solving the problem by a) including most Python dependencies 
within Schevo in a manner that doesn't clash with site-packages, and b) 
renaming the PyDispatch package from 'dispatch' to 'pydispatch' within 
those included dependencies.

This turns out to work pretty well for us even though there is some 
sleight of hand involved, since we depend on the CVS/SVN HEAD versions 
of both PEAK and the 'py' library -- it ensures that the correct 
versions are used, users working out of our Subversion repository don't 
have to get their hands dirty with a lot of dependencies, and anything 
installed in site-packages is left untouched.


-- 
Matthew R. Scott

Re: [Pydispatcher-devel] 'dispatch' naming clash

[Drew P. <dr...@do...>]

> You may be aware that Phillip Eby's PEAK package has a
> sub-package called 'dispatch', which unfortunately (and,
> IMO, rather arrogantly ;) PEAK installs into the
> top-level site-packages namespace.  So, since I already

Possibly this is related to [1], where PJE goes into some detail about the
"important" differences between PyDispatcher and his (second?) rewrite.

I don't know exactly what his needs are, but if even if they do need a
variation on PyDispatcher, I would certainly have asked this group first
about offering that feature on the real thing, myself. 


[1] http://cvs.osafoundation.org/viewcvs.cgi/internal/Spike/src/spike/events.txt?rev=1.6&content-type=text/vnd.viewcvs-markup

Re: [Pydispatcher-devel] 'dispatch' naming clash

[Mike C. F. <mcf...@ro...>]

Stephen Waterbury wrote:

> Mike,
>
> You may be aware that Phillip Eby's PEAK package has a
> sub-package called 'dispatch', which unfortunately (and,
> IMO, rather arrogantly ;) PEAK installs into the
> top-level site-packages namespace.  So, since I already
> had PEAK installed when I installed PyDispatcher, I
> did an 'import dispatch' and 'help(dispatch)' and was
> completely astonished at the result until I realized
> it was coming from PEAK's dispatch package.

Have never actually used or installed PEAK (though I did plumb through 
it's code looking for metaclass examples for a talk once), so no, wasn't 
aware of it.

> Anyway, I realize that may or may not have practical
> consequences, but it does make me uncomfortable.  I need
> both PEAK and PyDispatch in my application, so I was
> wondering if anyone had tried that and, if so, whether
> they had experienced anything untoward.

Given that no-one has complained to Phillip yet (to my knowledge, backed 
up by the fact that it's still there), probably not ;) .

That said, as long as PEAK's module doesn't overwrite any of the modules 
in the PyDispatcher package (and as long as importing it's __init__.py 
doesn't completely kark up the object model in some inventive and evil 
way (this being PEAK, after all)), you should be able to continue using 
PyDispatcher... in other words, our __init__ file doesn't do anything 
special.

> If no one here has tried it, I'll contact Phillip Eby
> and see if it would break anything in PEAK if I just
> delete his dispatch package.  (Actually, all I'm using
> from PEAK is his XMI stuff, so as long as it doesn't
> break *that* I'm okay -- I couldn't care less about the
> rest of PEAK! ;).

Please just go ahead and register your (infinite) disapproval, even if 
someone pipes up to say they've tried it ;) :) .  If it's a sub-package 
it really should be a sub-package, not a top-level package, if it's a 
competing global dispatch system, well, then, they'll just have to find 
a different top-level package name (or Phillip and I will have to rumble 
at PyCon (hope he's not a jujitsu master or something)).

Have fun,
Mike

________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://www.vrplumber.com
  http://blog.vrplumber.com
                              PyCon is coming...

[Pydispatcher-devel] 'dispatch' naming clash

[Stephen W. <go...@co...>]

Mike,

You may be aware that Phillip Eby's PEAK package has a
sub-package called 'dispatch', which unfortunately (and,
IMO, rather arrogantly ;) PEAK installs into the
top-level site-packages namespace.  So, since I already
had PEAK installed when I installed PyDispatcher, I
did an 'import dispatch' and 'help(dispatch)' and was
completely astonished at the result until I realized
it was coming from PEAK's dispatch package.

Anyway, I realize that may or may not have practical
consequences, but it does make me uncomfortable.  I need
both PEAK and PyDispatch in my application, so I was
wondering if anyone had tried that and, if so, whether
they had experienced anything untoward.

If no one here has tried it, I'll contact Phillip Eby
and see if it would break anything in PEAK if I just
delete his dispatch package.  (Actually, all I'm using
from PEAK is his XMI stuff, so as long as it doesn't
break *that* I'm okay -- I couldn't care less about the
rest of PEAK! ;).

Cheers,
Steve

[Pydispatcher-devel] __version__ is '1.0.0' even in release 1.0.2

[Drew P. <dr...@bi...>]

In trying to troubleshoot the source of these errors at program exit:

Traceback (most recent call last):
  File "/usr/local/lib/python2.3/site-packages/dispatch/saferef.py", line 111, in remove
    function( self )
  File "/usr/local/lib/python2.3/site-packages/dispatch/dispatcher.py", line 362, in _removeReceiver
    for senderkey in sendersBack.get(backKey,()):
AttributeError: 'NoneType' object has no attribute 'get'
Traceback (most recent call last):
  File "/usr/local/lib/python2.3/site-packages/dispatch/saferef.py", line 111, in remove
    function( self )
  File "/usr/local/lib/python2.3/site-packages/dispatch/dispatcher.py", line 362, in _removeReceiver
    for senderkey in sendersBack.get(backKey,()):
AttributeError: 'NoneType' object has no attribute 'get'


I was trying to compare pydispatcher versions on different boxes. The
__init__.py __version__ has been left at '1.0.0' for a while now, which
led to some confusion.

[Pydispatcher-devel] Another leak fixed, hopefully the last one

[Mike C. F. <mcf...@ro...>]

Found another leak in the back-reference handling.  Added code to check 
for any leaks to the test suite, so hopefully there won't be the need 
for any more minor point releases.

Have fun all,
Mike

Mike C. Fletcher wrote:

> It's a bug in how receivers are cleaned up and another in how they are 
> registered for cleanup.

...

________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://www.vrplumber.com
  http://blog.vrplumber.com

[Pydispatcher-devel] Found (and fixed) memory leak in dispatcher

[Mike C. F. <mcf...@ro...>]

It's a bug in how receivers are cleaned up and another in how they are 
registered for cleanup.

    * The _removeReceiver function didn't remove the sendersBack
      reference (i.e. it would leak the list+all of the integers in it)
    * There were also a problem in connect, which look like I may have
      mistakenly hit "indent", which made the logic such that every
      receiver beyond the first which was registered would not have
      their back-references registered, so would not get cleaned up
    * Did some cleanup work here and there as well

Likely release this as a bug-fix release some time this evening.

Have fun all,
Mike

________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://www.vrplumber.com
  http://blog.vrplumber.com

[Pydispatcher-devel] Properly cleaning up receiver weakrefs for Any sender

[David B. <db...@fi...>]

In using PyDispatcher, I was surprised to find that although the docs
mention the use of weak references to automatically clean up when senders or
receivers go away, it didn't seem to work in the first case I was using,
where my receivers were all connected with the default Any sender.

For example:

Python 2.3.4 (#53, May 25 2004, 21:17:02) [MSC v.1200 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from dispatch import dispatcher
>>> class Foo:
...   def __init__(self):
...     dispatcher.connect(self.callback, 'TEST')
...   def callback(self):
...     print 'callback'
... 
>>> x = Foo()
>>> dispatcher.connections
{8252872: {'TEST': [BoundMethodWeakref( <__main__.Foo instance at
0x007D7030>.callback )]}}
>>> dispatcher.sendersBack
{}
>>> del x
>>> dispatcher.connections
{8252872: {'TEST': [BoundMethodWeakref( None.callback )]}}
>>> id(dispatcher.Any)
8252872

I would have expected the link to x.callback to have been removed along when
x was destroyed.

I traced it down to the fact that there is a specific exclusion in connect()
for senders of (None, Any, Anonymous) in installing them in the sendersBack
dictionary.  However, _removeReceiver requires that the sender be in
sendersBack before it will prune the receiver from the connections list.  So
it's impossible to prune any receivers that may be accepting signals from
None (not sure what this does), Any or Anonymous.

Adding the excluded senders into the list of senders that _removeReceiver
works through seems to handle the case, but since I wasn't positive why they
were being excluded in the first place (the comment in connect() seems to
indicate an uncertainty about Anonymous, but my case is Any), I'm interested
if anyone thinks this would cause a problem.

Thanks.

-- David

PS: Here's the change I tried .. in real use I'd probably build the list of
Any/Anonymous
ids at the point they are instantiated, since that will never change.

*** dispatcher.py.orig	Sun Jun 20 21:01:34 2004
--- dispatcher.py	Thu Aug 12 19:05:15 2004
***************
*** 359,365 ****
  def _removeReceiver(receiver):
  	"""Remove receiver from connections."""
  	backKey = id(receiver)
! 	for senderkey in sendersBack.get(backKey,()):
  		try:
  			for signal in connections[senderkey].keys():
  				try:
--- 359,365 ----
  def _removeReceiver(receiver):
  	"""Remove receiver from connections."""
  	backKey = id(receiver)
! 	for senderkey in
(id(Any),id(Anonymous))+sendersBack.get(backKey,()):
  		try:
  			for signal in connections[senderkey].keys():
  				try:

Re: [Pydispatcher-devel] PyDispatcher 1.0.0 (final) released

[Matthew S. <gld...@us...>]

Mike C. Fletcher wrote:
> Release 1.0.0 has no major changes from the 1.0.0 alpha release.
> 
> Enjoy,
> Mike


Excellent!  Thank you for your work on this Mike.  Now the list of 
dependencies in the project I'm working on will include one less "CVS 
HEAD from project XYZ" ;-)


A suggestion for the next release:

Right now, in some common setUp() code I am using in some unit tests, I 
make sure dispatcher is in a "clean" state by doing this:

         dispatcher.connections = {}
         dispatcher.senders = {}
         dispatcher.sendersBack = {}

I'm assuming that is all that is necessary to reset it to its initial 
state.  It would be handy though, to have a dispatcher.reset() function 
that would do the same -- and could be updated in the future if it ever 
needed to do anything else to reset its state.


Your thoughts/comments on that would be appreciated.


Thanks,

- Matthew

[Pydispatcher-devel] PyDispatcher 1.0.0 (final) released

[Mike C. F. <mcf...@ro...>]

What is it?

    PyDispatcher provides the Python programmer with a
    multiple-producer-multiple-consumer signal-registration and routing
    infrastructure for use in multiple contexts.  The mechanism of
    PyDispatcher started life as a highly rated recipe
    <http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/87056> in
    the Python Cookbook
    <http://aspn.activestate.com/ASPN/Python/Cookbook/>.  The
    SourceForge project <https://sourceforge.net/projects/pydispatcher/>
    aims to include various enhancements to the recipe developed during
    use in various applications.

Where to get more information and/or download pydispatcher:
    http://pydispatcher.sourceforge.net/

Release 1.0.0 has no major changes from the 1.0.0 alpha release.

Enjoy,
Mike

________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/
  blog: http://zope.vex.net/~mcfletch/plumbing/

Re: [Pydispatcher-devel] Time to do a 1.0.0 final release?

[<po...@or...>]

"Mike C. Fletcher" <mcf...@ro...> writes:

> I decided to use PyDispatcher as a test/learning package for building
> a Gentoo ebuild for a Python package, which resulted in a few fixes to
> the MANIFEST.in to make sure that all of the source files are included
> in the source packages.  However, doing it made me realise (again)
> that we've been stuck in pre-release limbo for far too long on
> pydispatcher.
> 
> Anyway, if Patrick agrees I'll build .tar.gz, .zip and win32 exe
> packages sometime in the next two days and do a 1.0.0 final release on
> SourceForge.
> 
> You can see the ebuild either in CVS or posted to my blog here:
>     http://zope.vex.net/~mcfletch/plumbing/213

+1

-- 
Patrick K. O'Brien
Orbtech  http://www.orbtech.com
Blog     http://www.sum-ergo-cogito.com

[Pydispatcher-devel] Time to do a 1.0.0 final release?

[Mike C. F. <mcf...@ro...>]

I decided to use PyDispatcher as a test/learning package for building a 
Gentoo ebuild for a Python package, which resulted in a few fixes to the 
MANIFEST.in to make sure that all of the source files are included in 
the source packages.  However, doing it made me realise (again) that 
we've been stuck in pre-release limbo for far too long on pydispatcher.

Anyway, if Patrick agrees I'll build .tar.gz, .zip and win32 exe 
packages sometime in the next two days and do a 1.0.0 final release on 
SourceForge.

You can see the ebuild either in CVS or posted to my blog here:
    http://zope.vex.net/~mcfletch/plumbing/213

Have fun all,
Mike

________________________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/
  blog: http://zope.vex.net/~mcfletch/plumbing/

Re: [Pydispatcher-devel] PyDispatcher and thread safety

[Mike C. F. <mcf...@ro...>]

Patrick K. O'Brien wrote:

>"Matias Guijarro" <gui...@es...> writes:
>  
>
...

>>Reading source code, I did not notice any
>>special code for dealing with this kind of
>>usage. I expected to see some Queues,
>>or Locks, etc.
>>
>>However I tried to use it with threads, and
>>for my particular example it worked... As
>>it is said to be a 'multi-producer, multi-
>>consumer signals dispatcher', I think it is
>>thread-safe ?
>>    
>>
Thread-safe is a matter of degree, a judgement based on experience that 
there are no places in the code where a thread deadlock/contention or 
unwanted interaction could occur.  I've just done a quick audit of the 
core dispatch.py module, and from what I can see, it's *almost* 
thread-safe (which is the most dangerous type of thread-safe ;) ).

>>If so, how it is achieved ?
>>I would like to understand :-)
>>    
>>
Defensive coding, in this particular case, rather than queues and locks, 
the code uses try: except: blocks throughout to catch the cases where 
another thread has changed the situation.  I think Patrick probably put 
them in to deal with synchronous changes (changes in the same thread), 
but they have the general effect of guarding against multi-thread 
problems as well.  The results may not be what you'd expect (receivers 
you expected to be called (because you looked at them before calling 
send) aren't because they've been deleted by some other thread, for 
instance), but generally the code would get through the situation, and 
if you think about it, those are actually predictable results when you 
consider the nature of threading.

There are a number of other things that one has to look out for, 
however; such as structures being replaced by processed versions of 
themselves (we seem pretty safe there), value checks where the value 
used to make the decision changes while working on the assumption, and 
objects mutating while you have a reference to them (e.g. by removing a 
value) which is a more general case of the previous problem.

Recognise that last one :) ?  PyDispatcher actually uses 
receivers.remove(receiver) to remove items from internal lists.  If this 
is done while another thread is iterating over the list (there's no copy 
taken, in fact we use generators explicitly to avoid making such a copy 
for performance reasons), the result will be the iterating thread will 
miss a receiver.  That's a fairly rare situation, so you're not going to 
see it in a few test runs, but it's definitely going to happen some day.

There's a similar potential problem with _cleanupConnections, where the 
"if not receivers:" check could happen before another thread inserts a 
receiver, but the deletion occurs afterwards, resulting in a lost 
receiver (table) again.

>>Thanks in advance for your answer!
>>    
>>
>
>The original was not designed to be thread-safe.  I'm not sure if Mike
>changed anything in the current version to make it so, but I doubt
>it.  I don't know too much about threading with Python, so I'm not
>sure how hard it would be to make the dispatching thread-safe.  If it
>could be done rather simply and elegantly I'm sure we'd love to have a
>patch that adds this feature.
>  
>
The changes to increase the reliability of the code in the face of 
deletions shouldn't be too significant, but honestly I can't see myself 
getting a chance to work on it in the next few weeks (just too many 
other projects in the queue that need what little free time I can 
spare).  I'd be concerned that we not kill performance, incidentally, as 
using locks can be very expensive for single-threading apps.  One of the 
most useful things that someone could do would be to write test cases 
that *break* the existing model in the face of multiple threads (without 
worrying about fixing them).

Have fun all,
Mike

_______________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/

Re: [Pydispatcher-devel] PyDispatcher and thread safety

[<po...@or...>]

"Matias Guijarro" <gui...@es...> writes:

> Hello,
> 
> I would like to know if it is thread-safe to
> use PyDispatcher when signals are emitted
> from a thread, with receivers from other
> threads and vice versa.
> 
> Reading source code, I did not notice any
> special code for dealing with this kind of
> usage. I expected to see some Queues,
> or Locks, etc.
> 
> However I tried to use it with threads, and
> for my particular example it worked... As
> it is said to be a 'multi-producer, multi-
> consumer signals dispatcher', I think it is
> thread-safe ?
> 
> If so, how it is achieved ?
> I would like to understand :-)
> 
> Thanks in advance for your answer!

The original was not designed to be thread-safe.  I'm not sure if Mike
changed anything in the current version to make it so, but I doubt
it.  I don't know too much about threading with Python, so I'm not
sure how hard it would be to make the dispatching thread-safe.  If it
could be done rather simply and elegantly I'm sure we'd love to have a
patch that adds this feature.

-- 
Patrick K. O'Brien
Orbtech  http://www.orbtech.com
Blog     http://www.sum-ergo-cogito.com

[Pydispatcher-devel] PyDispatcher and thread safety

[Matias G. <gui...@es...>]

Hello,

I would like to know if it is thread-safe to
use PyDispatcher when signals are emitted
from a thread, with receivers from other
threads and vice versa.

Reading source code, I did not notice any
special code for dealing with this kind of
usage. I expected to see some Queues,
or Locks, etc.

However I tried to use it with threads, and
for my particular example it worked... As
it is said to be a 'multi-producer, multi-
consumer signals dispatcher', I think it is
thread-safe ?

If so, how it is achieved ?
I would like to understand :-)

Thanks in advance for your answer!

Regards,
M. Guijarro.