pypes-developer Mailing List for pypes
Status: Beta
Brought to you by:
egaumer
You can subscribe to this list here.
2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(13) |
Oct
|
Nov
|
Dec
|
---|---|---|---|---|---|---|---|---|---|---|---|---|
2010 |
Jan
|
Feb
(5) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: SourceForge.net <no...@so...> - 2010-02-09 03:45:03
|
Bugs item #2948226, was opened at 2010-02-08 22:45 Message generated for change (Tracker Item Submitted) made by egaumer You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1155513&aid=2948226&group_id=271766 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Interface Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Eric Gaumer (egaumer) Assigned to: Eric Gaumer (egaumer) Summary: Clicking OK button on config dialog not working in Firefox Initial Comment: If a component has no configurable parameters, clicking the OK button does not dismiss the dialog in Firefox. You have to click the cancel button instead. The OK button appears to work fine in Safari & Chrome (i.e., WebKit). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1155513&aid=2948226&group_id=271766 |
From: SourceForge.net <no...@so...> - 2010-02-07 23:53:53
|
Bugs item #2947547, was opened at 2010-02-07 16:33 Message generated for change (Comment added) made by egaumer You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1155513&aid=2947547&group_id=271766 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Interface Group: None >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Matt Weber (mattrweber) Assigned to: Eric Gaumer (egaumer) Summary: 2nd InputPort not connected correctly to 2nd OutputPort Initial Comment: When you have a pipeline with a component that has 2 output ports followed by a component with 2 input ports, the 2nd output port does not get connected to the 2nd output port correctly. To reproduce: 1. Start pypes with empty pipeline 2. Create the following pipeline TextReader -> Split -> Collate -> Debug. 3. Send any text file though the pipeline Take a look at the debug output and you will see that only one document has been output. If you take a look at the component inputs dictionary (Component._inputs) you will see the 2nd input contains a None object: {'in2': [None, 'Second Input Port'], 'in': [<pypes.pype.Pype object at 0x9f6c76c>, 'Default input port']} Now, if you break the connection between the 2 output ports, everything works just fine. For example put a copy field between the 2nd output port and the 2nd input port and you will now see the expected behavior. This leads me to believe that the UI code that connects the ports has a bug connecting components with multiple ports. ---------------------------------------------------------------------- >Comment By: Eric Gaumer (egaumer) Date: 2010-02-07 18:53 Message: Pypes does not support multgraphs. Not a bug. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1155513&aid=2947547&group_id=271766 |
From: SourceForge.net <no...@so...> - 2010-02-07 21:33:55
|
Bugs item #2947547, was opened at 2010-02-07 13:33 Message generated for change (Tracker Item Submitted) made by mattrweber You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1155513&aid=2947547&group_id=271766 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Interface Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Matt Weber (mattrweber) Assigned to: Eric Gaumer (egaumer) Summary: 2nd InputPort not connected correctly to 2nd OutputPort Initial Comment: When you have a pipeline with a component that has 2 output ports followed by a component with 2 input ports, the 2nd output port does not get connected to the 2nd output port correctly. To reproduce: 1. Start pypes with empty pipeline 2. Create the following pipeline TextReader -> Split -> Collate -> Debug. 3. Send any text file though the pipeline Take a look at the debug output and you will see that only one document has been output. If you take a look at the component inputs dictionary (Component._inputs) you will see the 2nd input contains a None object: {'in2': [None, 'Second Input Port'], 'in': [<pypes.pype.Pype object at 0x9f6c76c>, 'Default input port']} Now, if you break the connection between the 2 output ports, everything works just fine. For example put a copy field between the 2nd output port and the 2nd input port and you will now see the expected behavior. This leads me to believe that the UI code that connects the ports has a bug connecting components with multiple ports. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1155513&aid=2947547&group_id=271766 |
From: SourceForge.net <no...@so...> - 2010-02-07 14:01:03
|
Bugs item #2947260, was opened at 2010-02-07 01:12 Message generated for change (Settings changed) made by egaumer You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1155513&aid=2947260&group_id=271766 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Interface Group: None >Status: Closed >Resolution: Fixed Priority: 9 Private: No Submitted By: Matt Weber (mattrweber) Assigned to: Eric Gaumer (egaumer) Summary: unable to drag components to the canvas Initial Comment: This bug was introduced at rev44. To reproduce: 1. start pypes 2. try to drag any component onto the canvas I have backed out rev44 changes starting at rev48 and attached them as a patch here. Once we fix this issue we can re-commit. ---------------------------------------------------------------------- >Comment By: Eric Gaumer (egaumer) Date: 2010-02-07 09:01 Message: Fix attached. Apply both of the attached patches. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1155513&aid=2947260&group_id=271766 |
From: SourceForge.net <no...@so...> - 2010-02-07 06:12:23
|
Bugs item #2947260, was opened at 2010-02-06 22:12 Message generated for change (Tracker Item Submitted) made by mattrweber You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1155513&aid=2947260&group_id=271766 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Interface Group: None Status: Open Resolution: None Priority: 9 Private: No Submitted By: Matt Weber (mattrweber) Assigned to: Eric Gaumer (egaumer) Summary: unable to drag components to the canvas Initial Comment: This bug was introduced at rev44. To reproduce: 1. start pypes 2. try to drag any component onto the canvas I have backed out rev44 changes starting at rev48 and attached them as a patch here. Once we fix this issue we can re-commit. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1155513&aid=2947260&group_id=271766 |
From: Matt W. <ma...@ma...> - 2009-09-22 02:37:55
|
On Sep 21, 2009, at 8:17 AM, Matt Weber wrote: > On Sep 21, 2009, at 6:52 AM, Eric Gaumer wrote: > >> Looking at releasing the 3rd and final beta. >> >> Matt, go ahead and complete 2860469 and commit your patch for >> 2860459. I have a fix for 2862158. Once these are complete we'll >> release 01.0b3. That will put us in a feature freeze and we'll >> concentrate on an RC release where we can focus on strictly bug >> fixes. >> > > Committed 2860459. Will work on 2860469 later today. > > Thanks, > Matt Weber > Just committed the following: - Crawler option to recursively crawl directories (2860465) - Crawler option to enable compression (2860469) - Crawler option to limit to specific file extensions (2860468) - Crawler option to have verbose output - Fix for error in packet __eq__ when comparing non-Packet objects - Refactored docs controller to use new packet api Once you commit your fix for 2862158 and we should be ready to focus on an RC release. Thanks, Matt Weber |
From: Matt W. <ma...@ma...> - 2009-09-21 15:18:18
|
On Sep 21, 2009, at 6:52 AM, Eric Gaumer wrote: > Looking at releasing the 3rd and final beta. > > Matt, go ahead and complete 2860469 and commit your patch for > 2860459. I have a fix for 2862158. Once these are complete we'll > release 01.0b3. That will put us in a feature freeze and we'll > concentrate on an RC release where we can focus on strictly bug fixes. > Committed 2860459. Will work on 2860469 later today. Thanks, Matt Weber |
From: Eric G. <eg...@py...> - 2009-09-21 14:19:49
|
Looking at releasing the 3rd and final beta. Matt, go ahead and complete 2860469 and commit your patch for 2860459. I have a fix for 2862158. Once these are complete we'll release 01.0b3. That will put us in a feature freeze and we'll concentrate on an RC release where we can focus on strictly bug fixes. -Eric |
From: Eric G. <eg...@py...> - 2009-09-21 13:24:21
|
On Sep 19, 2009, at 12:03 PM, Matt Weber wrote: > Also, do we really need all these comments for each method: > > @status: stable > @since: Jul 18 2009 > @author: U{Eric Gaumer<mailto:eg...@ma...>} > > I want things like description and input/output parameters, but all > these tags are redundant and basically same the same thing. I think > these particular things can be dropped because it is going to be > hard to keep them up to date, and they really add no value. Can I > remove them? Yea these can be removed. I'm looking to move away from epydoc anyway (in favor of sphinx). -Eric |
From: Eric G. <eg...@py...> - 2009-09-21 13:21:18
|
On Sep 20, 2009, at 3:08 PM, Matt Weber wrote: > Attached is a patch that completely refactors the packet api to be > PEP-8 compliant. I have also added unit tests for every single > method to ensure that it works 100% as expected. During the > refactor, I removed some of the methods I felt were unnecessary due > to duplicated functionality. For example I removed GetValue, and > moved its functionality into Get. Why would we need a Get that > raises a KeyError, or a GetValue that returns a different value? We > can always use GetValue and check for the default. In FAST stages, > I never used anything but GetValue. If you disagree, it will be > trivial to add these methods back. Please review the patch and let > me know what you think before I start refactoring the components to > work with this api. Agreed. Those methods are silly and should be removed. -Eric |
From: Eric G. <eg...@py...> - 2009-09-21 13:08:21
|
<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">On Sep 20, 2009, at 5:53 PM, Matt Weber wrote:<br><br><blockquote type="cite">See feature request 2860459 for a patch that includes all the<br></blockquote><blockquote type="cite">refactored components.<br></blockquote><br>Please don't top post, it makes it very difficult for others to follow the thread.<br><br>-Eric<br><br><br></body></html> |
From: Matt W. <ma...@ma...> - 2009-09-20 21:53:55
|
See feature request 2860459 for a patch that includes all the refactored components. Thanks, Matt Weber On Sep 20, 2009, at 12:08 PM, Matt Weber wrote: > Attached is a patch that completely refactors the packet api to be > PEP-8 compliant. I have also added unit tests for every single > method to ensure that it works 100% as expected. During the > refactor, I removed some of the methods I felt were unnecessary due > to duplicated functionality. For example I removed GetValue, and > moved its functionality into Get. Why would we need a Get that > raises a KeyError, or a GetValue that returns a different value? We > can always use GetValue and check for the default. In FAST stages, > I never used anything but GetValue. If you disagree, it will be > trivial to add these methods back. Please review the patch and let > me know what you think before I start refactoring the components to > work with this api. > > Thanks, > > Matt Weber > > <pypes-2860459.patch> > > On Sep 19, 2009, at 9:03 AM, Matt Weber wrote: > >> I wanted to use the current GetMeta just like said with the default >> attribute name, etc, but didn't want to break any components, so I >> just adapted it to work. I definitely think this is how metadata >> should work though. >> >> As far as the method names go.... I absolutely HATE the camel >> case. As you said, your intentions were good, but I don't think it >> is necessary. FAST is old and clunky, and people are going to have >> to do some type of refactoring, so changing a few method names is >> not that big of a deal. All of the core pypes code should follow >> PEP-8 standards. I will refactor this patch and all the >> components that break from the change. >> >> Also, do we really need all these comments for each method: >> >> @status: stable >> @since: Jul 18 2009 >> @author: U{Eric Gaumer<mailto:eg...@ma...>} >> >> I want things like description and input/output parameters, but all >> these tags are redundant and basically same the same thing. I >> think these particular things can be dropped because it is going to >> be hard to keep them up to date, and they really add no value. Can >> I remove them? >> >> Thanks, >> >> Matt Weber >> >> On Sep 19, 2009, at 5:29 AM, Eric Gaumer wrote: >> >>> >>> On Sep 19, 2009, at 1:58 AM, Matt Weber wrote: >>> >>>> I wanted to added packet level metadata, so I created a patch that >>>> sets packet metadata to a special attribute called '__meta__'. >>>> Now, >>>> this can be a problem if someone wants to use '__meta__' as an >>>> attribute in their packet. This is also a current problem with >>>> metadata because internally attribute values for packets are >>>> stored in >>>> a dictionary field called 'value'. Metadata for attributes are >>>> stored >>>> in whatever meta key you specify. So, if someone to set a metadata >>>> field called 'value' on any field, it will break the packet api. >>>> >>>> To resolve these problems, '__meta__' should be a reserved >>>> attribute >>>> and 'value' should be a reserved metadata keyword. When you >>>> attempt >>>> to set a reserved attribute or meta key, you should get KeyError. >>>> >>>> I implemented this functionality in the patch attached to bug >>>> 2860459. The patch completely refactors the internal packet api to >>>> support this functionality. All unit tests pass, as do the new >>>> ones I >>>> added. Also note that I refactored the 'value' key into >>>> '__value__' >>>> to be consistent with my '__meta__' attribute and reduce the >>>> chances >>>> for a name conflict. >>>> >>>> What do you think of this "reserved" concept? See http://sourceforge.net/tracker/?func=detail&aid=2860459&group_id=271766&atid=1155513 >>> >>> Conceptually I think it's fine although I'm not sure the interface >>> is very intuitive. As it exists today you have: >>> >>> GetMetaValue('myattr', 'metaname', default=None) >>> >>> Which basically says, get the meta value on the field myattr for >>> metaname. >>> >>> You're proposing that the new interface would be: >>> >>> GetMetaValue(None, 'metaname', default=None) >>> >>> This seems unintuitive to me. Why not just have it be: >>> >>> GetMetaValue('metaname', default=None) >>> >>> Why have to pass in an attribute of None? It will always be None >>> in the case of packet level meta-data so why not just create >>> additional methods to handle packet level meta-data? >>> >>> Of course the above syntax wouldn't work since you can't overload >>> methods in Python (at least not in the traditional C++ sense). So >>> we have a few alternatives. >>> >>> GetPacketMetaValue('metaname', default=None) >>> >>> These long names are a bit frustrating in my opinion. What about: >>> >>> GetMetaValue('metaname', attr=None, default=None) >>> >>> If we flip the parameters then we can set a default value of None >>> on the attribute. >>> >>> This whole Packet interface was designed to mirror the FAST >>> docproc API. The idea being that this would make converting FAST >>> pipeline stages simpler since the method signatures match (that's >>> why we're using the camel case etc. here). >>> >>> Is it worthwhile to even do this? Does it really simplify things >>> that much? I'd much rather see: >>> >>> get_meta_value('metaname', attr=None, default=None) >>> >>> So now we would have: >>> >>> get_meta_value('lang', 'title') # get me the language of the >>> title attribute >>> get_meta_value('lang') # get me the language of the packet >>> >>> This should be a pretty simple re-factor but it means we'll have >>> to re-factor the existing components. If we're to do this then now >>> is the time (before too many components get written or too many >>> people start using this API). >>> >>> The question is do we really care to provide the same interface as >>> FAST? I would argue it's not necessary. Their interface is >>> antiquated and doesn't conform to PEP 8 standards. >>> >>> -Eric >>> >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Come build with us! The BlackBerry® Developer Conference in >>> SF, CA >>> is the only developer event you need to attend this year. >>> Jumpstart your >>> developing skills, take BlackBerry mobile applications to market >>> and stay >>> ahead of the curve. Join us from November 9-12, 2009. Register >>> now! >>> http://p.sf.net/sfu/devconf_______________________________________________ >>> Pypes-developer mailing list >>> Pyp...@li... >>> https://lists.sourceforge.net/lists/listinfo/pypes-developer >> >> ------------------------------------------------------------------------------ >> Come build with us! The BlackBerry® Developer Conference in SF, >> CA >> is the only developer event you need to attend this year. Jumpstart >> your >> developing skills, take BlackBerry mobile applications to market >> and stay >> ahead of the curve. Join us from November 9-12, 2009. Register >> now! >> http://p.sf.net/sfu/devconf_______________________________________________ >> Pypes-developer mailing list >> Pyp...@li... >> https://lists.sourceforge.net/lists/listinfo/pypes-developer > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart > your > developing skills, take BlackBerry mobile applications to market and > stay > ahead of the curve. Join us from November 9-12, 2009. Register > now! > http://p.sf.net/sfu/devconf_______________________________________________ > Pypes-developer mailing list > Pyp...@li... > https://lists.sourceforge.net/lists/listinfo/pypes-developer Thanks, Matt Weber |
From: Matt W. <ma...@ma...> - 2009-09-20 19:09:17
|
Attached is a patch that completely refactors the packet api to be PEP-8 compliant. I have also added unit tests for every single method to ensure that it works 100% as expected. During the refactor, I removed some of the methods I felt were unnecessary due to duplicated functionality. For example I removed GetValue, and moved its functionality into Get. Why would we need a Get that raises a KeyError, or a GetValue that returns a different value? We can always use GetValue and check for the default. In FAST stages, I never used anything but GetValue. If you disagree, it will be trivial to add these methods back. Please review the patch and let me know what you think before I start refactoring the components to work with this api. Thanks, Matt Weber On Sep 19, 2009, at 9:03 AM, Matt Weber wrote: > I wanted to use the current GetMeta just like said with the default > attribute name, etc, but didn't want to break any components, so I > just adapted it to work. I definitely think this is how metadata > should work though. > > As far as the method names go.... I absolutely HATE the camel case. > As you said, your intentions were good, but I don't think it is > necessary. FAST is old and clunky, and people are going to have to > do some type of refactoring, so changing a few method names is not > that big of a deal. All of the core pypes code should follow PEP-8 > standards. I will refactor this patch and all the components that > break from the change. > > Also, do we really need all these comments for each method: > > @status: stable > @since: Jul 18 2009 > @author: U{Eric Gaumer<mailto:eg...@ma...>} > > I want things like description and input/output parameters, but all > these tags are redundant and basically same the same thing. I think > these particular things can be dropped because it is going to be > hard to keep them up to date, and they really add no value. Can I > remove them? > > Thanks, > > Matt Weber > > On Sep 19, 2009, at 5:29 AM, Eric Gaumer wrote: > >> >> On Sep 19, 2009, at 1:58 AM, Matt Weber wrote: >> >>> I wanted to added packet level metadata, so I created a patch that >>> sets packet metadata to a special attribute called '__meta__'. Now, >>> this can be a problem if someone wants to use '__meta__' as an >>> attribute in their packet. This is also a current problem with >>> metadata because internally attribute values for packets are >>> stored in >>> a dictionary field called 'value'. Metadata for attributes are >>> stored >>> in whatever meta key you specify. So, if someone to set a metadata >>> field called 'value' on any field, it will break the packet api. >>> >>> To resolve these problems, '__meta__' should be a reserved attribute >>> and 'value' should be a reserved metadata keyword. When you attempt >>> to set a reserved attribute or meta key, you should get KeyError. >>> >>> I implemented this functionality in the patch attached to bug >>> 2860459. The patch completely refactors the internal packet api to >>> support this functionality. All unit tests pass, as do the new >>> ones I >>> added. Also note that I refactored the 'value' key into >>> '__value__' >>> to be consistent with my '__meta__' attribute and reduce the chances >>> for a name conflict. >>> >>> What do you think of this "reserved" concept? See http://sourceforge.net/tracker/?func=detail&aid=2860459&group_id=271766&atid=1155513 >> >> Conceptually I think it's fine although I'm not sure the interface >> is very intuitive. As it exists today you have: >> >> GetMetaValue('myattr', 'metaname', default=None) >> >> Which basically says, get the meta value on the field myattr for >> metaname. >> >> You're proposing that the new interface would be: >> >> GetMetaValue(None, 'metaname', default=None) >> >> This seems unintuitive to me. Why not just have it be: >> >> GetMetaValue('metaname', default=None) >> >> Why have to pass in an attribute of None? It will always be None in >> the case of packet level meta-data so why not just create >> additional methods to handle packet level meta-data? >> >> Of course the above syntax wouldn't work since you can't overload >> methods in Python (at least not in the traditional C++ sense). So >> we have a few alternatives. >> >> GetPacketMetaValue('metaname', default=None) >> >> These long names are a bit frustrating in my opinion. What about: >> >> GetMetaValue('metaname', attr=None, default=None) >> >> If we flip the parameters then we can set a default value of None >> on the attribute. >> >> This whole Packet interface was designed to mirror the FAST docproc >> API. The idea being that this would make converting FAST pipeline >> stages simpler since the method signatures match (that's why we're >> using the camel case etc. here). >> >> Is it worthwhile to even do this? Does it really simplify things >> that much? I'd much rather see: >> >> get_meta_value('metaname', attr=None, default=None) >> >> So now we would have: >> >> get_meta_value('lang', 'title') # get me the language of the title >> attribute >> get_meta_value('lang') # get me the language of the packet >> >> This should be a pretty simple re-factor but it means we'll have to >> re-factor the existing components. If we're to do this then now is >> the time (before too many components get written or too many people >> start using this API). >> >> The question is do we really care to provide the same interface as >> FAST? I would argue it's not necessary. Their interface is >> antiquated and doesn't conform to PEP 8 standards. >> >> -Eric >> >> >> >> >> ------------------------------------------------------------------------------ >> Come build with us! The BlackBerry® Developer Conference in SF, >> CA >> is the only developer event you need to attend this year. Jumpstart >> your >> developing skills, take BlackBerry mobile applications to market >> and stay >> ahead of the curve. Join us from November 9-12, 2009. Register >> now! >> http://p.sf.net/sfu/devconf_______________________________________________ >> Pypes-developer mailing list >> Pyp...@li... >> https://lists.sourceforge.net/lists/listinfo/pypes-developer > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart > your > developing skills, take BlackBerry mobile applications to market and > stay > ahead of the curve. Join us from November 9-12, 2009. Register > now! > http://p.sf.net/sfu/devconf_______________________________________________ > Pypes-developer mailing list > Pyp...@li... > https://lists.sourceforge.net/lists/listinfo/pypes-developer |
From: Matt W. <ma...@ma...> - 2009-09-19 16:03:41
|
I wanted to use the current GetMeta just like said with the default attribute name, etc, but didn't want to break any components, so I just adapted it to work. I definitely think this is how metadata should work though. As far as the method names go.... I absolutely HATE the camel case. As you said, your intentions were good, but I don't think it is necessary. FAST is old and clunky, and people are going to have to do some type of refactoring, so changing a few method names is not that big of a deal. All of the core pypes code should follow PEP-8 standards. I will refactor this patch and all the components that break from the change. Also, do we really need all these comments for each method: @status: stable @since: Jul 18 2009 @author: U{Eric Gaumer<mailto:eg...@ma...>} I want things like description and input/output parameters, but all these tags are redundant and basically same the same thing. I think these particular things can be dropped because it is going to be hard to keep them up to date, and they really add no value. Can I remove them? Thanks, Matt Weber On Sep 19, 2009, at 5:29 AM, Eric Gaumer wrote: > > On Sep 19, 2009, at 1:58 AM, Matt Weber wrote: > >> I wanted to added packet level metadata, so I created a patch that >> sets packet metadata to a special attribute called '__meta__'. Now, >> this can be a problem if someone wants to use '__meta__' as an >> attribute in their packet. This is also a current problem with >> metadata because internally attribute values for packets are stored >> in >> a dictionary field called 'value'. Metadata for attributes are >> stored >> in whatever meta key you specify. So, if someone to set a metadata >> field called 'value' on any field, it will break the packet api. >> >> To resolve these problems, '__meta__' should be a reserved attribute >> and 'value' should be a reserved metadata keyword. When you attempt >> to set a reserved attribute or meta key, you should get KeyError. >> >> I implemented this functionality in the patch attached to bug >> 2860459. The patch completely refactors the internal packet api to >> support this functionality. All unit tests pass, as do the new >> ones I >> added. Also note that I refactored the 'value' key into '__value__' >> to be consistent with my '__meta__' attribute and reduce the chances >> for a name conflict. >> >> What do you think of this "reserved" concept? See http://sourceforge.net/tracker/?func=detail&aid=2860459&group_id=271766&atid=1155513 > > Conceptually I think it's fine although I'm not sure the interface > is very intuitive. As it exists today you have: > > GetMetaValue('myattr', 'metaname', default=None) > > Which basically says, get the meta value on the field myattr for > metaname. > > You're proposing that the new interface would be: > > GetMetaValue(None, 'metaname', default=None) > > This seems unintuitive to me. Why not just have it be: > > GetMetaValue('metaname', default=None) > > Why have to pass in an attribute of None? It will always be None in > the case of packet level meta-data so why not just create additional > methods to handle packet level meta-data? > > Of course the above syntax wouldn't work since you can't overload > methods in Python (at least not in the traditional C++ sense). So we > have a few alternatives. > > GetPacketMetaValue('metaname', default=None) > > These long names are a bit frustrating in my opinion. What about: > > GetMetaValue('metaname', attr=None, default=None) > > If we flip the parameters then we can set a default value of None on > the attribute. > > This whole Packet interface was designed to mirror the FAST docproc > API. The idea being that this would make converting FAST pipeline > stages simpler since the method signatures match (that's why we're > using the camel case etc. here). > > Is it worthwhile to even do this? Does it really simplify things > that much? I'd much rather see: > > get_meta_value('metaname', attr=None, default=None) > > So now we would have: > > get_meta_value('lang', 'title') # get me the language of the title > attribute > get_meta_value('lang') # get me the language of the packet > > This should be a pretty simple re-factor but it means we'll have to > re-factor the existing components. If we're to do this then now is > the time (before too many components get written or too many people > start using this API). > > The question is do we really care to provide the same interface as > FAST? I would argue it's not necessary. Their interface is > antiquated and doesn't conform to PEP 8 standards. > > -Eric > > > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart > your > developing skills, take BlackBerry mobile applications to market and > stay > ahead of the curve. Join us from November 9-12, 2009. Register > now! > http://p.sf.net/sfu/devconf_______________________________________________ > Pypes-developer mailing list > Pyp...@li... > https://lists.sourceforge.net/lists/listinfo/pypes-developer |
From: Eric G. <eg...@py...> - 2009-09-19 12:56:56
|
On Sep 19, 2009, at 1:58 AM, Matt Weber wrote: > I wanted to added packet level metadata, so I created a patch that > sets packet metadata to a special attribute called '__meta__'. Now, > this can be a problem if someone wants to use '__meta__' as an > attribute in their packet. This is also a current problem with > metadata because internally attribute values for packets are stored in > a dictionary field called 'value'. Metadata for attributes are stored > in whatever meta key you specify. So, if someone to set a metadata > field called 'value' on any field, it will break the packet api. > > To resolve these problems, '__meta__' should be a reserved attribute > and 'value' should be a reserved metadata keyword. When you attempt > to set a reserved attribute or meta key, you should get KeyError. > > I implemented this functionality in the patch attached to bug > 2860459. The patch completely refactors the internal packet api to > support this functionality. All unit tests pass, as do the new ones I > added. Also note that I refactored the 'value' key into '__value__' > to be consistent with my '__meta__' attribute and reduce the chances > for a name conflict. > > What do you think of this "reserved" concept? See http://sourceforge.net/tracker/?func=detail&aid=2860459&group_id=271766&atid=1155513 Conceptually I think it's fine although I'm not sure the interface is very intuitive. As it exists today you have: GetMetaValue('myattr', 'metaname', default=None) Which basically says, get the meta value on the field myattr for metaname. You're proposing that the new interface would be: GetMetaValue(None, 'metaname', default=None) This seems unintuitive to me. Why not just have it be: GetMetaValue('metaname', default=None) Why have to pass in an attribute of None? It will always be None in the case of packet level meta-data so why not just create additional methods to handle packet level meta-data? Of course the above syntax wouldn't work since you can't overload methods in Python (at least not in the traditional C++ sense). So we have a few alternatives. GetPacketMetaValue('metaname', default=None) These long names are a bit frustrating in my opinion. What about: GetMetaValue('metaname', attr=None, default=None) If we flip the parameters then we can set a default value of None on the attribute. This whole Packet interface was designed to mirror the FAST docproc API. The idea being that this would make converting FAST pipeline stages simpler since the method signatures match (that's why we're using the camel case etc. here). Is it worthwhile to even do this? Does it really simplify things that much? I'd much rather see: get_meta_value('metaname', attr=None, default=None) So now we would have: get_meta_value('lang', 'title') # get me the language of the title attribute get_meta_value('lang') # get me the language of the packet This should be a pretty simple re-factor but it means we'll have to re- factor the existing components. If we're to do this then now is the time (before too many components get written or too many people start using this API). The question is do we really care to provide the same interface as FAST? I would argue it's not necessary. Their interface is antiquated and doesn't conform to PEP 8 standards. -Eric |
From: Matt W. <ma...@ma...> - 2009-09-19 05:58:44
|
I wanted to added packet level metadata, so I created a patch that sets packet metadata to a special attribute called '__meta__'. Now, this can be a problem if someone wants to use '__meta__' as an attribute in their packet. This is also a current problem with metadata because internally attribute values for packets are stored in a dictionary field called 'value'. Metadata for attributes are stored in whatever meta key you specify. So, if someone to set a metadata field called 'value' on any field, it will break the packet api. To resolve these problems, '__meta__' should be a reserved attribute and 'value' should be a reserved metadata keyword. When you attempt to set a reserved attribute or meta key, you should get KeyError. I implemented this functionality in the patch attached to bug 2860459. The patch completely refactors the internal packet api to support this functionality. All unit tests pass, as do the new ones I added. Also note that I refactored the 'value' key into '__value__' to be consistent with my '__meta__' attribute and reduce the chances for a name conflict. What do you think of this "reserved" concept? See http://sourceforge.net/tracker/?func=detail&aid=2860459&group_id=271766&atid=1155513 . Thanks, Matt Weber |
From: Eric G. <eg...@py...> - 2009-09-17 13:03:24
|
On Sep 17, 2009, at 1:21 AM, Matt Weber wrote: > I am getting this error while building the developer buildout: > > matt@matt-laptop:(pypes)$ bin/buildout > Develop: '/Users/matt/Projects/pypes/core/trunk' > warning: no files found matching '*' under directory 'api-docs' > Develop: '/Users/matt/Projects/pypes/ui/trunk' > Traceback (most recent call last): > File "/var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+++TI/-Tmp-/tmpOM2nxS", > line 11, in <module> > execfile('/Users/matt/Projects/pypes/ui/trunk/setup.py') > File "/Users/matt/Projects/pypes/ui/trunk/setup.py", line 138, in > <module> > """, > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/distutils/core.py", line 113, in setup > _setup_distribution = dist = klass(attrs) > File "build/bdist.macosx-10.3-i386/egg/setuptools/dist.py", line > 223, in __init__ > File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/distutils/dist.py", line 270, in __init__ > self.finalize_options() > File "build/bdist.macosx-10.3-i386/egg/setuptools/dist.py", line > 255, in finalize_options > File "build/bdist.macosx-10.3-i386/egg/pkg_resources.py", line > 1925, in require > File "build/bdist.macosx-10.3-i386/egg/pkg_resources.py", line 522, > in resolve > File "build/bdist.macosx-10.3-i386/egg/pkg_resources.py", line 758, > in best_match > File "build/bdist.macosx-10.3-i386/egg/pkg_resources.py", line 770, > in obtain > File "build/bdist.macosx-10.3-i386/egg/setuptools/dist.py", line > 286, in fetch_build_egg > File "build/bdist.macosx-10.3-i386/egg/setuptools/command/ > easy_install.py", line 446, in easy_install > File "build/bdist.macosx-10.3-i386/egg/setuptools/command/ > easy_install.py", line 476, in install_item > File "build/bdist.macosx-10.3-i386/egg/setuptools/command/ > easy_install.py", line 655, in install_eggs > File "build/bdist.macosx-10.3-i386/egg/setuptools/command/ > easy_install.py", line 930, in build_and_install > File "build/bdist.macosx-10.3-i386/egg/setuptools/command/ > easy_install.py", line 921, in run_setup > distutils.errors.DistutilsError: Setup script exited with error: Setup > script exited with error: Setup script exited with error: Setup script > exited with error: Setup script exited with error: Setup script exited > with error: Setup script exited with error: Setup script exited with > error: Setup script exited with error: Setup script exited with error: > Setup script exited with error: Setup script exited with error: Setup > script exited with error: Setup script exited with error: Setup script > exited with error: Setup script exited with error: Setup script exited > with error: Setup script exited with error: Setup script exited with > error: Setup script exited with error: Setup script exited with error: > Setup script exited with error: /var/folders/fq/ > fquxhtMKGyiIfBwE3TLSwk+ > ++TI/-Tmp-/easy_install-h5NFz2/AuthKit-0.4.4/temp/easy_install-7TewvZ/ > py-dom-xpath-0.1/temp/easy_install-CFQAna/AuthKit-0.4.4/temp/ > easy_install-rdtYHv/py-dom-xpath-0.1/temp/easy_install-JCPCBK/ > AuthKit-0.4.4/temp/easy_install-FlRFwU/py-dom-xpath-0.1/temp/ > easy_install-FYcFs1/AuthKit-0.4.4/temp/easy_install-50g9Hq/py-dom- > xpath-0.1/temp/easy_install-jLerWf/AuthKit-0.4.4/temp/easy_install- > sBs654/py-dom-xpath-0.1/temp/easy_install-3ZVl_l/AuthKit-0.4.4/temp/ > easy_install-E1qFm1/py-dom-xpath-0.1/temp/easy_install-amc5Vn/ > AuthKit-0.4.4/temp/easy_install-m6gvKZ/py-dom-xpath-0.1/temp/ > easy_install-32FW7J/AuthKit-0.4.4/temp/easy_install-6PzeZa/py-dom- > xpath-0.1/temp/easy_install-Y1_YxH/AuthKit-0.4.4/temp/easy_install- > sLgNQ1/py-dom-xpath-0.1/temp/easy_install-LAwJ9j/AuthKit-0.4.4/temp/ > easy_install-aNPLvc/py-dom-xpath-0.1/temp/easy_install-vP55ak/ > AuthKit-0.4.4/temp/easy_install-MNhq0t/py-dom-xpath-0.1/temp/ > easy_install-Euyh60/AuthKit-0.4.4/authkit/users/sqlalchemy_driver/ > sqlalchemy_044.py: File name too long > While: > Installing. > Processing develop directory '/Users/matt/Projects/pypes/ui/trunk'. > > An internal error occured due to a bug in either zc.buildout or in a > recipe being used: > Traceback (most recent call last): > File "/private/var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+++TI/-Tmp-/ > tmp6SLLvn/zc.buildout-1.4.1-py2.6.egg/zc/buildout/buildout.py", line > 1659, in main > File "/private/var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+++TI/-Tmp-/ > tmp6SLLvn/zc.buildout-1.4.1-py2.6.egg/zc/buildout/buildout.py", line > 394, in install > File "/private/var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+++TI/-Tmp-/ > tmp6SLLvn/zc.buildout-1.4.1-py2.6.egg/zc/buildout/buildout.py", line > 634, in _develop > File "/private/var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+++TI/-Tmp-/ > tmp6SLLvn/zc.buildout-1.4.1-py2.6.egg/zc/buildout/easy_install.py", > line 895, in develop > AssertionError Looks like a bug in setuptools. Other projects have reported the same error. Looks like it's adding dependencies recursively. It's not buildout because I was able to reproduce the problem using setuptools (same exact error). I resolved it by removing some eggs from my global site-packages directory (Pylons and its dependencies). The reason I tried this is because I had a virtual env setup and the build worked fine. Same build produced the above error using global python. The quick fix that others seems to be using is to just delete the egg- info directory. In this case, Rm -Rf the pypesvds.egg-info and the build should run fine. Still would like to figure out the root cause. -Eric |
From: Matt W. <ma...@ma...> - 2009-09-17 05:21:18
|
I am getting this error while building the developer buildout: matt@matt-laptop:(pypes)$ bin/buildout Develop: '/Users/matt/Projects/pypes/core/trunk' warning: no files found matching '*' under directory 'api-docs' Develop: '/Users/matt/Projects/pypes/ui/trunk' Traceback (most recent call last): File "/var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+++TI/-Tmp-/tmpOM2nxS", line 11, in <module> execfile('/Users/matt/Projects/pypes/ui/trunk/setup.py') File "/Users/matt/Projects/pypes/ui/trunk/setup.py", line 138, in <module> """, File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ python2.6/distutils/core.py", line 113, in setup _setup_distribution = dist = klass(attrs) File "build/bdist.macosx-10.3-i386/egg/setuptools/dist.py", line 223, in __init__ File "/Library/Frameworks/Python.framework/Versions/2.6/lib/ python2.6/distutils/dist.py", line 270, in __init__ self.finalize_options() File "build/bdist.macosx-10.3-i386/egg/setuptools/dist.py", line 255, in finalize_options File "build/bdist.macosx-10.3-i386/egg/pkg_resources.py", line 1925, in require File "build/bdist.macosx-10.3-i386/egg/pkg_resources.py", line 522, in resolve File "build/bdist.macosx-10.3-i386/egg/pkg_resources.py", line 758, in best_match File "build/bdist.macosx-10.3-i386/egg/pkg_resources.py", line 770, in obtain File "build/bdist.macosx-10.3-i386/egg/setuptools/dist.py", line 286, in fetch_build_egg File "build/bdist.macosx-10.3-i386/egg/setuptools/command/ easy_install.py", line 446, in easy_install File "build/bdist.macosx-10.3-i386/egg/setuptools/command/ easy_install.py", line 476, in install_item File "build/bdist.macosx-10.3-i386/egg/setuptools/command/ easy_install.py", line 655, in install_eggs File "build/bdist.macosx-10.3-i386/egg/setuptools/command/ easy_install.py", line 930, in build_and_install File "build/bdist.macosx-10.3-i386/egg/setuptools/command/ easy_install.py", line 921, in run_setup distutils.errors.DistutilsError: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: Setup script exited with error: /var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+ ++TI/-Tmp-/easy_install-h5NFz2/AuthKit-0.4.4/temp/easy_install-7TewvZ/ py-dom-xpath-0.1/temp/easy_install-CFQAna/AuthKit-0.4.4/temp/ easy_install-rdtYHv/py-dom-xpath-0.1/temp/easy_install-JCPCBK/ AuthKit-0.4.4/temp/easy_install-FlRFwU/py-dom-xpath-0.1/temp/ easy_install-FYcFs1/AuthKit-0.4.4/temp/easy_install-50g9Hq/py-dom- xpath-0.1/temp/easy_install-jLerWf/AuthKit-0.4.4/temp/easy_install- sBs654/py-dom-xpath-0.1/temp/easy_install-3ZVl_l/AuthKit-0.4.4/temp/ easy_install-E1qFm1/py-dom-xpath-0.1/temp/easy_install-amc5Vn/ AuthKit-0.4.4/temp/easy_install-m6gvKZ/py-dom-xpath-0.1/temp/ easy_install-32FW7J/AuthKit-0.4.4/temp/easy_install-6PzeZa/py-dom- xpath-0.1/temp/easy_install-Y1_YxH/AuthKit-0.4.4/temp/easy_install- sLgNQ1/py-dom-xpath-0.1/temp/easy_install-LAwJ9j/AuthKit-0.4.4/temp/ easy_install-aNPLvc/py-dom-xpath-0.1/temp/easy_install-vP55ak/ AuthKit-0.4.4/temp/easy_install-MNhq0t/py-dom-xpath-0.1/temp/ easy_install-Euyh60/AuthKit-0.4.4/authkit/users/sqlalchemy_driver/ sqlalchemy_044.py: File name too long While: Installing. Processing develop directory '/Users/matt/Projects/pypes/ui/trunk'. An internal error occured due to a bug in either zc.buildout or in a recipe being used: Traceback (most recent call last): File "/private/var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+++TI/-Tmp-/ tmp6SLLvn/zc.buildout-1.4.1-py2.6.egg/zc/buildout/buildout.py", line 1659, in main File "/private/var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+++TI/-Tmp-/ tmp6SLLvn/zc.buildout-1.4.1-py2.6.egg/zc/buildout/buildout.py", line 394, in install File "/private/var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+++TI/-Tmp-/ tmp6SLLvn/zc.buildout-1.4.1-py2.6.egg/zc/buildout/buildout.py", line 634, in _develop File "/private/var/folders/fq/fquxhtMKGyiIfBwE3TLSwk+++TI/-Tmp-/ tmp6SLLvn/zc.buildout-1.4.1-py2.6.egg/zc/buildout/easy_install.py", line 895, in develop AssertionError Any ideas? Thanks, Matt Weber |