pysnmp-users Mailing List for SNMP library for Python (Page 62)
Brought to you by:
elie
Menu
▾
▴
pysnmp-users — PySNMP usage issues
You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(2) |
Jul
|
Aug
(2) |
Sep
(3) |
Oct
(2) |
Nov
(4) |
Dec
(4) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
|
Feb
(3) |
Mar
(16) |
Apr
(3) |
May
(6) |
Jun
|
Jul
(7) |
Aug
(12) |
Sep
(4) |
Oct
(2) |
Nov
(11) |
Dec
(6) |
| 2004 |
Jan
(1) |
Feb
(41) |
Mar
(4) |
Apr
(1) |
May
(6) |
Jun
|
Jul
(8) |
Aug
(5) |
Sep
(8) |
Oct
(10) |
Nov
(3) |
Dec
(1) |
| 2005 |
Jan
|
Feb
(4) |
Mar
(8) |
Apr
(4) |
May
(10) |
Jun
(5) |
Jul
(10) |
Aug
(6) |
Sep
(6) |
Oct
(39) |
Nov
(20) |
Dec
(21) |
| 2006 |
Jan
(14) |
Feb
(6) |
Mar
(15) |
Apr
|
May
(5) |
Jun
(16) |
Jul
(16) |
Aug
(18) |
Sep
(35) |
Oct
(12) |
Nov
(3) |
Dec
(3) |
| 2007 |
Jan
(12) |
Feb
(19) |
Mar
(27) |
Apr
(14) |
May
(32) |
Jun
|
Jul
(35) |
Aug
(11) |
Sep
(11) |
Oct
(6) |
Nov
(13) |
Dec
(4) |
| 2008 |
Jan
(7) |
Feb
(4) |
Mar
(2) |
Apr
(3) |
May
(3) |
Jun
(5) |
Jul
(5) |
Aug
(1) |
Sep
(2) |
Oct
(3) |
Nov
(19) |
Dec
(7) |
| 2009 |
Jan
(2) |
Feb
(2) |
Mar
(12) |
Apr
(16) |
May
(16) |
Jun
(23) |
Jul
(7) |
Aug
(2) |
Sep
(17) |
Oct
(20) |
Nov
(20) |
Dec
(5) |
| 2010 |
Jan
(11) |
Feb
(17) |
Mar
(20) |
Apr
(7) |
May
(6) |
Jun
(14) |
Jul
(5) |
Aug
(10) |
Sep
(23) |
Oct
(16) |
Nov
(32) |
Dec
(21) |
| 2011 |
Jan
(6) |
Feb
(11) |
Mar
(3) |
Apr
(1) |
May
(4) |
Jun
(1) |
Jul
(4) |
Aug
(8) |
Sep
(20) |
Oct
(9) |
Nov
(29) |
Dec
(25) |
| 2012 |
Jan
(2) |
Feb
(5) |
Mar
(2) |
Apr
(22) |
May
(21) |
Jun
(7) |
Jul
(6) |
Aug
(2) |
Sep
(12) |
Oct
(20) |
Nov
(17) |
Dec
(17) |
| 2013 |
Jan
(4) |
Feb
(4) |
Mar
(8) |
Apr
(8) |
May
(10) |
Jun
(2) |
Jul
(23) |
Aug
(12) |
Sep
(14) |
Oct
(12) |
Nov
(4) |
Dec
(18) |
| 2014 |
Jan
(2) |
Feb
(7) |
Mar
(3) |
Apr
(8) |
May
(7) |
Jun
(1) |
Jul
(5) |
Aug
(2) |
Sep
(8) |
Oct
|
Nov
(2) |
Dec
(1) |
| 2015 |
Jan
(3) |
Feb
(15) |
Mar
(14) |
Apr
(4) |
May
(6) |
Jun
(3) |
Jul
(1) |
Aug
(6) |
Sep
(5) |
Oct
(21) |
Nov
(43) |
Dec
(10) |
| 2016 |
Jan
|
Feb
(2) |
Mar
(6) |
Apr
(4) |
May
(6) |
Jun
(2) |
Jul
(6) |
Aug
(5) |
Sep
(4) |
Oct
|
Nov
(3) |
Dec
(11) |
| 2017 |
Jan
(1) |
Feb
(8) |
Mar
|
Apr
(7) |
May
|
Jun
|
Jul
(5) |
Aug
|
Sep
(9) |
Oct
(8) |
Nov
|
Dec
|
| 2018 |
Jan
|
Feb
|
Mar
(2) |
Apr
(2) |
May
(1) |
Jun
(2) |
Jul
(2) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2019 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
|
Dec
|
| 2020 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
|
|
From: Mike C. F. <mcf...@ro...> - 2004-02-17 12:04:35
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> I've moved TwistedSNMP to a SourceForge project. You can get the latest version (0.2.6) there. In the future I will only be announcing particularly notable changes to the package here, as people can subscribe to the project releases on SourceForge if they are really interested in version-by-version updates. http://twistedsnmp.sourceforge.net/ Have fun all, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-17 12:01:18
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Ilya Etingof wrote: >>Anyway, the question is, how do we go about collaborating, particularly >>submitting the patches? Do we want to use the SourceForge project's >>patch-manager for such things? Or just directly email Ilya? Or do >>something else? >> >> > >Sorry for being off-line for a while. > >Let's do it through SF patch management tool if you don't mind. > > Okay, will do. Have fun, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Richard S. <ri...@go...> - 2004-02-16 23:24:44
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Ilya Etingof wrote: > You seem to be getting an InformRequest PDU in a message from your > cisco box instead of Trap PDU which is expected by snmptrapd script. > If my guess is correct, I'd take a look at your cisco config/debug > to see why it's doing so. This would be a class example of 'Too many cooks, spoiling the broth'... It seems someone else was doing some config changes and added the Informs in for me... /me slaps person responsible upside the head. Thanks -- ~ Richard |
|
From: Ilya E. <il...@gl...> - 2004-02-16 21:27:43
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> 1) If you are not using pysnmp-3.4.1 & pysnmp-apps -- please, upgrade 2) Otherwise, try adding '-d' at the command line and send me the output. ps. sorry for delay -- I seem to've overlooked your posting > I'm trying to get several values from my snmp agents. I want to pass > several oid's, several are leafs and the others not. But if I pass this > list to pysnmpwalk, something strange happens: > A) Walk > $ ./snmpwalk.py pierre public .1.3.6.1.4.1.2021.4 > .1.3.6.1.4.1.2021.4.1.0 ---> 0 > .1.3.6.1.4.1.2021.4.2.0 ---> 'swap' > .1.3.6.1.4.1.2021.4.3.0 ---> 0 > .1.3.6.1.4.1.2021.4.4.0 ---> 0 > .1.3.6.1.4.1.2021.4.5.0 ---> 190960 > .1.3.6.1.4.1.2021.4.6.0 ---> 9760 > .1.3.6.1.4.1.2021.4.11.0 ---> 9760 > .1.3.6.1.4.1.2021.4.12.0 ---> 16000 > .1.3.6.1.4.1.2021.4.13.0 ---> 0 > .1.3.6.1.4.1.2021.4.14.0 ---> 4280 > .1.3.6.1.4.1.2021.4.15.0 ---> 133016 > .1.3.6.1.4.1.2021.4.100.0 ---> 1 > .1.3.6.1.4.1.2021.4.101.0 ---> 'Running out of swap space (0)' > > B) a leaf > ./snmpwalk.py -C i pierre hada .1.3.6.1.4.1.2021.11.9.0 > .1.3.6.1.4.1.2021.11.9.0 ---> 4 > > C) Both > ./snmpwalk.py -C i pierre hada .1.3.6.1.4.1.2021.4 > .1.3.6.1.4.1.2021.11.9.0 > .1.3.6.1.4.1.2021.4 ---> '' > .1.3.6.1.4.1.2021.11.9.0 ---> '' <------------ ??????? > .1.3.6.1.4.1.2021.4.1.0 ---> 0 > .1.3.6.1.4.1.2021.4.2.0 ---> 'swap' > .1.3.6.1.4.1.2021.4.3.0 ---> 0 > .1.3.6.1.4.1.2021.4.4.0 ---> 0 > .1.3.6.1.4.1.2021.4.5.0 ---> 190960 > .1.3.6.1.4.1.2021.4.6.0 ---> 9728 > .1.3.6.1.4.1.2021.4.11.0 ---> 9728 > .1.3.6.1.4.1.2021.4.12.0 ---> 16000 > .1.3.6.1.4.1.2021.4.13.0 ---> 0 > .1.3.6.1.4.1.2021.4.14.0 ---> 4280 > .1.3.6.1.4.1.2021.4.15.0 ---> 133044 > .1.3.6.1.4.1.2021.4.100.0 ---> 1 > .1.3.6.1.4.1.2021.4.101.0 ---> 'Running out of swap space (0)' > > Why now the oid .1.3.6.1.4.1.2021.11.9.0 returns ''. |
|
From: Ilya E. <il...@gl...> - 2004-02-16 21:09:31
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> You seem to be getting an InformRequest PDU in a message from your cisco box instead of Trap PDU which is expected by snmptrapd script. If my guess is correct, I'd take a look at your cisco config/debug to see why it's doing so. In any case, I'd suggest you to upgrade to pysnmp-3.4.1 & pysnmp-apps as they, besides other improvements, might have some old bugs fixed (and new created ;). -ilya > I'm just putting together a quick monitoring Trap Server (Quick and > dirty for the moment (don't ask...)) > > I'm running a script similar to the example snmptrapd.py supplied in the > examples, however I keep getting the following: > > Traceback (most recent call last): > File "./snmptrapd.py", line 75, in ? > req.decode(question) > File > "/usr/lib/python2.2/site-packages/pysnmp/asn1/encoding/ber/base.py", > line 276, in berDecode > input = self[key].berDecode(input) > File > "/usr/lib/python2.2/site-packages/pysnmp/asn1/encoding/ber/base.py", > line 348, in berDecode > raise error.TypeMismatchError('Unregistered tag %o at %s' %\ > pysnmp.asn1.encoding.ber.error.TypeMismatchError: Unregistered tag 246 > at Pdus |
|
From: Richard S. <ri...@go...> - 2004-02-16 20:45:40
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Hi guy's I'm just putting together a quick monitoring Trap Server (Quick and dirty for the moment (don't ask...)) I'm running a script similar to the example snmptrapd.py supplied in the examples, however I keep getting the following: Traceback (most recent call last): File "./snmptrapd.py", line 75, in ? req.decode(question) File "/usr/lib/python2.2/site-packages/pysnmp/asn1/encoding/ber/base.py", line 276, in berDecode input = self[key].berDecode(input) File "/usr/lib/python2.2/site-packages/pysnmp/asn1/encoding/ber/base.py", line 348, in berDecode raise error.TypeMismatchError('Unregistered tag %o at %s' %\ pysnmp.asn1.encoding.ber.error.TypeMismatchError: Unregistered tag 246 at Pdus These traps are coming from a Cisco 7206(IOS: 12.1 (again, enough with the asking)) Any ideas? I'm a bit stumped really... -- ~ Richard |
|
From: Ilya E. <il...@gl...> - 2004-02-16 16:00:51
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> [ skipped ] > That second if check would seem to be something that could be done once > at class initialisation time, rather than continually checking in the > very tight loop (this method is called 157,000 some odd times just in > the (rather small) test suite for TwistedSNMP). The idea behind this runtime checking was to support dynamic changes to ASN1 objects constraints... > Haven't gone through to try dictionary lookups for static-value sets. > That's only going to trim a few percent of overhead at maximum (small > sets) I would guess. Right, constraints lists are basically just a few items long. I feel like it's worth re-working the whole constraints checking machinery... If you've already done something about it, please, let me know. BTW, I've added a pyunit-based test suite (pysnmp.test at version-3-3-x-branch at CVS) which might also test the most recent pysnmp code. So, if you're making changes to the asn1 stuff, maybe it's worth trying it out. -ilya |
|
From: Ilya E. <il...@gl...> - 2004-02-16 07:33:03
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> > Anyway, the question is, how do we go about collaborating, particularly > submitting the patches? Do we want to use the SourceForge project's > patch-manager for such things? Or just directly email Ilya? Or do > something else? Sorry for being off-line for a while. Let's do it through SF patch management tool if you don't mind. Thanks, ilya |
|
From: John P S. <sp...@is...> - 2004-02-16 01:53:14
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> On Sat, Feb 14, 2004 at 06:43:52AM -0500, Mike C. Fletcher wrote: > Anyway, the question is, how do we go about collaborating, particularly > submitting the patches? Do we want to use the SourceForge project's > patch-manager for such things? Or just directly email Ilya? Or do > something else? Since I'm interested in seeing patches, I'd be in favor of you posting the patch on SF.net and just sending the URL for the patch to the list. |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-14 11:45:56
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> I've just done a bit of optimising the _subtype_constraint method and breaking the constraint-checking code into its own classes & module to make creating an accelerator in C easier. Even without the accelerator module, these changes take the _subtype_constraint method from 7.9/10.5% of load down to 2.6/7.4% of total load under the TwistedSNMP test suite (that's still 14s of 191s, btw). The two methods ValueSizeConstraint.__call__ and ValueRangeConstraint.__call__ each take 2.6% of total cumulative time. They are incredibly simple functions (and are basically identical, save for whether they operate on the value or len(value)), so rewriting in C should give a decent boost for very minimal work. Note: the new code does *not* do the checks for _subtype_single_value_constraint and the like. Basically, if a constraint object is present it is considered allowed. I would put those checks in a meta-class initialiser, but figuring out the whole class hierarchy to do so is a bit daunting. Anyway, the question is, how do we go about collaborating, particularly submitting the patches? Do we want to use the SourceForge project's patch-manager for such things? Or just directly email Ilya? Or do something else? Have fun all, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-13 09:18:21
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Mostly this release is just code cleanup and documentation so that others who want to work with it can find their way around. In particular, docstrings are updated throughout most of the code to provide an idea what each class/method/module is trying to do. Enjoy yourselves, Mike Version 0.2.5 * Altered timeout methods to catch exceptions within themselves and relay to the caller via the registered deferred object * Documentation enhancements * Patch to work around Python 2.3.x bug in the BSDDB wrapper class * Eliminated old algorithms for MassRetriever * Reduced default MassRetriever iteration delay to 0.005 instead of 0.01 seconds Hmm, forgot to send this message out earlier... _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-13 07:53:43
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Okay, finding that speed is actually turning into an issue, I've spent a few minutes profiling the test suite for TwistedSNMP. There are a few methods in particular that seem to really slow down the system: asn1\base.py:83 _subtype_constraint 7.9% total time 10.5% cumulative asn1\base.py:240 set 5.7% 19.0% asn1\base.py:479 __setitem__ 3.1% 5.4% asn1\encoding\ber\base.py:241 berEncode 3.0% 9.7% asn1\base.py:209 _setRawAsn1Value 2.9% 9.0% all in all, message decoding is taking 17.7% of total run-time, message encoding is taking 11.5% of total. Looking at _subtype_constraint, there's an unnecessary call (and lookup) for each len() call, and an unnecessarily iteration through a list for each of singleValueConstraint, permittedAlphabetConstraint, which, compared to the C-implemented dictionary lookup is likely going to be pretty slow. Eliminating just the len() calls drops the totals to 7.1% and 9.7%. if self.singleValueConstraint: if self._subtype_single_value_constraint is None: raise error.ValueConstraintError('Single value constraint not applicible to %s' % self.__class__.__name__) if value not in self.singleValueConstraint: raise error.ValueConstraintError('Single value constraint for %s: %s not within allowed values' % (self.__class__.__name__, str(value))) That second if check would seem to be something that could be done once at class initialisation time, rather than continually checking in the very tight loop (this method is called 157,000 some odd times just in the (rather small) test suite for TwistedSNMP). Haven't gone through to try dictionary lookups for static-value sets. That's only going to trim a few percent of overhead at maximum (small sets) I would guess. Anyway, no more time today. Enjoy all, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Ilya E. <il...@gl...> - 2004-02-11 10:39:44
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> > Hmm, caching is fine, but it can only go so far, and it's often > detrimental to application structure and maintainability (particularly > when a library starts requiring client apps to do caching of internal That's right. I just wanted to point out a hint that improves SNMP manager performance whenever it performs multiple queries within its lifetime. That also holds true for SNMP agent as it's inherently deals with request message many times in row. > objects). All in all I'd rather spend the time speeding up the core > structures if/when I do wind up needing more speed. Right. In fact, I've been approaching that several times but there still might be plenty of things to improve. ;) > Looking at the asn.base module, it looks as though there's a lot of > internal structure being allocated for each ASN object, an instance > dictionary, and two or three lists. For most of those objects I'd guess > we could use __slots__ to reduce memory thrashing somewhat (about 26 > bytes/object + 8 bytes/(attribute*record) if my calculations are > right). Slots are a bit of a pain, but this is exactly the kind of > situation for which they are intended, huge numbers of tiny objects with > well-defined structure. Well, that's probably a way to go. I wonder, though, whether slot'ting instance variables would noticably save CPU cycles? [ skipped ] > Need to do a hotspot profile of running code to know what to start > working on. The initialisation code looks pretty involved in a number > of cases... anyway, no time tonight, have to work on application-level > stuff. That's right -- on instantiation or value assignment to an asn1.* object, ASN1 constraints checking code gets evaluated. -ilya |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-11 08:56:04
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Ilya Etingof wrote: >>>but if anyone is interested in bringing a full high performance ASN >>>coder-decoder to twisted, i'd be willing to help. >>> >>> >>Ilya (creator of PySNMP) just released a new version of PySNMP >>yesterday. It's noticeably faster feeling than the 3.3.x version. I >>don't have time to do timings with it, but I'd guess it's probably >>taking 3/4 of the time of the previous version. Still likely not 0.02 >>seconds on getbulk requests, but noticeably faster. >> >> ... >Perhaps the most noticable bottleneck there lays in the fact that building >a tree of ASN1 objects involves massive classes instantiations. In the >latest pysnmp code (3.4.x) I attempted to save on object creation by caching >and reusing once created ASN1 objects. > > Hmm, caching is fine, but it can only go so far, and it's often detrimental to application structure and maintainability (particularly when a library starts requiring client apps to do caching of internal objects). All in all I'd rather spend the time speeding up the core structures if/when I do wind up needing more speed. Looking at the asn.base module, it looks as though there's a lot of internal structure being allocated for each ASN object, an instance dictionary, and two or three lists. For most of those objects I'd guess we could use __slots__ to reduce memory thrashing somewhat (about 26 bytes/object + 8 bytes/(attribute*record) if my calculations are right). Slots are a bit of a pain, but this is exactly the kind of situation for which they are intended, huge numbers of tiny objects with well-defined structure. For the fixed types may be possible to convert the base class to meta-classes in order to have them calculate and then share the _names lists (~ another 26 bytes/object + 4bytes/(name*object)) among all instances (preferably as a tuple in that case, of course)? Would need to study how these classes are used throughout the system to really guess what can be optimised. Need to do a hotspot profile of running code to know what to start working on. The initialisation code looks pretty involved in a number of cases... anyway, no time tonight, have to work on application-level stuff. Have fun all, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Ilya E. <il...@gl...> - 2004-02-11 08:09:24
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> > > but if anyone is interested in bringing a full high performance ASN > > coder-decoder to twisted, i'd be willing to help. > > Ilya (creator of PySNMP) just released a new version of PySNMP > yesterday. It's noticeably faster feeling than the 3.3.x version. I > don't have time to do timings with it, but I'd guess it's probably > taking 3/4 of the time of the previous version. Still likely not 0.02 > seconds on getbulk requests, but noticeably faster. Just a little bit of clarification: My ASN1 decoder is designed in a way similar to a top-down text parser -- it reads octet stream (text) and builds a tree of ASN1 objects (AST). Depending of the mix-ins to those ASN1 objects, walking the tree may produce something concrete. Perhaps the most noticable bottleneck there lays in the fact that building a tree of ASN1 objects involves massive classes instantiations. In the latest pysnmp code (3.4.x) I attempted to save on object creation by caching and reusing once created ASN1 objects. To make the most of this trick I'd suggest to cache and re-use ASN1 objects (SNMP message objects for example) at your application whenever possible. -ilya |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-11 01:03:06
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Discovered that the bug discovered in 0.2.3 really was on the manager side (was altering the wrong list of oids when dealing with error responses), releasing new version now. http://members.rogers.com/mcfletch/programming/index.htm#TwistedSNMP Have fun, all, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Nicola L. <ni...@te...> - 2004-02-10 11:08:34
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Ilya (creator of PySNMP) just released a new version of PySNMP > yesterday. As a data point, TwistedSNMP just appearead on Daily Python-URL: Two Python SNMP packages updated http://www.pycs.net/users/0000231/weblog/2004/02/05.html - -- Nicola Larosa - nic...@m-... "Hope" is the thing with feathers - That perches in the soul - And sings the tune without the words - And never stops - at all - -- Emily Dickinson -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFAKLuGXv0hgDImBm4RAmfeAKCmLWv313qygJ2GLj5vqkPe/7UnxACdGt8Q fzPd1AEB5PE6HvGVZhsF50E= =S9cz -----END PGP SIGNATURE----- -- Nicola Larosa - ni...@te... "Hope" is the thing with feathers - That perches in the soul - And sings the tune without the words - And never stops - at all - -- Emily Dickinson |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-10 09:26:56
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Thomas Favier wrote: > Mike C. Fletcher wrote: > >> If for no other reason than that it convinced me there wasn't a >> simple and elegant solution readily available to be dropped in for a >> UDP system :) . In the final analysis, what was happening is that >> individual queries were timing out *before the queries were even >> sent* because it was taking so incredibly long to format and send all >> of the queries. > > > You bring my point with SNMP : I've tried to build a SNMP poller based > on twisted and pySNMP. Main trouble i've had is the decoding time of > the pure python ASN parser. It spent about 1 full second decoding > (packets are about 7ko, as i get all values and tables with a > GET-BULK, there is room for improvement by splitting requests, but > that's not significant for that matter). So i switched to a threaded > implementation using yapsnmp (and ugly deferToThreads). Decoding times > are about 0.02 seconds for the same requests. Load on the server was > more than halved. > > Seems that you are hit by the same kind of problems... ... > but if anyone is interested in bringing a full high performance ASN > coder-decoder to twisted, i'd be willing to help. Ilya (creator of PySNMP) just released a new version of PySNMP yesterday. It's noticeably faster feeling than the 3.3.x version. I don't have time to do timings with it, but I'd guess it's probably taking 3/4 of the time of the previous version. Still likely not 0.02 seconds on getbulk requests, but noticeably faster. Profiling the ASN parser and writing a small accelerator module would likely be quite doable. From what I've seen there's not a lot of complex code in there. It's just hidden deep in the package structure. Nice thing about that approach is that it's possible to port to new platforms and only worry about the C module being present if you need the speed there. My primary interest here isn't really in speed, we query 8000 agents at a time, and only need to query once every few *hours*. The flexibility & robustness of the code is my primary interest. Too much C gets me worried :) , particularly with networking code, where buffer overflows and the like are a pain to avoid. Have fun, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-10 09:04:17
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> This release's primary reason for existence is to fix the mass retriever algorithm, particularly when used with large numbers of agents (hundreds or thousands). The new approach is much simpler and less error-prone than the old approach. In live tests, I'm still seeing strange incorrectly-formatted responses from some modems, haven't yet tracked down whether that's a problem with TwistedSNMP or not. I'm assuming it's either the port's buffer getting full (thus returning a message-fragment instead of a full message (this seems most likely)), something exotic within PySNMP (seems doubtful), or an actual problem with the modem returning a messed up response message. I also discovered a bug that affects all versions of TwistedSNMP, hopefully I'll have some time to work on fixing that this weekend (we don't actually use the functionality in our production systems (it's a problem with the Agent side, we only do manager-side programming), so can't do it on company time). BTW, Ilya was definitely right about PySNMP 3.4.x being much faster than 3.3.x. Enjoy yourselves, Mike Version 0.2.3 * Rework of mass retriever algorithm (yet again) to be more robust when dealing with larger AgentProxy sets o Much simpler mechanism o Throttles to initiate less than 200 queries/second by default, but ignores the number in play, which, as it turns out, seems to be somewhat useless for managing load. This is not ideal, as, at least on our testing systems, the hardware can readily handle a few thousand queries/second (it's Twisted's structure that requires the throttling). * Internal rework so that timeouts are cancelled ASAP for messages received o To prevent timeouts issuing before the system gets around to calling the callback * Discovered (as-yet-unfixed) bug in Agent's (as distinct from AgentProxy) processing of table requests o Basically it doesn't properly handle tables of different lengths. _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Ilya E. <il...@gl...> - 2004-02-09 11:07:17
|
Ah, right, I've missed the support for None -> Null() value translation as existed at "generic" API. I've just fixed this and your tests seem to pass alright now. The patch is attached, CVS version (version-3-3-x-branch) is up to date. Thanks, ilya On Sat, 7 Feb 2004, Mike C. Fletcher wrote: > A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> > List-Post: <mailto:pys...@li...> > List-Help: <mailto:pys...@li...?subject=help> > List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, > <mailto:pys...@li...?subject=subscribe> > List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> > Date: Sat, 07 Feb 2004 15:21:12 -0500 > > Ilya Etingof wrote: > > >I've just released the next version of PySNMP package (3.4.1) along with a > >newcomer -- PySNMPApps package holds basic SNMP tools written on top of > >PySNMP. > > > > > I've just been working at porting TwistedSNMP over to FreeBSD (I've only > ever run it on Windows before), and in so doing I tried out PySNMP 3.4.1 > versus 3.3.5, there's something going on such that previously functional > code is no longer functional under 3.4.1. Tracebacks follow: > > (There's lots of these, btw. Found by running TwistedSNMP 0.2.2 test > suite against PySNMP 3.4.1) > > Enjoy all, > Mike > > ERROR: After a replace-set, is the set value retrieved? > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/mcfletch/cvstw/twistedsnmp/test/test_set.py", line 32, in > test_setReplaceAnOID > d = self.client.get( > File "/home/mcfletch/cvstw/twistedsnmp/agentproxy.py", line 51, in get > request = self.encode(oids, self.community) > File "/home/mcfletch/cvstw/twistedsnmp/agentproxy.py", line 146, in encode > pdu.apiGenSetVarBind(variables) > File "/home/mcfletch/cvstw/pysnmp/proto/api/generic/rfc1157.py", line > 25, in apiGenSetVarBind > apply(self.apiAlphaSetVarBindList, varBinds) > File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1157.py", line > 27, in apiAlphaSetVarBindList > varBindList[idx].apiAlphaSetOidVal(varBind) > File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1157.py", line > 10, in apiAlphaSetOidVal > self['value'].apiAlphaSetTerminalValue(val) > File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1155.py", line > 51, in apiAlphaSetTerminalValue > raise error.BadArgumentError('Unapplicable component %s at %s'\ > BadArgumentError: Unapplicable component None at ObjectSyntax > > ERROR: After a set, is the set value retrieved? > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/mcfletch/cvstw/twistedsnmp/test/test_set.py", line 19, in > test_setEndOfOIDs > d = self.client.get( > File "/home/mcfletch/cvstw/twistedsnmp/agentproxy.py", line 51, in get > request = self.encode(oids, self.community) > File "/home/mcfletch/cvstw/twistedsnmp/agentproxy.py", line 146, in encode > pdu.apiGenSetVarBind(variables) > File "/home/mcfletch/cvstw/pysnmp/proto/api/generic/rfc1157.py", line > 25, in apiGenSetVarBind > apply(self.apiAlphaSetVarBindList, varBinds) > File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1905.py", line > 24, in apiAlphaSetVarBindList > varBindList[idx].apiAlphaSetOidVal(varBind) > File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1157.py", line > 10, in apiAlphaSetOidVal > self['value'].apiAlphaSetTerminalValue(val) > File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1155.py", line > 51, in apiAlphaSetTerminalValue > raise error.BadArgumentError('Unapplicable component %s at %s'\ > BadArgumentError: Unapplicable component None at BindValue > > _______________________________________ > Mike C. Fletcher > Designer, VR Plumber, Coder > http://members.rogers.com/mcfletch/ > > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > pysnmp-users mailing list > pys...@li... > https://lists.sourceforge.net/lists/listinfo/pysnmp-users > |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-07 20:23:38
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Minor release, mostly just bug-fixes for operation in (more restrictive) Unix environments. http://members.rogers.com/mcfletch/programming/index.htm#TwistedSNMP Have fun all, Mike Version 0.2.2 * Bug-fix in utilities/mirroragent.py * Test module cleanup for cross-platform operation o Allow for operation when no bsddb module is available, as there are platforms on which it's not available by default o Use non-privileged port if the privileged 161 is not bindable * Added check to twineTables for NULL OID set (passing in such an OID set makes no sense, and would cause silent, difficult-to-track-down errors) _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-07 20:21:19
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Ilya Etingof wrote: >I've just released the next version of PySNMP package (3.4.1) along with a >newcomer -- PySNMPApps package holds basic SNMP tools written on top of >PySNMP. > > I've just been working at porting TwistedSNMP over to FreeBSD (I've only ever run it on Windows before), and in so doing I tried out PySNMP 3.4.1 versus 3.3.5, there's something going on such that previously functional code is no longer functional under 3.4.1. Tracebacks follow: (There's lots of these, btw. Found by running TwistedSNMP 0.2.2 test suite against PySNMP 3.4.1) Enjoy all, Mike ERROR: After a replace-set, is the set value retrieved? ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mcfletch/cvstw/twistedsnmp/test/test_set.py", line 32, in test_setReplaceAnOID d = self.client.get( File "/home/mcfletch/cvstw/twistedsnmp/agentproxy.py", line 51, in get request = self.encode(oids, self.community) File "/home/mcfletch/cvstw/twistedsnmp/agentproxy.py", line 146, in encode pdu.apiGenSetVarBind(variables) File "/home/mcfletch/cvstw/pysnmp/proto/api/generic/rfc1157.py", line 25, in apiGenSetVarBind apply(self.apiAlphaSetVarBindList, varBinds) File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1157.py", line 27, in apiAlphaSetVarBindList varBindList[idx].apiAlphaSetOidVal(varBind) File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1157.py", line 10, in apiAlphaSetOidVal self['value'].apiAlphaSetTerminalValue(val) File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1155.py", line 51, in apiAlphaSetTerminalValue raise error.BadArgumentError('Unapplicable component %s at %s'\ BadArgumentError: Unapplicable component None at ObjectSyntax ERROR: After a set, is the set value retrieved? ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/mcfletch/cvstw/twistedsnmp/test/test_set.py", line 19, in test_setEndOfOIDs d = self.client.get( File "/home/mcfletch/cvstw/twistedsnmp/agentproxy.py", line 51, in get request = self.encode(oids, self.community) File "/home/mcfletch/cvstw/twistedsnmp/agentproxy.py", line 146, in encode pdu.apiGenSetVarBind(variables) File "/home/mcfletch/cvstw/pysnmp/proto/api/generic/rfc1157.py", line 25, in apiGenSetVarBind apply(self.apiAlphaSetVarBindList, varBinds) File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1905.py", line 24, in apiAlphaSetVarBindList varBindList[idx].apiAlphaSetOidVal(varBind) File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1157.py", line 10, in apiAlphaSetOidVal self['value'].apiAlphaSetTerminalValue(val) File "/home/mcfletch/cvstw/pysnmp/proto/api/alpha/rfc1155.py", line 51, in apiAlphaSetTerminalValue raise error.BadArgumentError('Unapplicable component %s at %s'\ BadArgumentError: Unapplicable component None at BindValue _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-06 22:16:07
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> The tarfile uploaded yesterday was missing some files from the distribution. New (tested) tarfile uploaded now: http://members.rogers.com/mcfletch/programming/ Sorry about that all, and thanks to John Speno for pointing out the problem, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
|
From: Ilya E. <il...@gl...> - 2004-02-06 16:19:52
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> Updated documentation and examples have been committed to http://pysnmp.sourceforge.net/ . If anyone has an established project that uses PySNMP and don't mind me putting a link to it, please, let me know. Thanks, ilya |
|
From: Mike C. F. <mcf...@ro...> - 2004-02-06 08:41:49
|
A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net> List-Post: <mailto:pys...@li...> List-Help: <mailto:pys...@li...?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>, <mailto:pys...@li...?subject=subscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users> This is a minor release bump, mostly to get the cleaned up API for mass retrieval out in the wild before anyone starts writing code against the old API. There's also a bug-fix in the table-retriever class. http://members.rogers.com/mcfletch/programming/ Have fun all, Mike Version 0.2.1 * Cleaned up and reworked mass-query mechanism; o patterned loosely after the table-query mechanism o fixed at least one logic bug in the batching machinery o added adaptive heuristics for determining number of queries to keep in play o changes the api for retrieval in a backward-incompatible way * Fixed shallow bug in TableRetriever's handling of responses with unequal rootOID and (OID,value) set lengths _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ |
25 messages has been excluded from this view by a project administrator.
