[pysnmp-users] Speed suckers...

[Mike C. F. <mcf...@ro...>]

A list for users of pure-Python SNMP framework <pysnmp-users.lists.sourceforge.net>
List-Post: <mailto:pys...@li...>
List-Help: <mailto:pys...@li...?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/pysnmp-users>,
	<mailto:pys...@li...?subject=subscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum=pysnmp-users>

Okay, finding that speed is actually turning into an issue, I've spent a 
few minutes profiling the test suite for TwistedSNMP.  There are a few 
methods in particular that seem to really slow down the system:

asn1\base.py:83 	_subtype_constraint 	7.9% total time 	10.5% cumulative
asn1\base.py:240 	set
	5.7% 	19.0%
asn1\base.py:479 	__setitem__ 	3.1% 	5.4%
asn1\encoding\ber\base.py:241
	berEncode
	3.0%
	9.7%
asn1\base.py:209 	_setRawAsn1Value
	2.9%
	9.0%


all in all, message decoding is taking 17.7% of total run-time, message 
encoding is taking 11.5% of total.

Looking at _subtype_constraint, there's an unnecessary call (and lookup) 
for each len() call, and an unnecessarily iteration through a list for 
each of singleValueConstraint, permittedAlphabetConstraint, which, 
compared to the C-implemented dictionary lookup is likely going to be 
pretty slow.  Eliminating just the len() calls drops the totals to 7.1% 
and 9.7%.

        if self.singleValueConstraint:
            if self._subtype_single_value_constraint is None:
                raise error.ValueConstraintError('Single value 
constraint not applicible to %s' % self.__class__.__name__)
            if value not in self.singleValueConstraint:
                raise error.ValueConstraintError('Single value 
constraint for %s: %s not within allowed values' % 
(self.__class__.__name__, str(value)))

That second if check would seem to be something that could be done once 
at class initialisation time, rather than continually checking in the 
very tight loop (this method is called 157,000 some odd times just in 
the (rather small) test suite for TwistedSNMP).

Haven't gone through to try dictionary lookups for static-value sets.  
That's only going to trim a few percent of overhead at maximum (small 
sets) I would guess.

Anyway, no more time today. Enjoy all,
Mike

_______________________________________
  Mike C. Fletcher
  Designer, VR Plumber, Coder
  http://members.rogers.com/mcfletch/