Re: [pysnmp-dev] Regarding use of Dispatcher with Multiprocessing

[Ilya E. <il...@gl...>]

Hi Ani,

There’s no easy way to access *parts* of serialised message. May be you could just decode the whole message (like you did), peek into varbinds to select appropriate queue and then send the whole *original* SNMP TRAP/INFORM message to a new destination (e.g. wholeMsg)?

An alternative is to re-encode reqPDU to get serialised data just for it:

    from pyasn1.codec.ber import encoder

    rawData = encoder.encode(reqPDU)

But that is less efficient.

Let me know if this is not what you want.

-ilya


> On 02 Mar 2015, at 15:13, Ani A <ani...@gm...> wrote:
> 
> Thanks Ilya,
> 
> I looked at `snmpfwd`, I think , at the moment, I don't need the full
> functionality, but rather just a small subset, for
> the trap processing.
> 
> And, one more thing I feel is, in the notification forwarder, I just
> need to peek into the varbinds, and then if I
> can get just the PDU (as _data_ rather than Object), I can
> re-encapsulate it in a different UDP packet, and send
> But I don't this there is any API to get the PDU as _raw-data_ - is
> that correct ?
> 
> Please see the attached code, that I have so far (I have added
> comments on what I am trying to accomplish).
> 
> --
> Regards,
> A. Aniruddha
> 
> 
> On Sun, Mar 1, 2015 at 1:44 AM, Ilya Etingof <il...@gl...> wrote:
>> Hi Ani,
>> 
>> There’s similar app snmpfwd [1], however it does yet not support SNMP
>> notifications at the moment.
>> 
>> The most straightforward way to do a proxy with pysnmp is to run both sides
>> — Notification Receiver & Notification Originator together within a single
>> main loop. Since pysnmp natively supports asynchronous communication, while
>> one request is waiting for I/O, the other could be served concurrently. But
>> two strictly simultaneous requests would queue up at shared SNMP engine
>> waiting for its services. Here’s an example script [2] that implements SNMP
>> Commands forwarding in the above design. It could be modified to support
>> Notifications. With pysnmp API all the low-level processing and ACKs could
>> be done automatically so you could deal with PDUs and targets.
>> 
>> If you wish to use threads, you'd have to run many independent SNMP engines
>> (and main loops) local to each thread. Take a look at this example [3] to
>> get an idea. In case of notifications I imagine you would have the receiving
>> parts sitting each in its own main loop waiting for incoming queries and
>> pushing received requests to a queue. The sending parts would  run their
>> main loops briefly, just to push out request and optionally wait for
>> response (informs). So most of the time the sending parts would spend
>> waiting for queue to pull request from.
>> 
>> From my perspective, the threading approach may give better load
>> distribution (but unlikely better throughput considering GIL which does not
>> let you run threads truly simultaneously), however it may require more work
>> from your side and larger memory footprint due to SNMP engine duplication.
>> 
>> The third option could be to run multiple instances of your purely
>> asynchronous proxy (receivers) to improve concurrency.
>> 
>> -ilya
>> 
>> [1] http://sf.net/projects/snmpfwd/
>> [2]
>> http://pysnmp.sourceforge.net/examples/current/v3arch/proxy/cmd/v3-to-v2c-conversion.html
>> [3]
>> http://pysnmp.sourceforge.net/examples/current/v3arch/oneliner/manager/cmdgen/get-threaded-multiple-transports-and-protocols.html
>> 
>> 
>> On 27 Feb 2015, at 08:01, Ani A <ani...@gm...> wrote:
>> 
>> Hi Ilya,
>> 
>> I am trying to create something like a _proxy_, the goal is to be able to
>> intercept traps from the local
>> SNMP Agent, and then peeking into the varbinds, decide which one of the
>> remote trap-receivers to send to.
>> 
>> Traps are the first goal, then, for Informs, I need to be able to get the
>> ack, and send it back to the main agent as well.
>> 
>> Since I am using Multiprocessing module [1], do I still have to worry about
>> MT safe?
>> 
>> (the networking part of this, is that the receivers are all on different
>> networks
>> [ different Linux namespaces ], so I have to get the SNMP payload and then
>> build a separate UDP packet and
>> send, unless there is an easier way/API that I can use to just change the IP
>> and send)
>> 
>> Hope I explained the problem [that I am trying to solve], clearly. Any
>> help/tips/guidance very much welcome.
>> 
>> 
>> [1] https://docs.python.org/2/library/multiprocessing.html
>> 
>> --
>> Ani
>> 
>> 
>> On Fri, Feb 27, 2015 at 1:09 AM, Ilya Etingof <il...@gl...> wrote:
>>> 
>>> Hi Ani,
>>> 
>>> At what point you are going to employ threading? Just for SNMP receiving
>>> part (what is not easy since a single instance of SNMP engine is not
>>> MT-safe) or for some sort of custom processing logic beyond SNMP engine?
>>> 
>>> -ilya
>>> 
>>>> On 24 Feb 2015, at 09:44, Ani A <ani...@gm...> wrote:
>>>> 
>>>> Hello,
>>>> 
>>>> I was running through the examples in pysnmp (for Trap receiver with V3
>>>> case)
>>>> and I have a few doubts (maybe trivial! but somehow I am not getting it)
>>>> 
>>>> I want to use pysnmp, with Python's Multiprocessing module, so in
>>>> each Process's callback function, do I run the following ?
>>>> 
>>>>   try:
>>>>       snmpEngine.transportDispatcher.runDispatcher()
>>>>   except:
>>>>       snmpEngine.transportDispatcher.closeDispatcher()
>>>>       raise
>>>> 
>>>> i.e, what should my main-loop wait on? , since I have another Queue as
>>>> well
>>>> which I need to check.
>>>> 
>>>> Thanks.
>>>> —
>>>> Ani
>>> 
>> 
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming The Go Parallel Website,
>> sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub for
>> all
>> things parallel software development, from weekly thought leadership blogs
>> to
>> news, videos, case studies, tutorials and more. Take a look and join the
>> conversation now.
>> http://goparallel.sourceforge.net/_______________________________________________
>> pysnmp-dev mailing list
>> pys...@li...
>> https://lists.sourceforge.net/lists/listinfo/pysnmp-dev
>> 
>> 
> <snmpp.py>