net-snmp-coders — Discussion of internal project coding (Please use this OR -users, NOT both!)

Re: Core dump with net-snmp-5.8

[Anders W. <wal...@gm...>]

Hi Sam,

I'm not a maintainer for net-snmp, but it was merged into 5-8-patches
yesterday

Regards
Anders Wallin


On Wed, Apr 10, 2019 at 8:04 PM Sam Tannous <sta...@cu...>
wrote:

> Hi Anders,
>
> We've been testing the patch for
>
> https://sourceforge.net/p/net-snmp/bugs/2914/
>
> and this works.
>
> Before the patch, we were getting lots of core dumps where the flow in
> master.c hits the default case and memory gets double-freed and a core
> happens.
> We'd love to get this into V5-8-patches soon.
>
> Thanks,
> Sam
>
>
>
> On Tue, Apr 9, 2019 at 9:13 AM Sam Tannous <sta...@cu...>
> wrote:
>
>> Hi Anders,
>>
>> I fixed some snmpv3 (bulkget) coredumps a while ago.
>> https://sourceforge.net/p/net-snmp/patches/1388/
>>
>> While not directly related, the (double-free memory) core dumps
>> were easily triggered by any error condition within a v3 bulkget.
>>
>> I'm hoping my patch will get picked up soon :-(
>>
>> Thanks,
>> Sam
>>
>> On Tue, Apr 9, 2019 at 6:54 AM Anders Wallin <wal...@gm...> wrote:
>>
>>> Now it works fine!
>>>
>>> thx
>>> Anders Wallin
>>>
>>>
>>> On Tue, Apr 9, 2019 at 2:26 AM Masayoshi Mizuma <msy...@gm...>
>>> wrote:
>>>
>>>> Hi Anders,
>>>>
>>>> Thank you for your feedback!
>>>> I attach the v2 patch. Could you try it?
>>>>
>>>> On the v1 patch, I missed the check for the request callback. So, the
>>>> request
>>>> gets removed even though the callback doesn't run.
>>>>
>>>> Thanks,
>>>> Masa
>>>>
>>>> On 4/8/19 11:06 AM, Anders Wallin wrote:
>>>> > Hi Masa,
>>>> >
>>>> > looks like it solves the problem reported by Josef, BUT it breaks
>>>> DTLSUDP.
>>>> > I run the tests w/o analyzing why.
>>>> > To reproduce the issue I did the following using net-snmp master
>>>> branch,
>>>> > plus these patches
>>>> > 39485c6f2 - snmplib/snmp_api: Remove the request on the session when
>>>> the
>>>> > sending is failed (10 minutes ago) <Masayoshi Mizuma>
>>>> > 06a4d52d8 - agentx: logging to late responses (5 days ago) <Anders
>>>> Wallin>
>>>> > a420d87d3 - BUG2914: Agent master needs to treat resend as normal (5
>>>> days
>>>> > ago) <Anders Wallin>
>>>> > eaad09d04 - (origin/master, origin/HEAD, master) Merge branch
>>>> > 'V5-8-patches' (9 weeks ago) <Bart Van Assche>
>>>> >
>>>> > $ ./configure --prefix=/usr \
>>>> >                 --with-persistent-directory=/var/lib/net-snmp \
>>>> >                 --with-mib-modules='smux tlstm-mib tsm-mib
>>>> examples/example
>>>> > examples/notification' \
>>>> >                 --with-security-modules="tsm" \
>>>> >                 --with-transports="TLSTCP DTLSUDP" \
>>>> >                 --enable-shared \
>>>> >                 --with-defaults \
>>>> >                 --enable-ipv6 \
>>>> >                 --with-cflags="-g -O2" \
>>>> >                 --without-elf
>>>> >
>>>> > $ make install
>>>> > $ cd testing
>>>> > $ ./RUNFULLTESTS -g tls
>>>> > DTLS-UDP user certificate tests .......................... 41/?
>>>> >  This hangs forever in "41" with snmpd.log saying....
>>>> > ......
>>>> > 2019-04-08 16:29:11
>>>> > 2019-04-08 16:29:11
>>>> > Received 0 byte packet from DTLSUDP: unknown
>>>> > 2019-04-08 16:29:11
>>>> > 2019-04-08 16:29:13
>>>> > Received 0 byte packet from DTLSUDP: unknown
>>>> > 2019-04-08 16:29:13
>>>> > 2019-04-08 16:29:15
>>>> > Received 0 byte packet from DTLSUDP: unknown
>>>> > 2019-04-08 16:29:15
>>>> > 2019-04-08 16:29:15 tls verification failure: ok=0 ctx=0x55ee625b4170
>>>> > depth=0 err=18:self signed certificate
>>>> > 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
>>>> > 2019-04-08 16:29:15  TLS error: SSL_read: rc=-1, sslerror = 1
>>>> > (SSL_ERROR_SSL)
>>>> > 2019-04-08 16:29:15  TLS Error: certificate verify failed
>>>> > 2019-04-08 16:29:15 ---- End of OpenSSL Errors ----
>>>> > 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
>>>> > 2019-04-08 16:29:15 TLS error: SSL_read: rc=-1, sslerror = 5
>>>> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
>>>> > 2019-04-08 16:29:15 TLS Error: (null)
>>>> > 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
>>>> > 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
>>>> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
>>>> > 2019-04-08 16:29:16 TLS Error: (null)
>>>> > 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
>>>> > 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
>>>> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
>>>> > 2019-04-08 16:29:16 TLS Error: (null)
>>>> >
>>>> > With the fix suggested på Josef I don't see the DTLSUDP problem, but
>>>> maybe
>>>> > there are other problems.
>>>> >
>>>> > Regards
>>>> > Anders Wallin
>>>> >
>>>> > PS. thx for adding commit info to a420d87d3, I updated the patch with
>>>> your
>>>> > commit comments
>>>> >
>>>> >
>>>> > On Mon, Apr 8, 2019 at 3:27 PM Masayoshi Mizuma <
>>>> msy...@gm...>
>>>> > wrote:
>>>> >
>>>> >> Hi Josef,
>>>> >>
>>>> >> I attach two patches to fix the memory inconsistency if the request
>>>> is
>>>> >> resend and timed out.
>>>> >> Could you try them?
>>>> >>
>>>> >> - 0001-agentx-master-Return-when-NETSNMP_CALLBACK_OP_RESEND.patch
>>>> >>
>>>> >>   This patch was posted by Anders, and I tried to add the
>>>> description.
>>>> >>   This patch fixes the missing NETSNMP_CALLBACK_OP_RESEND callback.
>>>> >>
>>>> >> - 0002-snmplib-snmp_api-Remove-the-request-on-the-session-w.patch
>>>> >>
>>>> >>   This patch fixes the race between NETSNMP_CALLBACK_OP_SEND_FAILED
>>>> >>   and NETSNMP_CALLBACK_OP_TIMED_OUT callback. If the request is
>>>> failed,
>>>> >>   then remove the request from the internal session.
>>>> >>
>>>> >> Thanks,
>>>> >> Masa
>>>> >>
>>>> >> On 4/3/19 9:34 AM, Anders Wallin wrote:
>>>> >>> The introduction of that code fixes another issue;
>>>> >>> "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
>>>> >>> Author: Bill Fenner <fe...@gm...>
>>>> >>> Date:   Wed Dec 20 21:52:10 2017 +0000
>>>> >>>
>>>> >>>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad
>>>> SNMPv3
>>>> >>> agent
>>>> >>>
>>>> >>>     With the patch in 1214, the snmp_api code assumed that if magic
>>>> was
>>>> >>>     set, it was the "struct synch-state" from snmp_client.  Of
>>>> course,
>>>> >>>     magic belongs to the caller, and the perl library uses it
>>>> >> differently,
>>>> >>>     so reaching into it is verboten.  Introduce a new callback (that
>>>> >>>     was already introduced in 5.8) to report this "retries exceeded"
>>>> >>>     state, and use it in snmp_client."
>>>> >>>
>>>> >>> I think the problem is really about shutting down the agentx
>>>> connection
>>>> >>> when one(1) response is to late. I have
>>>> >>> done 2 patches (one that only write a better log message and one
>>>> that
>>>> >>> removes the "bad" code.
>>>> >>> With these patches I don't get any crash. I think that 5.7.3 has
>>>> this
>>>> >> issue
>>>> >>> as well, but it can not be crashed with the agentofdead code
>>>> >>>
>>>> >>> Can you please try this?
>>>> >>>
>>>> >>> Regards
>>>> >>> Anders Wallin
>>>> >>>
>>>> >>>
>>>> >>> On Wed, Apr 3, 2019 at 12:35 PM Josef Ridky <jr...@re...>
>>>> wrote:
>>>> >>>
>>>> >>>> Hi,
>>>> >>>>
>>>> >>>> I have compared net-snmp-5.7.3 and net-snmp-5.8 and I have found,
>>>> that
>>>> >>>> following callbacks in snmplib/snmp_api.c causes the core dump
>>>> issue:
>>>> >>>>
>>>> >>>> --- old/snmplib/snmp_api.c      2019-04-03 12:13:55.126769866 +0200
>>>> >>>> +++ new/snmplib/snmp_api.c      2019-04-03 12:15:18.353420790 +0200
>>>> >>>> @@ -6731,9 +6731,9 @@ snmp_resend_request(struct session_list
>>>> >>>>          sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
>>>> >>>>          sp->s_errno = errno;
>>>> >>>>          snmp_set_detail(strerror(errno));
>>>> >>>> -        if (rp->callback)
>>>> >>>> +/*        if (rp->callback)
>>>> >>>>              rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
>>>> >>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>>>> >>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>>>> >>>>          return -1;
>>>> >>>>      } else {
>>>> >>>>          netsnmp_get_monotonic_clock(&now);
>>>> >>>> @@ -6743,9 +6743,9 @@ snmp_resend_request(struct session_list
>>>> >>>>          tv.tv_sec += tv.tv_usec / 1000000L;
>>>> >>>>          tv.tv_usec %= 1000000L;
>>>> >>>>          rp->expireM = tv;
>>>> >>>> -        if (rp->callback)
>>>> >>>> +/*        if (rp->callback)
>>>> >>>>              rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
>>>> >>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>>>> >>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>>>> >>>>      }
>>>> >>>>      return 0;
>>>> >>>>  }
>>>> >>>>
>>>> >>>> Without them, all works as expected.
>>>> >>>>
>>>> >>>> Josef Ridky
>>>> >>>> Software Engineer
>>>> >>>> Core Services Team
>>>> >>>> Red Hat Czech, s.r.o.
>>>> >>>>
>>>> >>>> ----- Original Message -----
>>>> >>>> | From: "Anders Wallin" <wal...@gm...>
>>>> >>>> | To: "Josef Ridky" <jr...@re...>
>>>> >>>> | Cc: "net-snmp-coders" <net...@li...>
>>>> >>>> | Sent: Tuesday, April 2, 2019 6:27:54 PM
>>>> >>>> | Subject: Re: Core dump with net-snmp-5.8
>>>> >>>> |
>>>> >>>> | Hi Josef,
>>>> >>>> | I can reproduce the issue using the master branch, I will take a
>>>> look
>>>> >> at
>>>> >>>> it
>>>> >>>> | later tonight or tomorrow
>>>> >>>> |
>>>> >>>> | Regards
>>>> >>>> | Anders Wallin
>>>> >>>> |
>>>> >>>> |
>>>> >>>> | On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...>
>>>> wrote:
>>>> >>>> |
>>>> >>>> | > Hi,
>>>> >>>> | >
>>>> >>>> | > thanks for your patch. Unfortunately, even when I have applied
>>>> it,
>>>> >> it
>>>> >>>> | > still ends with core dump due of 'double free or corruption
>>>> >> (fasttop)'
>>>> >>>> | >
>>>> >>>> | > When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
>>>> >>>> | >
>>>> >>>> | > agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
>>>> >>>> | > snmp_agent: delegate session == 0x56207e165240
>>>> >>>> | > snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
>>>> >>>> | > agentx/master: callback resend
>>>> >>>> | > agentx/master: callback resend
>>>> >>>> | > agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
>>>> >>>> | > agentx/master: close 0x56207dfd5400, -1
>>>> >>>> | > snmp_agent: removed 40 delegated request(s) for session
>>>> >> 0x56207dfce490
>>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e165240
>>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e165240
>>>> >>>> | > snmp_agent: REMOVE session == 0x56207e165240
>>>> >>>> | > snmp_agent: agent_session 0x56207e165240 released
>>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e1041a0
>>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e1041a0
>>>> >>>> | > snmp_agent: REMOVE session == 0x56207e1041a0
>>>> >>>> | > snmp_agent: agent_session 0x56207e1041a0 released
>>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e1656c0
>>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e1656c0
>>>> >>>> | > snmp_agent: REMOVE session == 0x56207e1656c0
>>>> >>>> | > snmp_agent: agent_session 0x56207e1656c0 released
>>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11af40
>>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11af40
>>>> >>>> | > snmp_agent: REMOVE session == 0x56207e11af40
>>>> >>>> | > snmp_agent: agent_session 0x56207e11af40 released
>>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e118f00
>>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e118f00
>>>> >>>> | > snmp_agent: REMOVE session == 0x56207e118f00
>>>> >>>> | > snmp_agent: agent_session 0x56207e118f00 released
>>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11b540
>>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11b540
>>>> >>>> | > snmp_agent: REMOVE session == 0x56207e11b540
>>>> >>>> | > snmp_agent: agent_session 0x56207e11b540 released
>>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11bd00
>>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11bd00
>>>> >>>> | > snmp_agent: REMOVE session == 0x56207e11bd00
>>>> >>>> | > snmp_agent: agent_session 0x56207e11bd00 released
>>>> >>>> | > agentx/master: Continue removing delegated subsession reqests
>>>> >>>> | > agentx/master: close transport
>>>> >>>> | > snmp_agent: REMOVE session == 0x56207dfd5400
>>>> >>>> | > agentx/master: response too late on session 0x56207dfd5400
>>>> >>>> | > agentx/master: response too late on session 0x56207dfd5400
>>>> >>>> | > double free or corruption (fasttop)
>>>> >>>> | > Aborted (core dumped)
>>>> >>>> | >
>>>> >>>> | >
>>>> >>>> | > What's interesting, when I run it with -DALL it pass (at least
>>>> for
>>>> >>>> several
>>>> >>>> | > rounds).
>>>> >>>> | > It looks like some strange race condition.
>>>> >>>> | >
>>>> >>>> | > Regards
>>>> >>>> | >
>>>> >>>> | > Josef Ridky
>>>> >>>> | > Software Engineer
>>>> >>>> | > Core Services Team
>>>> >>>> | > Red Hat Czech, s.r.o.
>>>> >>>> | >
>>>> >>>> | > ----- Original Message -----
>>>> >>>> | > | From: "Anders Wallin" <wal...@gm...>
>>>> >>>> | > | To: "Josef Ridky" <jr...@re...>
>>>> >>>> | > | Cc: "net-snmp-coders" <net...@li...
>>>> >
>>>> >>>> | > | Sent: Tuesday, April 2, 2019 1:46:40 PM
>>>> >>>> | > | Subject: Re: Core dump with net-snmp-5.8
>>>> >>>> | > |
>>>> >>>> | > | Hi Josef,
>>>> >>>> | > |
>>>> >>>> | > | I think it's the same issue as
>>>> >>>> | > https://sourceforge.net/p/net-snmp/bugs/2914/
>>>> >>>> | > | (where I also posted the solution)
>>>> >>>> | > | Regards
>>>> >>>> | > | Anders Wallin
>>>> >>>> | > |
>>>> >>>> | > |
>>>> >>>> | > | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <
>>>> jr...@re...>
>>>> >>>> wrote:
>>>> >>>> | > |
>>>> >>>> | > | > Hi,
>>>> >>>> | > | >
>>>> >>>> | > | > recently, I have hit to an issue in net-snmp-5.8, that is
>>>> >>>> connected to
>>>> >>>> | > the
>>>> >>>> | > | > bug report [1].
>>>> >>>> | > | >
>>>> >>>> | > | > When I tried to run agentofdeath test from [1], snmpd
>>>> daemon
>>>> >> will
>>>> >>>> crash
>>>> >>>> | > | > with malloc(): smallbin double linked list corrupted or
>>>> double
>>>> >>>> free()
>>>> >>>> | > issue
>>>> >>>> | > | > and dumps core (see bellow).
>>>> >>>> | > | > From log file, I can identified one issue with "Unknown
>>>> >> operation".
>>>> >>>> | > | >
>>>> >>>> | > | > This issue is in the agentx_got_response function
>>>> >>>> | > | > (agent/mibgroup/agentx/master.c). There isn't implemented
>>>> action
>>>> >>>> for
>>>> >>>> | > | > NETSNMP_CALLBACK_OP_RESEND (defined in
>>>> >>>> | > | > include/net-snmp/library/snmp_api.h).
>>>> >>>> | > | > As result "Unknown operation 6 in agentx_got_response" is
>>>> shown
>>>> >> in
>>>> >>>> log
>>>> >>>> | > | > file.
>>>> >>>> | > | >
>>>> >>>> | > | > /var/log/messages
>>>> >>>> | > | > -------------------------------
>>>> >>>> | > | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation
>>>> 6 in
>>>> >>>> | > | > agentx_got_response
>>>> >>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation
>>>> 6 in
>>>> >>>> | > | > agentx_got_response
>>>> >>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin
>>>> >> double
>>>> >>>> | > linked
>>>> >>>> | > | > list corrupted
>>>> >>>> | > | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core
>>>> Dump
>>>> >>>> (PID
>>>> >>>> | > | > 13652/UID 0).
>>>> >>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main
>>>> >> process
>>>> >>>> | > exited,
>>>> >>>> | > | > code=dumped, status=6/ABRT
>>>> >>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service:
>>>> Failed with
>>>> >>>> result
>>>> >>>> | > | > 'core-dump'.
>>>> >>>> | > | > -------------------------------
>>>> >>>> | > | >
>>>> >>>> | > | > The "Unknown operation" callback is caused by newly added
>>>> piece
>>>> >> of
>>>> >>>> | > code in
>>>> >>>> | > | > snmplib/snmp_api.c:
>>>> >>>> | > | >
>>>> >>>> | > | >  static int
>>>> >>>> | > | >  snmp_resend_request(struct session_list *slp,
>>>> >> netsnmp_request_list
>>>> >>>> | > *rp,
>>>> >>>> | > | >  int incr_retries)
>>>> >>>> | > | >  {
>>>> >>>> | > | >
>>>> >>>> | > | > ...
>>>> >>>> | > | >
>>>> >>>> | > | >          tv.tv_sec += tv.tv_usec / 1000000L;
>>>> >>>> | > | >          tv.tv_usec %= 1000000L;
>>>> >>>> | > | >          rp->expireM = tv;
>>>> >>>> | > | > +        if (rp->callback)
>>>> >>>> | > | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
>>>> >>>> | > | > +                         rp->pdu->reqid, rp->pdu,
>>>> rp->cb_data);
>>>> >>>> | > | >      }
>>>> >>>> | > | >      return 0;
>>>> >>>> | > | >  }
>>>> >>>> | > | >
>>>> >>>> | > | >
>>>> >>>> | > | > When I tried to remove it, it just stop complaining about
>>>> >>>> operation 6,
>>>> >>>> | > but
>>>> >>>> | > | > the core dump is still present.
>>>> >>>> | > | >
>>>> >>>> | > | > May I ask you for help with this issue? Do you have any
>>>> idea,
>>>> >> what
>>>> >>>> | > causing
>>>> >>>> | > | > this issue in 5.8 and how to fix it?
>>>> >>>> | > | > I know, that Jan Safranek has fixed this for 5.7 by commit
>>>> [2],
>>>> >>>> but it
>>>> >>>> | > | > looks like something other has changed and this issue is
>>>> current
>>>> >>>> again.
>>>> >>>> | > | >
>>>> >>>> | > | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
>>>> >>>> | > | > [2]
>>>> >>>> | > | >
>>>> >>>> | >
>>>> >>>>
>>>> >>
>>>> https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
>>>> >>>> | > | >
>>>> >>>> | > | > Regards
>>>> >>>> | > | >
>>>> >>>> | > | > Josef Ridky
>>>> >>>> | > | > Software Engineer
>>>> >>>> | > | > Core Services Team
>>>> >>>> | > | > Red Hat Czech, s.r.o.
>>>> >>>> | > | >
>>>> >>>> | > | >
>>>> >>>> | > | >
>>>> >>>> | > | > _______________________________________________
>>>> >>>> | > | > Net-snmp-coders mailing list
>>>> >>>> | > | > Net...@li...
>>>> >>>> | > | >
>>>> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>>>> >>>> | > | >
>>>> >>>> | > |
>>>> >>>> | >
>>>> >>>> |
>>>> >>>>
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> _______________________________________________
>>>> >>> Net-snmp-coders mailing list
>>>> >>> Net...@li...
>>>> >>> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>>>> >>>
>>>> >>
>>>> >
>>>>
>>> _______________________________________________
>>> Net-snmp-coders mailing list
>>> Net...@li...
>>> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>>>
>>
>>
>> --
>> *Sam Tannous*
>> Engineering
>> Cumulus Networks®
>> +1 650 383 6700 x 1106
>> <http://www.cumulusnetworks,com>www.cumulusnetworks.com
>>
>> Evaluate Cumulus® Linux®
>> https://cumulusnetworks.com/product/secure/evaluate/
>>
>> Become a Partner
>> http://cumulusnetworks.com/partners/become-a-partner/
>>
>
>
> --
> *Sam Tannous*
> Engineering
> Cumulus Networks®
> +1 650 383 6700 x 1106
> <http://www.cumulusnetworks,com>www.cumulusnetworks.com
>
> Evaluate Cumulus® Linux®
> https://cumulusnetworks.com/product/secure/evaluate/
>
> Become a Partner
> http://cumulusnetworks.com/partners/become-a-partner/
>

Re: Core dump with net-snmp-5.8

[Josef R. <jr...@re...>]

Hi folks,

thanks for your solution, I have tested it internally and all works as expected.
I hope, this will be soon part of net-snmp-5.8.

Regards

Josef Ridky
Software Engineer
Core Services Team
Red Hat Czech, s.r.o.

----- Original Message -----
| From: "Anders Wallin" <wal...@gm...>
| To: "Masayoshi Mizuma" <msy...@gm...>
| Cc: "Josef Ridky" <jr...@re...>, "Net-SNMP Coders" <net...@li...>
| Sent: Tuesday, April 9, 2019 12:54:09 PM
| Subject: Re: Core dump with net-snmp-5.8
| 
| Now it works fine!
| 
| thx
| Anders Wallin
| 
| 
| On Tue, Apr 9, 2019 at 2:26 AM Masayoshi Mizuma <msy...@gm...>
| wrote:
| 
| > Hi Anders,
| >
| > Thank you for your feedback!
| > I attach the v2 patch. Could you try it?
| >
| > On the v1 patch, I missed the check for the request callback. So, the
| > request
| > gets removed even though the callback doesn't run.
| >
| > Thanks,
| > Masa
| >
| > On 4/8/19 11:06 AM, Anders Wallin wrote:
| > > Hi Masa,
| > >
| > > looks like it solves the problem reported by Josef, BUT it breaks
| > DTLSUDP.
| > > I run the tests w/o analyzing why.
| > > To reproduce the issue I did the following using net-snmp master branch,
| > > plus these patches
| > > 39485c6f2 - snmplib/snmp_api: Remove the request on the session when the
| > > sending is failed (10 minutes ago) <Masayoshi Mizuma>
| > > 06a4d52d8 - agentx: logging to late responses (5 days ago) <Anders
| > Wallin>
| > > a420d87d3 - BUG2914: Agent master needs to treat resend as normal (5 days
| > > ago) <Anders Wallin>
| > > eaad09d04 - (origin/master, origin/HEAD, master) Merge branch
| > > 'V5-8-patches' (9 weeks ago) <Bart Van Assche>
| > >
| > > $ ./configure --prefix=/usr \
| > >                 --with-persistent-directory=/var/lib/net-snmp \
| > >                 --with-mib-modules='smux tlstm-mib tsm-mib
| > examples/example
| > > examples/notification' \
| > >                 --with-security-modules="tsm" \
| > >                 --with-transports="TLSTCP DTLSUDP" \
| > >                 --enable-shared \
| > >                 --with-defaults \
| > >                 --enable-ipv6 \
| > >                 --with-cflags="-g -O2" \
| > >                 --without-elf
| > >
| > > $ make install
| > > $ cd testing
| > > $ ./RUNFULLTESTS -g tls
| > > DTLS-UDP user certificate tests .......................... 41/?
| > >  This hangs forever in "41" with snmpd.log saying....
| > > ......
| > > 2019-04-08 16:29:11
| > > 2019-04-08 16:29:11
| > > Received 0 byte packet from DTLSUDP: unknown
| > > 2019-04-08 16:29:11
| > > 2019-04-08 16:29:13
| > > Received 0 byte packet from DTLSUDP: unknown
| > > 2019-04-08 16:29:13
| > > 2019-04-08 16:29:15
| > > Received 0 byte packet from DTLSUDP: unknown
| > > 2019-04-08 16:29:15
| > > 2019-04-08 16:29:15 tls verification failure: ok=0 ctx=0x55ee625b4170
| > > depth=0 err=18:self signed certificate
| > > 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
| > > 2019-04-08 16:29:15  TLS error: SSL_read: rc=-1, sslerror = 1
| > > (SSL_ERROR_SSL)
| > > 2019-04-08 16:29:15  TLS Error: certificate verify failed
| > > 2019-04-08 16:29:15 ---- End of OpenSSL Errors ----
| > > 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
| > > 2019-04-08 16:29:15 TLS error: SSL_read: rc=-1, sslerror = 5
| > > (SSL_ERROR_SYSCALL): system_error=0 (Success)
| > > 2019-04-08 16:29:15 TLS Error: (null)
| > > 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
| > > 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
| > > (SSL_ERROR_SYSCALL): system_error=0 (Success)
| > > 2019-04-08 16:29:16 TLS Error: (null)
| > > 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
| > > 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
| > > (SSL_ERROR_SYSCALL): system_error=0 (Success)
| > > 2019-04-08 16:29:16 TLS Error: (null)
| > >
| > > With the fix suggested på Josef I don't see the DTLSUDP problem, but
| > maybe
| > > there are other problems.
| > >
| > > Regards
| > > Anders Wallin
| > >
| > > PS. thx for adding commit info to a420d87d3, I updated the patch with
| > your
| > > commit comments
| > >
| > >
| > > On Mon, Apr 8, 2019 at 3:27 PM Masayoshi Mizuma <msy...@gm...>
| > > wrote:
| > >
| > >> Hi Josef,
| > >>
| > >> I attach two patches to fix the memory inconsistency if the request is
| > >> resend and timed out.
| > >> Could you try them?
| > >>
| > >> - 0001-agentx-master-Return-when-NETSNMP_CALLBACK_OP_RESEND.patch
| > >>
| > >>   This patch was posted by Anders, and I tried to add the description.
| > >>   This patch fixes the missing NETSNMP_CALLBACK_OP_RESEND callback.
| > >>
| > >> - 0002-snmplib-snmp_api-Remove-the-request-on-the-session-w.patch
| > >>
| > >>   This patch fixes the race between NETSNMP_CALLBACK_OP_SEND_FAILED
| > >>   and NETSNMP_CALLBACK_OP_TIMED_OUT callback. If the request is failed,
| > >>   then remove the request from the internal session.
| > >>
| > >> Thanks,
| > >> Masa
| > >>
| > >> On 4/3/19 9:34 AM, Anders Wallin wrote:
| > >>> The introduction of that code fixes another issue;
| > >>> "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
| > >>> Author: Bill Fenner <fe...@gm...>
| > >>> Date:   Wed Dec 20 21:52:10 2017 +0000
| > >>>
| > >>>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
| > >>> agent
| > >>>
| > >>>     With the patch in 1214, the snmp_api code assumed that if magic was
| > >>>     set, it was the "struct synch-state" from snmp_client.  Of course,
| > >>>     magic belongs to the caller, and the perl library uses it
| > >> differently,
| > >>>     so reaching into it is verboten.  Introduce a new callback (that
| > >>>     was already introduced in 5.8) to report this "retries exceeded"
| > >>>     state, and use it in snmp_client."
| > >>>
| > >>> I think the problem is really about shutting down the agentx connection
| > >>> when one(1) response is to late. I have
| > >>> done 2 patches (one that only write a better log message and one that
| > >>> removes the "bad" code.
| > >>> With these patches I don't get any crash. I think that 5.7.3 has this
| > >> issue
| > >>> as well, but it can not be crashed with the agentofdead code
| > >>>
| > >>> Can you please try this?
| > >>>
| > >>> Regards
| > >>> Anders Wallin
| > >>>
| > >>>
| > >>> On Wed, Apr 3, 2019 at 12:35 PM Josef Ridky <jr...@re...> wrote:
| > >>>
| > >>>> Hi,
| > >>>>
| > >>>> I have compared net-snmp-5.7.3 and net-snmp-5.8 and I have found, that
| > >>>> following callbacks in snmplib/snmp_api.c causes the core dump issue:
| > >>>>
| > >>>> --- old/snmplib/snmp_api.c      2019-04-03 12:13:55.126769866 +0200
| > >>>> +++ new/snmplib/snmp_api.c      2019-04-03 12:15:18.353420790 +0200
| > >>>> @@ -6731,9 +6731,9 @@ snmp_resend_request(struct session_list
| > >>>>          sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
| > >>>>          sp->s_errno = errno;
| > >>>>          snmp_set_detail(strerror(errno));
| > >>>> -        if (rp->callback)
| > >>>> +/*        if (rp->callback)
| > >>>>              rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
| > >>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
| > >>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
| > >>>>          return -1;
| > >>>>      } else {
| > >>>>          netsnmp_get_monotonic_clock(&now);
| > >>>> @@ -6743,9 +6743,9 @@ snmp_resend_request(struct session_list
| > >>>>          tv.tv_sec += tv.tv_usec / 1000000L;
| > >>>>          tv.tv_usec %= 1000000L;
| > >>>>          rp->expireM = tv;
| > >>>> -        if (rp->callback)
| > >>>> +/*        if (rp->callback)
| > >>>>              rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
| > >>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
| > >>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
| > >>>>      }
| > >>>>      return 0;
| > >>>>  }
| > >>>>
| > >>>> Without them, all works as expected.
| > >>>>
| > >>>> Josef Ridky
| > >>>> Software Engineer
| > >>>> Core Services Team
| > >>>> Red Hat Czech, s.r.o.
| > >>>>
| > >>>> ----- Original Message -----
| > >>>> | From: "Anders Wallin" <wal...@gm...>
| > >>>> | To: "Josef Ridky" <jr...@re...>
| > >>>> | Cc: "net-snmp-coders" <net...@li...>
| > >>>> | Sent: Tuesday, April 2, 2019 6:27:54 PM
| > >>>> | Subject: Re: Core dump with net-snmp-5.8
| > >>>> |
| > >>>> | Hi Josef,
| > >>>> | I can reproduce the issue using the master branch, I will take a
| > look
| > >> at
| > >>>> it
| > >>>> | later tonight or tomorrow
| > >>>> |
| > >>>> | Regards
| > >>>> | Anders Wallin
| > >>>> |
| > >>>> |
| > >>>> | On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...>
| > wrote:
| > >>>> |
| > >>>> | > Hi,
| > >>>> | >
| > >>>> | > thanks for your patch. Unfortunately, even when I have applied it,
| > >> it
| > >>>> | > still ends with core dump due of 'double free or corruption
| > >> (fasttop)'
| > >>>> | >
| > >>>> | > When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
| > >>>> | >
| > >>>> | > agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
| > >>>> | > snmp_agent: delegate session == 0x56207e165240
| > >>>> | > snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
| > >>>> | > agentx/master: callback resend
| > >>>> | > agentx/master: callback resend
| > >>>> | > agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
| > >>>> | > agentx/master: close 0x56207dfd5400, -1
| > >>>> | > snmp_agent: removed 40 delegated request(s) for session
| > >> 0x56207dfce490
| > >>>> | > snmp_agent: processing delegated request, asp = 0x56207e165240
| > >>>> | > snmp_agent: canceling next walk for asp 0x56207e165240
| > >>>> | > snmp_agent: REMOVE session == 0x56207e165240
| > >>>> | > snmp_agent: agent_session 0x56207e165240 released
| > >>>> | > snmp_agent: processing delegated request, asp = 0x56207e1041a0
| > >>>> | > snmp_agent: canceling next walk for asp 0x56207e1041a0
| > >>>> | > snmp_agent: REMOVE session == 0x56207e1041a0
| > >>>> | > snmp_agent: agent_session 0x56207e1041a0 released
| > >>>> | > snmp_agent: processing delegated request, asp = 0x56207e1656c0
| > >>>> | > snmp_agent: canceling next walk for asp 0x56207e1656c0
| > >>>> | > snmp_agent: REMOVE session == 0x56207e1656c0
| > >>>> | > snmp_agent: agent_session 0x56207e1656c0 released
| > >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11af40
| > >>>> | > snmp_agent: canceling next walk for asp 0x56207e11af40
| > >>>> | > snmp_agent: REMOVE session == 0x56207e11af40
| > >>>> | > snmp_agent: agent_session 0x56207e11af40 released
| > >>>> | > snmp_agent: processing delegated request, asp = 0x56207e118f00
| > >>>> | > snmp_agent: canceling next walk for asp 0x56207e118f00
| > >>>> | > snmp_agent: REMOVE session == 0x56207e118f00
| > >>>> | > snmp_agent: agent_session 0x56207e118f00 released
| > >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11b540
| > >>>> | > snmp_agent: canceling next walk for asp 0x56207e11b540
| > >>>> | > snmp_agent: REMOVE session == 0x56207e11b540
| > >>>> | > snmp_agent: agent_session 0x56207e11b540 released
| > >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11bd00
| > >>>> | > snmp_agent: canceling next walk for asp 0x56207e11bd00
| > >>>> | > snmp_agent: REMOVE session == 0x56207e11bd00
| > >>>> | > snmp_agent: agent_session 0x56207e11bd00 released
| > >>>> | > agentx/master: Continue removing delegated subsession reqests
| > >>>> | > agentx/master: close transport
| > >>>> | > snmp_agent: REMOVE session == 0x56207dfd5400
| > >>>> | > agentx/master: response too late on session 0x56207dfd5400
| > >>>> | > agentx/master: response too late on session 0x56207dfd5400
| > >>>> | > double free or corruption (fasttop)
| > >>>> | > Aborted (core dumped)
| > >>>> | >
| > >>>> | >
| > >>>> | > What's interesting, when I run it with -DALL it pass (at least for
| > >>>> several
| > >>>> | > rounds).
| > >>>> | > It looks like some strange race condition.
| > >>>> | >
| > >>>> | > Regards
| > >>>> | >
| > >>>> | > Josef Ridky
| > >>>> | > Software Engineer
| > >>>> | > Core Services Team
| > >>>> | > Red Hat Czech, s.r.o.
| > >>>> | >
| > >>>> | > ----- Original Message -----
| > >>>> | > | From: "Anders Wallin" <wal...@gm...>
| > >>>> | > | To: "Josef Ridky" <jr...@re...>
| > >>>> | > | Cc: "net-snmp-coders" <net...@li...>
| > >>>> | > | Sent: Tuesday, April 2, 2019 1:46:40 PM
| > >>>> | > | Subject: Re: Core dump with net-snmp-5.8
| > >>>> | > |
| > >>>> | > | Hi Josef,
| > >>>> | > |
| > >>>> | > | I think it's the same issue as
| > >>>> | > https://sourceforge.net/p/net-snmp/bugs/2914/
| > >>>> | > | (where I also posted the solution)
| > >>>> | > | Regards
| > >>>> | > | Anders Wallin
| > >>>> | > |
| > >>>> | > |
| > >>>> | > | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...>
| > >>>> wrote:
| > >>>> | > |
| > >>>> | > | > Hi,
| > >>>> | > | >
| > >>>> | > | > recently, I have hit to an issue in net-snmp-5.8, that is
| > >>>> connected to
| > >>>> | > the
| > >>>> | > | > bug report [1].
| > >>>> | > | >
| > >>>> | > | > When I tried to run agentofdeath test from [1], snmpd daemon
| > >> will
| > >>>> crash
| > >>>> | > | > with malloc(): smallbin double linked list corrupted or double
| > >>>> free()
| > >>>> | > issue
| > >>>> | > | > and dumps core (see bellow).
| > >>>> | > | > From log file, I can identified one issue with "Unknown
| > >> operation".
| > >>>> | > | >
| > >>>> | > | > This issue is in the agentx_got_response function
| > >>>> | > | > (agent/mibgroup/agentx/master.c). There isn't implemented
| > action
| > >>>> for
| > >>>> | > | > NETSNMP_CALLBACK_OP_RESEND (defined in
| > >>>> | > | > include/net-snmp/library/snmp_api.h).
| > >>>> | > | > As result "Unknown operation 6 in agentx_got_response" is
| > shown
| > >> in
| > >>>> log
| > >>>> | > | > file.
| > >>>> | > | >
| > >>>> | > | > /var/log/messages
| > >>>> | > | > -------------------------------
| > >>>> | > | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in
| > >>>> | > | > agentx_got_response
| > >>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in
| > >>>> | > | > agentx_got_response
| > >>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin
| > >> double
| > >>>> | > linked
| > >>>> | > | > list corrupted
| > >>>> | > | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core
| > Dump
| > >>>> (PID
| > >>>> | > | > 13652/UID 0).
| > >>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main
| > >> process
| > >>>> | > exited,
| > >>>> | > | > code=dumped, status=6/ABRT
| > >>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed
| > with
| > >>>> result
| > >>>> | > | > 'core-dump'.
| > >>>> | > | > -------------------------------
| > >>>> | > | >
| > >>>> | > | > The "Unknown operation" callback is caused by newly added
| > piece
| > >> of
| > >>>> | > code in
| > >>>> | > | > snmplib/snmp_api.c:
| > >>>> | > | >
| > >>>> | > | >  static int
| > >>>> | > | >  snmp_resend_request(struct session_list *slp,
| > >> netsnmp_request_list
| > >>>> | > *rp,
| > >>>> | > | >  int incr_retries)
| > >>>> | > | >  {
| > >>>> | > | >
| > >>>> | > | > ...
| > >>>> | > | >
| > >>>> | > | >          tv.tv_sec += tv.tv_usec / 1000000L;
| > >>>> | > | >          tv.tv_usec %= 1000000L;
| > >>>> | > | >          rp->expireM = tv;
| > >>>> | > | > +        if (rp->callback)
| > >>>> | > | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
| > >>>> | > | > +                         rp->pdu->reqid, rp->pdu,
| > rp->cb_data);
| > >>>> | > | >      }
| > >>>> | > | >      return 0;
| > >>>> | > | >  }
| > >>>> | > | >
| > >>>> | > | >
| > >>>> | > | > When I tried to remove it, it just stop complaining about
| > >>>> operation 6,
| > >>>> | > but
| > >>>> | > | > the core dump is still present.
| > >>>> | > | >
| > >>>> | > | > May I ask you for help with this issue? Do you have any idea,
| > >> what
| > >>>> | > causing
| > >>>> | > | > this issue in 5.8 and how to fix it?
| > >>>> | > | > I know, that Jan Safranek has fixed this for 5.7 by commit
| > [2],
| > >>>> but it
| > >>>> | > | > looks like something other has changed and this issue is
| > current
| > >>>> again.
| > >>>> | > | >
| > >>>> | > | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
| > >>>> | > | > [2]
| > >>>> | > | >
| > >>>> | >
| > >>>>
| > >>
| > https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
| > >>>> | > | >
| > >>>> | > | > Regards
| > >>>> | > | >
| > >>>> | > | > Josef Ridky
| > >>>> | > | > Software Engineer
| > >>>> | > | > Core Services Team
| > >>>> | > | > Red Hat Czech, s.r.o.
| > >>>> | > | >
| > >>>> | > | >
| > >>>> | > | >
| > >>>> | > | > _______________________________________________
| > >>>> | > | > Net-snmp-coders mailing list
| > >>>> | > | > Net...@li...
| > >>>> | > | > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
| > >>>> | > | >
| > >>>> | > |
| > >>>> | >
| > >>>> |
| > >>>>
| > >>>
| > >>>
| > >>>
| > >>> _______________________________________________
| > >>> Net-snmp-coders mailing list
| > >>> Net...@li...
| > >>> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
| > >>>
| > >>
| > >
| >
| 

Re: Core dump with net-snmp-5.8

[Sam T. <sta...@cu...>]

Hi Anders,

We've been testing the patch for

https://sourceforge.net/p/net-snmp/bugs/2914/

and this works.

Before the patch, we were getting lots of core dumps where the flow in
master.c hits the default case and memory gets double-freed and a core
happens.
We'd love to get this into V5-8-patches soon.

Thanks,
Sam



On Tue, Apr 9, 2019 at 9:13 AM Sam Tannous <sta...@cu...>
wrote:

> Hi Anders,
>
> I fixed some snmpv3 (bulkget) coredumps a while ago.
> https://sourceforge.net/p/net-snmp/patches/1388/
>
> While not directly related, the (double-free memory) core dumps
> were easily triggered by any error condition within a v3 bulkget.
>
> I'm hoping my patch will get picked up soon :-(
>
> Thanks,
> Sam
>
> On Tue, Apr 9, 2019 at 6:54 AM Anders Wallin <wal...@gm...> wrote:
>
>> Now it works fine!
>>
>> thx
>> Anders Wallin
>>
>>
>> On Tue, Apr 9, 2019 at 2:26 AM Masayoshi Mizuma <msy...@gm...>
>> wrote:
>>
>>> Hi Anders,
>>>
>>> Thank you for your feedback!
>>> I attach the v2 patch. Could you try it?
>>>
>>> On the v1 patch, I missed the check for the request callback. So, the
>>> request
>>> gets removed even though the callback doesn't run.
>>>
>>> Thanks,
>>> Masa
>>>
>>> On 4/8/19 11:06 AM, Anders Wallin wrote:
>>> > Hi Masa,
>>> >
>>> > looks like it solves the problem reported by Josef, BUT it breaks
>>> DTLSUDP.
>>> > I run the tests w/o analyzing why.
>>> > To reproduce the issue I did the following using net-snmp master
>>> branch,
>>> > plus these patches
>>> > 39485c6f2 - snmplib/snmp_api: Remove the request on the session when
>>> the
>>> > sending is failed (10 minutes ago) <Masayoshi Mizuma>
>>> > 06a4d52d8 - agentx: logging to late responses (5 days ago) <Anders
>>> Wallin>
>>> > a420d87d3 - BUG2914: Agent master needs to treat resend as normal (5
>>> days
>>> > ago) <Anders Wallin>
>>> > eaad09d04 - (origin/master, origin/HEAD, master) Merge branch
>>> > 'V5-8-patches' (9 weeks ago) <Bart Van Assche>
>>> >
>>> > $ ./configure --prefix=/usr \
>>> >                 --with-persistent-directory=/var/lib/net-snmp \
>>> >                 --with-mib-modules='smux tlstm-mib tsm-mib
>>> examples/example
>>> > examples/notification' \
>>> >                 --with-security-modules="tsm" \
>>> >                 --with-transports="TLSTCP DTLSUDP" \
>>> >                 --enable-shared \
>>> >                 --with-defaults \
>>> >                 --enable-ipv6 \
>>> >                 --with-cflags="-g -O2" \
>>> >                 --without-elf
>>> >
>>> > $ make install
>>> > $ cd testing
>>> > $ ./RUNFULLTESTS -g tls
>>> > DTLS-UDP user certificate tests .......................... 41/?
>>> >  This hangs forever in "41" with snmpd.log saying....
>>> > ......
>>> > 2019-04-08 16:29:11
>>> > 2019-04-08 16:29:11
>>> > Received 0 byte packet from DTLSUDP: unknown
>>> > 2019-04-08 16:29:11
>>> > 2019-04-08 16:29:13
>>> > Received 0 byte packet from DTLSUDP: unknown
>>> > 2019-04-08 16:29:13
>>> > 2019-04-08 16:29:15
>>> > Received 0 byte packet from DTLSUDP: unknown
>>> > 2019-04-08 16:29:15
>>> > 2019-04-08 16:29:15 tls verification failure: ok=0 ctx=0x55ee625b4170
>>> > depth=0 err=18:self signed certificate
>>> > 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
>>> > 2019-04-08 16:29:15  TLS error: SSL_read: rc=-1, sslerror = 1
>>> > (SSL_ERROR_SSL)
>>> > 2019-04-08 16:29:15  TLS Error: certificate verify failed
>>> > 2019-04-08 16:29:15 ---- End of OpenSSL Errors ----
>>> > 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
>>> > 2019-04-08 16:29:15 TLS error: SSL_read: rc=-1, sslerror = 5
>>> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
>>> > 2019-04-08 16:29:15 TLS Error: (null)
>>> > 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
>>> > 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
>>> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
>>> > 2019-04-08 16:29:16 TLS Error: (null)
>>> > 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
>>> > 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
>>> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
>>> > 2019-04-08 16:29:16 TLS Error: (null)
>>> >
>>> > With the fix suggested på Josef I don't see the DTLSUDP problem, but
>>> maybe
>>> > there are other problems.
>>> >
>>> > Regards
>>> > Anders Wallin
>>> >
>>> > PS. thx for adding commit info to a420d87d3, I updated the patch with
>>> your
>>> > commit comments
>>> >
>>> >
>>> > On Mon, Apr 8, 2019 at 3:27 PM Masayoshi Mizuma <msy...@gm...
>>> >
>>> > wrote:
>>> >
>>> >> Hi Josef,
>>> >>
>>> >> I attach two patches to fix the memory inconsistency if the request is
>>> >> resend and timed out.
>>> >> Could you try them?
>>> >>
>>> >> - 0001-agentx-master-Return-when-NETSNMP_CALLBACK_OP_RESEND.patch
>>> >>
>>> >>   This patch was posted by Anders, and I tried to add the description.
>>> >>   This patch fixes the missing NETSNMP_CALLBACK_OP_RESEND callback.
>>> >>
>>> >> - 0002-snmplib-snmp_api-Remove-the-request-on-the-session-w.patch
>>> >>
>>> >>   This patch fixes the race between NETSNMP_CALLBACK_OP_SEND_FAILED
>>> >>   and NETSNMP_CALLBACK_OP_TIMED_OUT callback. If the request is
>>> failed,
>>> >>   then remove the request from the internal session.
>>> >>
>>> >> Thanks,
>>> >> Masa
>>> >>
>>> >> On 4/3/19 9:34 AM, Anders Wallin wrote:
>>> >>> The introduction of that code fixes another issue;
>>> >>> "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
>>> >>> Author: Bill Fenner <fe...@gm...>
>>> >>> Date:   Wed Dec 20 21:52:10 2017 +0000
>>> >>>
>>> >>>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad
>>> SNMPv3
>>> >>> agent
>>> >>>
>>> >>>     With the patch in 1214, the snmp_api code assumed that if magic
>>> was
>>> >>>     set, it was the "struct synch-state" from snmp_client.  Of
>>> course,
>>> >>>     magic belongs to the caller, and the perl library uses it
>>> >> differently,
>>> >>>     so reaching into it is verboten.  Introduce a new callback (that
>>> >>>     was already introduced in 5.8) to report this "retries exceeded"
>>> >>>     state, and use it in snmp_client."
>>> >>>
>>> >>> I think the problem is really about shutting down the agentx
>>> connection
>>> >>> when one(1) response is to late. I have
>>> >>> done 2 patches (one that only write a better log message and one that
>>> >>> removes the "bad" code.
>>> >>> With these patches I don't get any crash. I think that 5.7.3 has this
>>> >> issue
>>> >>> as well, but it can not be crashed with the agentofdead code
>>> >>>
>>> >>> Can you please try this?
>>> >>>
>>> >>> Regards
>>> >>> Anders Wallin
>>> >>>
>>> >>>
>>> >>> On Wed, Apr 3, 2019 at 12:35 PM Josef Ridky <jr...@re...>
>>> wrote:
>>> >>>
>>> >>>> Hi,
>>> >>>>
>>> >>>> I have compared net-snmp-5.7.3 and net-snmp-5.8 and I have found,
>>> that
>>> >>>> following callbacks in snmplib/snmp_api.c causes the core dump
>>> issue:
>>> >>>>
>>> >>>> --- old/snmplib/snmp_api.c      2019-04-03 12:13:55.126769866 +0200
>>> >>>> +++ new/snmplib/snmp_api.c      2019-04-03 12:15:18.353420790 +0200
>>> >>>> @@ -6731,9 +6731,9 @@ snmp_resend_request(struct session_list
>>> >>>>          sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
>>> >>>>          sp->s_errno = errno;
>>> >>>>          snmp_set_detail(strerror(errno));
>>> >>>> -        if (rp->callback)
>>> >>>> +/*        if (rp->callback)
>>> >>>>              rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
>>> >>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>>> >>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>>> >>>>          return -1;
>>> >>>>      } else {
>>> >>>>          netsnmp_get_monotonic_clock(&now);
>>> >>>> @@ -6743,9 +6743,9 @@ snmp_resend_request(struct session_list
>>> >>>>          tv.tv_sec += tv.tv_usec / 1000000L;
>>> >>>>          tv.tv_usec %= 1000000L;
>>> >>>>          rp->expireM = tv;
>>> >>>> -        if (rp->callback)
>>> >>>> +/*        if (rp->callback)
>>> >>>>              rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
>>> >>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>>> >>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>>> >>>>      }
>>> >>>>      return 0;
>>> >>>>  }
>>> >>>>
>>> >>>> Without them, all works as expected.
>>> >>>>
>>> >>>> Josef Ridky
>>> >>>> Software Engineer
>>> >>>> Core Services Team
>>> >>>> Red Hat Czech, s.r.o.
>>> >>>>
>>> >>>> ----- Original Message -----
>>> >>>> | From: "Anders Wallin" <wal...@gm...>
>>> >>>> | To: "Josef Ridky" <jr...@re...>
>>> >>>> | Cc: "net-snmp-coders" <net...@li...>
>>> >>>> | Sent: Tuesday, April 2, 2019 6:27:54 PM
>>> >>>> | Subject: Re: Core dump with net-snmp-5.8
>>> >>>> |
>>> >>>> | Hi Josef,
>>> >>>> | I can reproduce the issue using the master branch, I will take a
>>> look
>>> >> at
>>> >>>> it
>>> >>>> | later tonight or tomorrow
>>> >>>> |
>>> >>>> | Regards
>>> >>>> | Anders Wallin
>>> >>>> |
>>> >>>> |
>>> >>>> | On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...>
>>> wrote:
>>> >>>> |
>>> >>>> | > Hi,
>>> >>>> | >
>>> >>>> | > thanks for your patch. Unfortunately, even when I have applied
>>> it,
>>> >> it
>>> >>>> | > still ends with core dump due of 'double free or corruption
>>> >> (fasttop)'
>>> >>>> | >
>>> >>>> | > When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
>>> >>>> | >
>>> >>>> | > agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
>>> >>>> | > snmp_agent: delegate session == 0x56207e165240
>>> >>>> | > snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
>>> >>>> | > agentx/master: callback resend
>>> >>>> | > agentx/master: callback resend
>>> >>>> | > agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
>>> >>>> | > agentx/master: close 0x56207dfd5400, -1
>>> >>>> | > snmp_agent: removed 40 delegated request(s) for session
>>> >> 0x56207dfce490
>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e165240
>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e165240
>>> >>>> | > snmp_agent: REMOVE session == 0x56207e165240
>>> >>>> | > snmp_agent: agent_session 0x56207e165240 released
>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e1041a0
>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e1041a0
>>> >>>> | > snmp_agent: REMOVE session == 0x56207e1041a0
>>> >>>> | > snmp_agent: agent_session 0x56207e1041a0 released
>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e1656c0
>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e1656c0
>>> >>>> | > snmp_agent: REMOVE session == 0x56207e1656c0
>>> >>>> | > snmp_agent: agent_session 0x56207e1656c0 released
>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11af40
>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11af40
>>> >>>> | > snmp_agent: REMOVE session == 0x56207e11af40
>>> >>>> | > snmp_agent: agent_session 0x56207e11af40 released
>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e118f00
>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e118f00
>>> >>>> | > snmp_agent: REMOVE session == 0x56207e118f00
>>> >>>> | > snmp_agent: agent_session 0x56207e118f00 released
>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11b540
>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11b540
>>> >>>> | > snmp_agent: REMOVE session == 0x56207e11b540
>>> >>>> | > snmp_agent: agent_session 0x56207e11b540 released
>>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11bd00
>>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11bd00
>>> >>>> | > snmp_agent: REMOVE session == 0x56207e11bd00
>>> >>>> | > snmp_agent: agent_session 0x56207e11bd00 released
>>> >>>> | > agentx/master: Continue removing delegated subsession reqests
>>> >>>> | > agentx/master: close transport
>>> >>>> | > snmp_agent: REMOVE session == 0x56207dfd5400
>>> >>>> | > agentx/master: response too late on session 0x56207dfd5400
>>> >>>> | > agentx/master: response too late on session 0x56207dfd5400
>>> >>>> | > double free or corruption (fasttop)
>>> >>>> | > Aborted (core dumped)
>>> >>>> | >
>>> >>>> | >
>>> >>>> | > What's interesting, when I run it with -DALL it pass (at least
>>> for
>>> >>>> several
>>> >>>> | > rounds).
>>> >>>> | > It looks like some strange race condition.
>>> >>>> | >
>>> >>>> | > Regards
>>> >>>> | >
>>> >>>> | > Josef Ridky
>>> >>>> | > Software Engineer
>>> >>>> | > Core Services Team
>>> >>>> | > Red Hat Czech, s.r.o.
>>> >>>> | >
>>> >>>> | > ----- Original Message -----
>>> >>>> | > | From: "Anders Wallin" <wal...@gm...>
>>> >>>> | > | To: "Josef Ridky" <jr...@re...>
>>> >>>> | > | Cc: "net-snmp-coders" <net...@li...>
>>> >>>> | > | Sent: Tuesday, April 2, 2019 1:46:40 PM
>>> >>>> | > | Subject: Re: Core dump with net-snmp-5.8
>>> >>>> | > |
>>> >>>> | > | Hi Josef,
>>> >>>> | > |
>>> >>>> | > | I think it's the same issue as
>>> >>>> | > https://sourceforge.net/p/net-snmp/bugs/2914/
>>> >>>> | > | (where I also posted the solution)
>>> >>>> | > | Regards
>>> >>>> | > | Anders Wallin
>>> >>>> | > |
>>> >>>> | > |
>>> >>>> | > | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <
>>> jr...@re...>
>>> >>>> wrote:
>>> >>>> | > |
>>> >>>> | > | > Hi,
>>> >>>> | > | >
>>> >>>> | > | > recently, I have hit to an issue in net-snmp-5.8, that is
>>> >>>> connected to
>>> >>>> | > the
>>> >>>> | > | > bug report [1].
>>> >>>> | > | >
>>> >>>> | > | > When I tried to run agentofdeath test from [1], snmpd daemon
>>> >> will
>>> >>>> crash
>>> >>>> | > | > with malloc(): smallbin double linked list corrupted or
>>> double
>>> >>>> free()
>>> >>>> | > issue
>>> >>>> | > | > and dumps core (see bellow).
>>> >>>> | > | > From log file, I can identified one issue with "Unknown
>>> >> operation".
>>> >>>> | > | >
>>> >>>> | > | > This issue is in the agentx_got_response function
>>> >>>> | > | > (agent/mibgroup/agentx/master.c). There isn't implemented
>>> action
>>> >>>> for
>>> >>>> | > | > NETSNMP_CALLBACK_OP_RESEND (defined in
>>> >>>> | > | > include/net-snmp/library/snmp_api.h).
>>> >>>> | > | > As result "Unknown operation 6 in agentx_got_response" is
>>> shown
>>> >> in
>>> >>>> log
>>> >>>> | > | > file.
>>> >>>> | > | >
>>> >>>> | > | > /var/log/messages
>>> >>>> | > | > -------------------------------
>>> >>>> | > | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6
>>> in
>>> >>>> | > | > agentx_got_response
>>> >>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6
>>> in
>>> >>>> | > | > agentx_got_response
>>> >>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin
>>> >> double
>>> >>>> | > linked
>>> >>>> | > | > list corrupted
>>> >>>> | > | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core
>>> Dump
>>> >>>> (PID
>>> >>>> | > | > 13652/UID 0).
>>> >>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main
>>> >> process
>>> >>>> | > exited,
>>> >>>> | > | > code=dumped, status=6/ABRT
>>> >>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed
>>> with
>>> >>>> result
>>> >>>> | > | > 'core-dump'.
>>> >>>> | > | > -------------------------------
>>> >>>> | > | >
>>> >>>> | > | > The "Unknown operation" callback is caused by newly added
>>> piece
>>> >> of
>>> >>>> | > code in
>>> >>>> | > | > snmplib/snmp_api.c:
>>> >>>> | > | >
>>> >>>> | > | >  static int
>>> >>>> | > | >  snmp_resend_request(struct session_list *slp,
>>> >> netsnmp_request_list
>>> >>>> | > *rp,
>>> >>>> | > | >  int incr_retries)
>>> >>>> | > | >  {
>>> >>>> | > | >
>>> >>>> | > | > ...
>>> >>>> | > | >
>>> >>>> | > | >          tv.tv_sec += tv.tv_usec / 1000000L;
>>> >>>> | > | >          tv.tv_usec %= 1000000L;
>>> >>>> | > | >          rp->expireM = tv;
>>> >>>> | > | > +        if (rp->callback)
>>> >>>> | > | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
>>> >>>> | > | > +                         rp->pdu->reqid, rp->pdu,
>>> rp->cb_data);
>>> >>>> | > | >      }
>>> >>>> | > | >      return 0;
>>> >>>> | > | >  }
>>> >>>> | > | >
>>> >>>> | > | >
>>> >>>> | > | > When I tried to remove it, it just stop complaining about
>>> >>>> operation 6,
>>> >>>> | > but
>>> >>>> | > | > the core dump is still present.
>>> >>>> | > | >
>>> >>>> | > | > May I ask you for help with this issue? Do you have any
>>> idea,
>>> >> what
>>> >>>> | > causing
>>> >>>> | > | > this issue in 5.8 and how to fix it?
>>> >>>> | > | > I know, that Jan Safranek has fixed this for 5.7 by commit
>>> [2],
>>> >>>> but it
>>> >>>> | > | > looks like something other has changed and this issue is
>>> current
>>> >>>> again.
>>> >>>> | > | >
>>> >>>> | > | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
>>> >>>> | > | > [2]
>>> >>>> | > | >
>>> >>>> | >
>>> >>>>
>>> >>
>>> https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
>>> >>>> | > | >
>>> >>>> | > | > Regards
>>> >>>> | > | >
>>> >>>> | > | > Josef Ridky
>>> >>>> | > | > Software Engineer
>>> >>>> | > | > Core Services Team
>>> >>>> | > | > Red Hat Czech, s.r.o.
>>> >>>> | > | >
>>> >>>> | > | >
>>> >>>> | > | >
>>> >>>> | > | > _______________________________________________
>>> >>>> | > | > Net-snmp-coders mailing list
>>> >>>> | > | > Net...@li...
>>> >>>> | > | >
>>> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>>> >>>> | > | >
>>> >>>> | > |
>>> >>>> | >
>>> >>>> |
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> Net-snmp-coders mailing list
>>> >>> Net...@li...
>>> >>> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>>> >>>
>>> >>
>>> >
>>>
>> _______________________________________________
>> Net-snmp-coders mailing list
>> Net...@li...
>> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>>
>
>
> --
> *Sam Tannous*
> Engineering
> Cumulus Networks®
> +1 650 383 6700 x 1106
> <http://www.cumulusnetworks,com>www.cumulusnetworks.com
>
> Evaluate Cumulus® Linux®
> https://cumulusnetworks.com/product/secure/evaluate/
>
> Become a Partner
> http://cumulusnetworks.com/partners/become-a-partner/
>


-- 
*Sam Tannous*
Engineering
Cumulus Networks®
+1 650 383 6700 x 1106
<http://www.cumulusnetworks,com>www.cumulusnetworks.com

Evaluate Cumulus® Linux®
https://cumulusnetworks.com/product/secure/evaluate/

Become a Partner
http://cumulusnetworks.com/partners/become-a-partner/

Re: Core dump with net-snmp-5.8

[Sam T. <sta...@cu...>]

Hi Anders,

I fixed some snmpv3 (bulkget) coredumps a while ago.
https://sourceforge.net/p/net-snmp/patches/1388/

While not directly related, the (double-free memory) core dumps
were easily triggered by any error condition within a v3 bulkget.

I'm hoping my patch will get picked up soon :-(

Thanks,
Sam

On Tue, Apr 9, 2019 at 6:54 AM Anders Wallin <wal...@gm...> wrote:

> Now it works fine!
>
> thx
> Anders Wallin
>
>
> On Tue, Apr 9, 2019 at 2:26 AM Masayoshi Mizuma <msy...@gm...>
> wrote:
>
>> Hi Anders,
>>
>> Thank you for your feedback!
>> I attach the v2 patch. Could you try it?
>>
>> On the v1 patch, I missed the check for the request callback. So, the
>> request
>> gets removed even though the callback doesn't run.
>>
>> Thanks,
>> Masa
>>
>> On 4/8/19 11:06 AM, Anders Wallin wrote:
>> > Hi Masa,
>> >
>> > looks like it solves the problem reported by Josef, BUT it breaks
>> DTLSUDP.
>> > I run the tests w/o analyzing why.
>> > To reproduce the issue I did the following using net-snmp master branch,
>> > plus these patches
>> > 39485c6f2 - snmplib/snmp_api: Remove the request on the session when the
>> > sending is failed (10 minutes ago) <Masayoshi Mizuma>
>> > 06a4d52d8 - agentx: logging to late responses (5 days ago) <Anders
>> Wallin>
>> > a420d87d3 - BUG2914: Agent master needs to treat resend as normal (5
>> days
>> > ago) <Anders Wallin>
>> > eaad09d04 - (origin/master, origin/HEAD, master) Merge branch
>> > 'V5-8-patches' (9 weeks ago) <Bart Van Assche>
>> >
>> > $ ./configure --prefix=/usr \
>> >                 --with-persistent-directory=/var/lib/net-snmp \
>> >                 --with-mib-modules='smux tlstm-mib tsm-mib
>> examples/example
>> > examples/notification' \
>> >                 --with-security-modules="tsm" \
>> >                 --with-transports="TLSTCP DTLSUDP" \
>> >                 --enable-shared \
>> >                 --with-defaults \
>> >                 --enable-ipv6 \
>> >                 --with-cflags="-g -O2" \
>> >                 --without-elf
>> >
>> > $ make install
>> > $ cd testing
>> > $ ./RUNFULLTESTS -g tls
>> > DTLS-UDP user certificate tests .......................... 41/?
>> >  This hangs forever in "41" with snmpd.log saying....
>> > ......
>> > 2019-04-08 16:29:11
>> > 2019-04-08 16:29:11
>> > Received 0 byte packet from DTLSUDP: unknown
>> > 2019-04-08 16:29:11
>> > 2019-04-08 16:29:13
>> > Received 0 byte packet from DTLSUDP: unknown
>> > 2019-04-08 16:29:13
>> > 2019-04-08 16:29:15
>> > Received 0 byte packet from DTLSUDP: unknown
>> > 2019-04-08 16:29:15
>> > 2019-04-08 16:29:15 tls verification failure: ok=0 ctx=0x55ee625b4170
>> > depth=0 err=18:self signed certificate
>> > 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
>> > 2019-04-08 16:29:15  TLS error: SSL_read: rc=-1, sslerror = 1
>> > (SSL_ERROR_SSL)
>> > 2019-04-08 16:29:15  TLS Error: certificate verify failed
>> > 2019-04-08 16:29:15 ---- End of OpenSSL Errors ----
>> > 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
>> > 2019-04-08 16:29:15 TLS error: SSL_read: rc=-1, sslerror = 5
>> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
>> > 2019-04-08 16:29:15 TLS Error: (null)
>> > 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
>> > 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
>> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
>> > 2019-04-08 16:29:16 TLS Error: (null)
>> > 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
>> > 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
>> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
>> > 2019-04-08 16:29:16 TLS Error: (null)
>> >
>> > With the fix suggested på Josef I don't see the DTLSUDP problem, but
>> maybe
>> > there are other problems.
>> >
>> > Regards
>> > Anders Wallin
>> >
>> > PS. thx for adding commit info to a420d87d3, I updated the patch with
>> your
>> > commit comments
>> >
>> >
>> > On Mon, Apr 8, 2019 at 3:27 PM Masayoshi Mizuma <msy...@gm...>
>> > wrote:
>> >
>> >> Hi Josef,
>> >>
>> >> I attach two patches to fix the memory inconsistency if the request is
>> >> resend and timed out.
>> >> Could you try them?
>> >>
>> >> - 0001-agentx-master-Return-when-NETSNMP_CALLBACK_OP_RESEND.patch
>> >>
>> >>   This patch was posted by Anders, and I tried to add the description.
>> >>   This patch fixes the missing NETSNMP_CALLBACK_OP_RESEND callback.
>> >>
>> >> - 0002-snmplib-snmp_api-Remove-the-request-on-the-session-w.patch
>> >>
>> >>   This patch fixes the race between NETSNMP_CALLBACK_OP_SEND_FAILED
>> >>   and NETSNMP_CALLBACK_OP_TIMED_OUT callback. If the request is failed,
>> >>   then remove the request from the internal session.
>> >>
>> >> Thanks,
>> >> Masa
>> >>
>> >> On 4/3/19 9:34 AM, Anders Wallin wrote:
>> >>> The introduction of that code fixes another issue;
>> >>> "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
>> >>> Author: Bill Fenner <fe...@gm...>
>> >>> Date:   Wed Dec 20 21:52:10 2017 +0000
>> >>>
>> >>>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad
>> SNMPv3
>> >>> agent
>> >>>
>> >>>     With the patch in 1214, the snmp_api code assumed that if magic
>> was
>> >>>     set, it was the "struct synch-state" from snmp_client.  Of course,
>> >>>     magic belongs to the caller, and the perl library uses it
>> >> differently,
>> >>>     so reaching into it is verboten.  Introduce a new callback (that
>> >>>     was already introduced in 5.8) to report this "retries exceeded"
>> >>>     state, and use it in snmp_client."
>> >>>
>> >>> I think the problem is really about shutting down the agentx
>> connection
>> >>> when one(1) response is to late. I have
>> >>> done 2 patches (one that only write a better log message and one that
>> >>> removes the "bad" code.
>> >>> With these patches I don't get any crash. I think that 5.7.3 has this
>> >> issue
>> >>> as well, but it can not be crashed with the agentofdead code
>> >>>
>> >>> Can you please try this?
>> >>>
>> >>> Regards
>> >>> Anders Wallin
>> >>>
>> >>>
>> >>> On Wed, Apr 3, 2019 at 12:35 PM Josef Ridky <jr...@re...>
>> wrote:
>> >>>
>> >>>> Hi,
>> >>>>
>> >>>> I have compared net-snmp-5.7.3 and net-snmp-5.8 and I have found,
>> that
>> >>>> following callbacks in snmplib/snmp_api.c causes the core dump issue:
>> >>>>
>> >>>> --- old/snmplib/snmp_api.c      2019-04-03 12:13:55.126769866 +0200
>> >>>> +++ new/snmplib/snmp_api.c      2019-04-03 12:15:18.353420790 +0200
>> >>>> @@ -6731,9 +6731,9 @@ snmp_resend_request(struct session_list
>> >>>>          sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
>> >>>>          sp->s_errno = errno;
>> >>>>          snmp_set_detail(strerror(errno));
>> >>>> -        if (rp->callback)
>> >>>> +/*        if (rp->callback)
>> >>>>              rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
>> >>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>> >>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>> >>>>          return -1;
>> >>>>      } else {
>> >>>>          netsnmp_get_monotonic_clock(&now);
>> >>>> @@ -6743,9 +6743,9 @@ snmp_resend_request(struct session_list
>> >>>>          tv.tv_sec += tv.tv_usec / 1000000L;
>> >>>>          tv.tv_usec %= 1000000L;
>> >>>>          rp->expireM = tv;
>> >>>> -        if (rp->callback)
>> >>>> +/*        if (rp->callback)
>> >>>>              rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
>> >>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>> >>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>> >>>>      }
>> >>>>      return 0;
>> >>>>  }
>> >>>>
>> >>>> Without them, all works as expected.
>> >>>>
>> >>>> Josef Ridky
>> >>>> Software Engineer
>> >>>> Core Services Team
>> >>>> Red Hat Czech, s.r.o.
>> >>>>
>> >>>> ----- Original Message -----
>> >>>> | From: "Anders Wallin" <wal...@gm...>
>> >>>> | To: "Josef Ridky" <jr...@re...>
>> >>>> | Cc: "net-snmp-coders" <net...@li...>
>> >>>> | Sent: Tuesday, April 2, 2019 6:27:54 PM
>> >>>> | Subject: Re: Core dump with net-snmp-5.8
>> >>>> |
>> >>>> | Hi Josef,
>> >>>> | I can reproduce the issue using the master branch, I will take a
>> look
>> >> at
>> >>>> it
>> >>>> | later tonight or tomorrow
>> >>>> |
>> >>>> | Regards
>> >>>> | Anders Wallin
>> >>>> |
>> >>>> |
>> >>>> | On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...>
>> wrote:
>> >>>> |
>> >>>> | > Hi,
>> >>>> | >
>> >>>> | > thanks for your patch. Unfortunately, even when I have applied
>> it,
>> >> it
>> >>>> | > still ends with core dump due of 'double free or corruption
>> >> (fasttop)'
>> >>>> | >
>> >>>> | > When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
>> >>>> | >
>> >>>> | > agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
>> >>>> | > snmp_agent: delegate session == 0x56207e165240
>> >>>> | > snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
>> >>>> | > agentx/master: callback resend
>> >>>> | > agentx/master: callback resend
>> >>>> | > agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
>> >>>> | > agentx/master: close 0x56207dfd5400, -1
>> >>>> | > snmp_agent: removed 40 delegated request(s) for session
>> >> 0x56207dfce490
>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e165240
>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e165240
>> >>>> | > snmp_agent: REMOVE session == 0x56207e165240
>> >>>> | > snmp_agent: agent_session 0x56207e165240 released
>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e1041a0
>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e1041a0
>> >>>> | > snmp_agent: REMOVE session == 0x56207e1041a0
>> >>>> | > snmp_agent: agent_session 0x56207e1041a0 released
>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e1656c0
>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e1656c0
>> >>>> | > snmp_agent: REMOVE session == 0x56207e1656c0
>> >>>> | > snmp_agent: agent_session 0x56207e1656c0 released
>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11af40
>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11af40
>> >>>> | > snmp_agent: REMOVE session == 0x56207e11af40
>> >>>> | > snmp_agent: agent_session 0x56207e11af40 released
>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e118f00
>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e118f00
>> >>>> | > snmp_agent: REMOVE session == 0x56207e118f00
>> >>>> | > snmp_agent: agent_session 0x56207e118f00 released
>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11b540
>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11b540
>> >>>> | > snmp_agent: REMOVE session == 0x56207e11b540
>> >>>> | > snmp_agent: agent_session 0x56207e11b540 released
>> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11bd00
>> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11bd00
>> >>>> | > snmp_agent: REMOVE session == 0x56207e11bd00
>> >>>> | > snmp_agent: agent_session 0x56207e11bd00 released
>> >>>> | > agentx/master: Continue removing delegated subsession reqests
>> >>>> | > agentx/master: close transport
>> >>>> | > snmp_agent: REMOVE session == 0x56207dfd5400
>> >>>> | > agentx/master: response too late on session 0x56207dfd5400
>> >>>> | > agentx/master: response too late on session 0x56207dfd5400
>> >>>> | > double free or corruption (fasttop)
>> >>>> | > Aborted (core dumped)
>> >>>> | >
>> >>>> | >
>> >>>> | > What's interesting, when I run it with -DALL it pass (at least
>> for
>> >>>> several
>> >>>> | > rounds).
>> >>>> | > It looks like some strange race condition.
>> >>>> | >
>> >>>> | > Regards
>> >>>> | >
>> >>>> | > Josef Ridky
>> >>>> | > Software Engineer
>> >>>> | > Core Services Team
>> >>>> | > Red Hat Czech, s.r.o.
>> >>>> | >
>> >>>> | > ----- Original Message -----
>> >>>> | > | From: "Anders Wallin" <wal...@gm...>
>> >>>> | > | To: "Josef Ridky" <jr...@re...>
>> >>>> | > | Cc: "net-snmp-coders" <net...@li...>
>> >>>> | > | Sent: Tuesday, April 2, 2019 1:46:40 PM
>> >>>> | > | Subject: Re: Core dump with net-snmp-5.8
>> >>>> | > |
>> >>>> | > | Hi Josef,
>> >>>> | > |
>> >>>> | > | I think it's the same issue as
>> >>>> | > https://sourceforge.net/p/net-snmp/bugs/2914/
>> >>>> | > | (where I also posted the solution)
>> >>>> | > | Regards
>> >>>> | > | Anders Wallin
>> >>>> | > |
>> >>>> | > |
>> >>>> | > | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...
>> >
>> >>>> wrote:
>> >>>> | > |
>> >>>> | > | > Hi,
>> >>>> | > | >
>> >>>> | > | > recently, I have hit to an issue in net-snmp-5.8, that is
>> >>>> connected to
>> >>>> | > the
>> >>>> | > | > bug report [1].
>> >>>> | > | >
>> >>>> | > | > When I tried to run agentofdeath test from [1], snmpd daemon
>> >> will
>> >>>> crash
>> >>>> | > | > with malloc(): smallbin double linked list corrupted or
>> double
>> >>>> free()
>> >>>> | > issue
>> >>>> | > | > and dumps core (see bellow).
>> >>>> | > | > From log file, I can identified one issue with "Unknown
>> >> operation".
>> >>>> | > | >
>> >>>> | > | > This issue is in the agentx_got_response function
>> >>>> | > | > (agent/mibgroup/agentx/master.c). There isn't implemented
>> action
>> >>>> for
>> >>>> | > | > NETSNMP_CALLBACK_OP_RESEND (defined in
>> >>>> | > | > include/net-snmp/library/snmp_api.h).
>> >>>> | > | > As result "Unknown operation 6 in agentx_got_response" is
>> shown
>> >> in
>> >>>> log
>> >>>> | > | > file.
>> >>>> | > | >
>> >>>> | > | > /var/log/messages
>> >>>> | > | > -------------------------------
>> >>>> | > | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6
>> in
>> >>>> | > | > agentx_got_response
>> >>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6
>> in
>> >>>> | > | > agentx_got_response
>> >>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin
>> >> double
>> >>>> | > linked
>> >>>> | > | > list corrupted
>> >>>> | > | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core
>> Dump
>> >>>> (PID
>> >>>> | > | > 13652/UID 0).
>> >>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main
>> >> process
>> >>>> | > exited,
>> >>>> | > | > code=dumped, status=6/ABRT
>> >>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed
>> with
>> >>>> result
>> >>>> | > | > 'core-dump'.
>> >>>> | > | > -------------------------------
>> >>>> | > | >
>> >>>> | > | > The "Unknown operation" callback is caused by newly added
>> piece
>> >> of
>> >>>> | > code in
>> >>>> | > | > snmplib/snmp_api.c:
>> >>>> | > | >
>> >>>> | > | >  static int
>> >>>> | > | >  snmp_resend_request(struct session_list *slp,
>> >> netsnmp_request_list
>> >>>> | > *rp,
>> >>>> | > | >  int incr_retries)
>> >>>> | > | >  {
>> >>>> | > | >
>> >>>> | > | > ...
>> >>>> | > | >
>> >>>> | > | >          tv.tv_sec += tv.tv_usec / 1000000L;
>> >>>> | > | >          tv.tv_usec %= 1000000L;
>> >>>> | > | >          rp->expireM = tv;
>> >>>> | > | > +        if (rp->callback)
>> >>>> | > | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
>> >>>> | > | > +                         rp->pdu->reqid, rp->pdu,
>> rp->cb_data);
>> >>>> | > | >      }
>> >>>> | > | >      return 0;
>> >>>> | > | >  }
>> >>>> | > | >
>> >>>> | > | >
>> >>>> | > | > When I tried to remove it, it just stop complaining about
>> >>>> operation 6,
>> >>>> | > but
>> >>>> | > | > the core dump is still present.
>> >>>> | > | >
>> >>>> | > | > May I ask you for help with this issue? Do you have any idea,
>> >> what
>> >>>> | > causing
>> >>>> | > | > this issue in 5.8 and how to fix it?
>> >>>> | > | > I know, that Jan Safranek has fixed this for 5.7 by commit
>> [2],
>> >>>> but it
>> >>>> | > | > looks like something other has changed and this issue is
>> current
>> >>>> again.
>> >>>> | > | >
>> >>>> | > | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
>> >>>> | > | > [2]
>> >>>> | > | >
>> >>>> | >
>> >>>>
>> >>
>> https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
>> >>>> | > | >
>> >>>> | > | > Regards
>> >>>> | > | >
>> >>>> | > | > Josef Ridky
>> >>>> | > | > Software Engineer
>> >>>> | > | > Core Services Team
>> >>>> | > | > Red Hat Czech, s.r.o.
>> >>>> | > | >
>> >>>> | > | >
>> >>>> | > | >
>> >>>> | > | > _______________________________________________
>> >>>> | > | > Net-snmp-coders mailing list
>> >>>> | > | > Net...@li...
>> >>>> | > | > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>> >>>> | > | >
>> >>>> | > |
>> >>>> | >
>> >>>> |
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Net-snmp-coders mailing list
>> >>> Net...@li...
>> >>> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>> >>>
>> >>
>> >
>>
> _______________________________________________
> Net-snmp-coders mailing list
> Net...@li...
> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>


-- 
*Sam Tannous*
Engineering
Cumulus Networks®
+1 650 383 6700 x 1106
<http://www.cumulusnetworks,com>www.cumulusnetworks.com

Evaluate Cumulus® Linux®
https://cumulusnetworks.com/product/secure/evaluate/

Become a Partner
http://cumulusnetworks.com/partners/become-a-partner/

Re: Core dump with net-snmp-5.8

[Anders W. <wal...@gm...>]

Now it works fine!

thx
Anders Wallin


On Tue, Apr 9, 2019 at 2:26 AM Masayoshi Mizuma <msy...@gm...>
wrote:

> Hi Anders,
>
> Thank you for your feedback!
> I attach the v2 patch. Could you try it?
>
> On the v1 patch, I missed the check for the request callback. So, the
> request
> gets removed even though the callback doesn't run.
>
> Thanks,
> Masa
>
> On 4/8/19 11:06 AM, Anders Wallin wrote:
> > Hi Masa,
> >
> > looks like it solves the problem reported by Josef, BUT it breaks
> DTLSUDP.
> > I run the tests w/o analyzing why.
> > To reproduce the issue I did the following using net-snmp master branch,
> > plus these patches
> > 39485c6f2 - snmplib/snmp_api: Remove the request on the session when the
> > sending is failed (10 minutes ago) <Masayoshi Mizuma>
> > 06a4d52d8 - agentx: logging to late responses (5 days ago) <Anders
> Wallin>
> > a420d87d3 - BUG2914: Agent master needs to treat resend as normal (5 days
> > ago) <Anders Wallin>
> > eaad09d04 - (origin/master, origin/HEAD, master) Merge branch
> > 'V5-8-patches' (9 weeks ago) <Bart Van Assche>
> >
> > $ ./configure --prefix=/usr \
> >                 --with-persistent-directory=/var/lib/net-snmp \
> >                 --with-mib-modules='smux tlstm-mib tsm-mib
> examples/example
> > examples/notification' \
> >                 --with-security-modules="tsm" \
> >                 --with-transports="TLSTCP DTLSUDP" \
> >                 --enable-shared \
> >                 --with-defaults \
> >                 --enable-ipv6 \
> >                 --with-cflags="-g -O2" \
> >                 --without-elf
> >
> > $ make install
> > $ cd testing
> > $ ./RUNFULLTESTS -g tls
> > DTLS-UDP user certificate tests .......................... 41/?
> >  This hangs forever in "41" with snmpd.log saying....
> > ......
> > 2019-04-08 16:29:11
> > 2019-04-08 16:29:11
> > Received 0 byte packet from DTLSUDP: unknown
> > 2019-04-08 16:29:11
> > 2019-04-08 16:29:13
> > Received 0 byte packet from DTLSUDP: unknown
> > 2019-04-08 16:29:13
> > 2019-04-08 16:29:15
> > Received 0 byte packet from DTLSUDP: unknown
> > 2019-04-08 16:29:15
> > 2019-04-08 16:29:15 tls verification failure: ok=0 ctx=0x55ee625b4170
> > depth=0 err=18:self signed certificate
> > 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
> > 2019-04-08 16:29:15  TLS error: SSL_read: rc=-1, sslerror = 1
> > (SSL_ERROR_SSL)
> > 2019-04-08 16:29:15  TLS Error: certificate verify failed
> > 2019-04-08 16:29:15 ---- End of OpenSSL Errors ----
> > 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
> > 2019-04-08 16:29:15 TLS error: SSL_read: rc=-1, sslerror = 5
> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
> > 2019-04-08 16:29:15 TLS Error: (null)
> > 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
> > 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
> > 2019-04-08 16:29:16 TLS Error: (null)
> > 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
> > 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
> > (SSL_ERROR_SYSCALL): system_error=0 (Success)
> > 2019-04-08 16:29:16 TLS Error: (null)
> >
> > With the fix suggested på Josef I don't see the DTLSUDP problem, but
> maybe
> > there are other problems.
> >
> > Regards
> > Anders Wallin
> >
> > PS. thx for adding commit info to a420d87d3, I updated the patch with
> your
> > commit comments
> >
> >
> > On Mon, Apr 8, 2019 at 3:27 PM Masayoshi Mizuma <msy...@gm...>
> > wrote:
> >
> >> Hi Josef,
> >>
> >> I attach two patches to fix the memory inconsistency if the request is
> >> resend and timed out.
> >> Could you try them?
> >>
> >> - 0001-agentx-master-Return-when-NETSNMP_CALLBACK_OP_RESEND.patch
> >>
> >>   This patch was posted by Anders, and I tried to add the description.
> >>   This patch fixes the missing NETSNMP_CALLBACK_OP_RESEND callback.
> >>
> >> - 0002-snmplib-snmp_api-Remove-the-request-on-the-session-w.patch
> >>
> >>   This patch fixes the race between NETSNMP_CALLBACK_OP_SEND_FAILED
> >>   and NETSNMP_CALLBACK_OP_TIMED_OUT callback. If the request is failed,
> >>   then remove the request from the internal session.
> >>
> >> Thanks,
> >> Masa
> >>
> >> On 4/3/19 9:34 AM, Anders Wallin wrote:
> >>> The introduction of that code fixes another issue;
> >>> "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
> >>> Author: Bill Fenner <fe...@gm...>
> >>> Date:   Wed Dec 20 21:52:10 2017 +0000
> >>>
> >>>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
> >>> agent
> >>>
> >>>     With the patch in 1214, the snmp_api code assumed that if magic was
> >>>     set, it was the "struct synch-state" from snmp_client.  Of course,
> >>>     magic belongs to the caller, and the perl library uses it
> >> differently,
> >>>     so reaching into it is verboten.  Introduce a new callback (that
> >>>     was already introduced in 5.8) to report this "retries exceeded"
> >>>     state, and use it in snmp_client."
> >>>
> >>> I think the problem is really about shutting down the agentx connection
> >>> when one(1) response is to late. I have
> >>> done 2 patches (one that only write a better log message and one that
> >>> removes the "bad" code.
> >>> With these patches I don't get any crash. I think that 5.7.3 has this
> >> issue
> >>> as well, but it can not be crashed with the agentofdead code
> >>>
> >>> Can you please try this?
> >>>
> >>> Regards
> >>> Anders Wallin
> >>>
> >>>
> >>> On Wed, Apr 3, 2019 at 12:35 PM Josef Ridky <jr...@re...> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I have compared net-snmp-5.7.3 and net-snmp-5.8 and I have found, that
> >>>> following callbacks in snmplib/snmp_api.c causes the core dump issue:
> >>>>
> >>>> --- old/snmplib/snmp_api.c      2019-04-03 12:13:55.126769866 +0200
> >>>> +++ new/snmplib/snmp_api.c      2019-04-03 12:15:18.353420790 +0200
> >>>> @@ -6731,9 +6731,9 @@ snmp_resend_request(struct session_list
> >>>>          sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
> >>>>          sp->s_errno = errno;
> >>>>          snmp_set_detail(strerror(errno));
> >>>> -        if (rp->callback)
> >>>> +/*        if (rp->callback)
> >>>>              rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
> >>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
> >>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
> >>>>          return -1;
> >>>>      } else {
> >>>>          netsnmp_get_monotonic_clock(&now);
> >>>> @@ -6743,9 +6743,9 @@ snmp_resend_request(struct session_list
> >>>>          tv.tv_sec += tv.tv_usec / 1000000L;
> >>>>          tv.tv_usec %= 1000000L;
> >>>>          rp->expireM = tv;
> >>>> -        if (rp->callback)
> >>>> +/*        if (rp->callback)
> >>>>              rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
> >>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
> >>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
> >>>>      }
> >>>>      return 0;
> >>>>  }
> >>>>
> >>>> Without them, all works as expected.
> >>>>
> >>>> Josef Ridky
> >>>> Software Engineer
> >>>> Core Services Team
> >>>> Red Hat Czech, s.r.o.
> >>>>
> >>>> ----- Original Message -----
> >>>> | From: "Anders Wallin" <wal...@gm...>
> >>>> | To: "Josef Ridky" <jr...@re...>
> >>>> | Cc: "net-snmp-coders" <net...@li...>
> >>>> | Sent: Tuesday, April 2, 2019 6:27:54 PM
> >>>> | Subject: Re: Core dump with net-snmp-5.8
> >>>> |
> >>>> | Hi Josef,
> >>>> | I can reproduce the issue using the master branch, I will take a
> look
> >> at
> >>>> it
> >>>> | later tonight or tomorrow
> >>>> |
> >>>> | Regards
> >>>> | Anders Wallin
> >>>> |
> >>>> |
> >>>> | On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...>
> wrote:
> >>>> |
> >>>> | > Hi,
> >>>> | >
> >>>> | > thanks for your patch. Unfortunately, even when I have applied it,
> >> it
> >>>> | > still ends with core dump due of 'double free or corruption
> >> (fasttop)'
> >>>> | >
> >>>> | > When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
> >>>> | >
> >>>> | > agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
> >>>> | > snmp_agent: delegate session == 0x56207e165240
> >>>> | > snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
> >>>> | > agentx/master: callback resend
> >>>> | > agentx/master: callback resend
> >>>> | > agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
> >>>> | > agentx/master: close 0x56207dfd5400, -1
> >>>> | > snmp_agent: removed 40 delegated request(s) for session
> >> 0x56207dfce490
> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e165240
> >>>> | > snmp_agent: canceling next walk for asp 0x56207e165240
> >>>> | > snmp_agent: REMOVE session == 0x56207e165240
> >>>> | > snmp_agent: agent_session 0x56207e165240 released
> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e1041a0
> >>>> | > snmp_agent: canceling next walk for asp 0x56207e1041a0
> >>>> | > snmp_agent: REMOVE session == 0x56207e1041a0
> >>>> | > snmp_agent: agent_session 0x56207e1041a0 released
> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e1656c0
> >>>> | > snmp_agent: canceling next walk for asp 0x56207e1656c0
> >>>> | > snmp_agent: REMOVE session == 0x56207e1656c0
> >>>> | > snmp_agent: agent_session 0x56207e1656c0 released
> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11af40
> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11af40
> >>>> | > snmp_agent: REMOVE session == 0x56207e11af40
> >>>> | > snmp_agent: agent_session 0x56207e11af40 released
> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e118f00
> >>>> | > snmp_agent: canceling next walk for asp 0x56207e118f00
> >>>> | > snmp_agent: REMOVE session == 0x56207e118f00
> >>>> | > snmp_agent: agent_session 0x56207e118f00 released
> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11b540
> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11b540
> >>>> | > snmp_agent: REMOVE session == 0x56207e11b540
> >>>> | > snmp_agent: agent_session 0x56207e11b540 released
> >>>> | > snmp_agent: processing delegated request, asp = 0x56207e11bd00
> >>>> | > snmp_agent: canceling next walk for asp 0x56207e11bd00
> >>>> | > snmp_agent: REMOVE session == 0x56207e11bd00
> >>>> | > snmp_agent: agent_session 0x56207e11bd00 released
> >>>> | > agentx/master: Continue removing delegated subsession reqests
> >>>> | > agentx/master: close transport
> >>>> | > snmp_agent: REMOVE session == 0x56207dfd5400
> >>>> | > agentx/master: response too late on session 0x56207dfd5400
> >>>> | > agentx/master: response too late on session 0x56207dfd5400
> >>>> | > double free or corruption (fasttop)
> >>>> | > Aborted (core dumped)
> >>>> | >
> >>>> | >
> >>>> | > What's interesting, when I run it with -DALL it pass (at least for
> >>>> several
> >>>> | > rounds).
> >>>> | > It looks like some strange race condition.
> >>>> | >
> >>>> | > Regards
> >>>> | >
> >>>> | > Josef Ridky
> >>>> | > Software Engineer
> >>>> | > Core Services Team
> >>>> | > Red Hat Czech, s.r.o.
> >>>> | >
> >>>> | > ----- Original Message -----
> >>>> | > | From: "Anders Wallin" <wal...@gm...>
> >>>> | > | To: "Josef Ridky" <jr...@re...>
> >>>> | > | Cc: "net-snmp-coders" <net...@li...>
> >>>> | > | Sent: Tuesday, April 2, 2019 1:46:40 PM
> >>>> | > | Subject: Re: Core dump with net-snmp-5.8
> >>>> | > |
> >>>> | > | Hi Josef,
> >>>> | > |
> >>>> | > | I think it's the same issue as
> >>>> | > https://sourceforge.net/p/net-snmp/bugs/2914/
> >>>> | > | (where I also posted the solution)
> >>>> | > | Regards
> >>>> | > | Anders Wallin
> >>>> | > |
> >>>> | > |
> >>>> | > | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...>
> >>>> wrote:
> >>>> | > |
> >>>> | > | > Hi,
> >>>> | > | >
> >>>> | > | > recently, I have hit to an issue in net-snmp-5.8, that is
> >>>> connected to
> >>>> | > the
> >>>> | > | > bug report [1].
> >>>> | > | >
> >>>> | > | > When I tried to run agentofdeath test from [1], snmpd daemon
> >> will
> >>>> crash
> >>>> | > | > with malloc(): smallbin double linked list corrupted or double
> >>>> free()
> >>>> | > issue
> >>>> | > | > and dumps core (see bellow).
> >>>> | > | > From log file, I can identified one issue with "Unknown
> >> operation".
> >>>> | > | >
> >>>> | > | > This issue is in the agentx_got_response function
> >>>> | > | > (agent/mibgroup/agentx/master.c). There isn't implemented
> action
> >>>> for
> >>>> | > | > NETSNMP_CALLBACK_OP_RESEND (defined in
> >>>> | > | > include/net-snmp/library/snmp_api.h).
> >>>> | > | > As result "Unknown operation 6 in agentx_got_response" is
> shown
> >> in
> >>>> log
> >>>> | > | > file.
> >>>> | > | >
> >>>> | > | > /var/log/messages
> >>>> | > | > -------------------------------
> >>>> | > | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in
> >>>> | > | > agentx_got_response
> >>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in
> >>>> | > | > agentx_got_response
> >>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin
> >> double
> >>>> | > linked
> >>>> | > | > list corrupted
> >>>> | > | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core
> Dump
> >>>> (PID
> >>>> | > | > 13652/UID 0).
> >>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main
> >> process
> >>>> | > exited,
> >>>> | > | > code=dumped, status=6/ABRT
> >>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed
> with
> >>>> result
> >>>> | > | > 'core-dump'.
> >>>> | > | > -------------------------------
> >>>> | > | >
> >>>> | > | > The "Unknown operation" callback is caused by newly added
> piece
> >> of
> >>>> | > code in
> >>>> | > | > snmplib/snmp_api.c:
> >>>> | > | >
> >>>> | > | >  static int
> >>>> | > | >  snmp_resend_request(struct session_list *slp,
> >> netsnmp_request_list
> >>>> | > *rp,
> >>>> | > | >  int incr_retries)
> >>>> | > | >  {
> >>>> | > | >
> >>>> | > | > ...
> >>>> | > | >
> >>>> | > | >          tv.tv_sec += tv.tv_usec / 1000000L;
> >>>> | > | >          tv.tv_usec %= 1000000L;
> >>>> | > | >          rp->expireM = tv;
> >>>> | > | > +        if (rp->callback)
> >>>> | > | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
> >>>> | > | > +                         rp->pdu->reqid, rp->pdu,
> rp->cb_data);
> >>>> | > | >      }
> >>>> | > | >      return 0;
> >>>> | > | >  }
> >>>> | > | >
> >>>> | > | >
> >>>> | > | > When I tried to remove it, it just stop complaining about
> >>>> operation 6,
> >>>> | > but
> >>>> | > | > the core dump is still present.
> >>>> | > | >
> >>>> | > | > May I ask you for help with this issue? Do you have any idea,
> >> what
> >>>> | > causing
> >>>> | > | > this issue in 5.8 and how to fix it?
> >>>> | > | > I know, that Jan Safranek has fixed this for 5.7 by commit
> [2],
> >>>> but it
> >>>> | > | > looks like something other has changed and this issue is
> current
> >>>> again.
> >>>> | > | >
> >>>> | > | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
> >>>> | > | > [2]
> >>>> | > | >
> >>>> | >
> >>>>
> >>
> https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
> >>>> | > | >
> >>>> | > | > Regards
> >>>> | > | >
> >>>> | > | > Josef Ridky
> >>>> | > | > Software Engineer
> >>>> | > | > Core Services Team
> >>>> | > | > Red Hat Czech, s.r.o.
> >>>> | > | >
> >>>> | > | >
> >>>> | > | >
> >>>> | > | > _______________________________________________
> >>>> | > | > Net-snmp-coders mailing list
> >>>> | > | > Net...@li...
> >>>> | > | > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
> >>>> | > | >
> >>>> | > |
> >>>> | >
> >>>> |
> >>>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Net-snmp-coders mailing list
> >>> Net...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
> >>>
> >>
> >
>

python code

[Sabitha C. <sab...@ya...>]

Hi,
I would like to put below commands in python code using python net-snmp library, can someone help me, where OIDs are hard coded.

snmpset -v2c -c xxxx <oid value> i 1

snmpset -v2c -c  xxxx <oid value> u xxx

snmpset -v2c -c  xxxx <oid value u xxx 

snmpset -v2c -c  xxxx <oid value> u xxx

snmpset -v2c -c  xxxx <oid value> u xxx

snmpset -v2c -c  xxxx <oid value> u xxx

snmpwalk -v2c -c  xxxx <oid value> u xxx 

Thanks in advanceChenna

Re: Core dump with net-snmp-5.8

[Masayoshi M. <msy...@gm...>]

Hi Anders,

Thank you for your feedback!
I attach the v2 patch. Could you try it?

On the v1 patch, I missed the check for the request callback. So, the request
gets removed even though the callback doesn't run.

Thanks,
Masa

On 4/8/19 11:06 AM, Anders Wallin wrote:
> Hi Masa,
> 
> looks like it solves the problem reported by Josef, BUT it breaks DTLSUDP.
> I run the tests w/o analyzing why.
> To reproduce the issue I did the following using net-snmp master branch,
> plus these patches
> 39485c6f2 - snmplib/snmp_api: Remove the request on the session when the
> sending is failed (10 minutes ago) <Masayoshi Mizuma>
> 06a4d52d8 - agentx: logging to late responses (5 days ago) <Anders Wallin>
> a420d87d3 - BUG2914: Agent master needs to treat resend as normal (5 days
> ago) <Anders Wallin>
> eaad09d04 - (origin/master, origin/HEAD, master) Merge branch
> 'V5-8-patches' (9 weeks ago) <Bart Van Assche>
> 
> $ ./configure --prefix=/usr \
>                 --with-persistent-directory=/var/lib/net-snmp \
>                 --with-mib-modules='smux tlstm-mib tsm-mib examples/example
> examples/notification' \
>                 --with-security-modules="tsm" \
>                 --with-transports="TLSTCP DTLSUDP" \
>                 --enable-shared \
>                 --with-defaults \
>                 --enable-ipv6 \
>                 --with-cflags="-g -O2" \
>                 --without-elf
> 
> $ make install
> $ cd testing
> $ ./RUNFULLTESTS -g tls
> DTLS-UDP user certificate tests .......................... 41/?
>  This hangs forever in "41" with snmpd.log saying....
> ......
> 2019-04-08 16:29:11
> 2019-04-08 16:29:11
> Received 0 byte packet from DTLSUDP: unknown
> 2019-04-08 16:29:11
> 2019-04-08 16:29:13
> Received 0 byte packet from DTLSUDP: unknown
> 2019-04-08 16:29:13
> 2019-04-08 16:29:15
> Received 0 byte packet from DTLSUDP: unknown
> 2019-04-08 16:29:15
> 2019-04-08 16:29:15 tls verification failure: ok=0 ctx=0x55ee625b4170
> depth=0 err=18:self signed certificate
> 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
> 2019-04-08 16:29:15  TLS error: SSL_read: rc=-1, sslerror = 1
> (SSL_ERROR_SSL)
> 2019-04-08 16:29:15  TLS Error: certificate verify failed
> 2019-04-08 16:29:15 ---- End of OpenSSL Errors ----
> 2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
> 2019-04-08 16:29:15 TLS error: SSL_read: rc=-1, sslerror = 5
> (SSL_ERROR_SYSCALL): system_error=0 (Success)
> 2019-04-08 16:29:15 TLS Error: (null)
> 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
> 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
> (SSL_ERROR_SYSCALL): system_error=0 (Success)
> 2019-04-08 16:29:16 TLS Error: (null)
> 2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
> 2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
> (SSL_ERROR_SYSCALL): system_error=0 (Success)
> 2019-04-08 16:29:16 TLS Error: (null)
> 
> With the fix suggested på Josef I don't see the DTLSUDP problem, but maybe
> there are other problems.
> 
> Regards
> Anders Wallin
> 
> PS. thx for adding commit info to a420d87d3, I updated the patch with your
> commit comments
> 
> 
> On Mon, Apr 8, 2019 at 3:27 PM Masayoshi Mizuma <msy...@gm...>
> wrote:
> 
>> Hi Josef,
>>
>> I attach two patches to fix the memory inconsistency if the request is
>> resend and timed out.
>> Could you try them?
>>
>> - 0001-agentx-master-Return-when-NETSNMP_CALLBACK_OP_RESEND.patch
>>
>>   This patch was posted by Anders, and I tried to add the description.
>>   This patch fixes the missing NETSNMP_CALLBACK_OP_RESEND callback.
>>
>> - 0002-snmplib-snmp_api-Remove-the-request-on-the-session-w.patch
>>
>>   This patch fixes the race between NETSNMP_CALLBACK_OP_SEND_FAILED
>>   and NETSNMP_CALLBACK_OP_TIMED_OUT callback. If the request is failed,
>>   then remove the request from the internal session.
>>
>> Thanks,
>> Masa
>>
>> On 4/3/19 9:34 AM, Anders Wallin wrote:
>>> The introduction of that code fixes another issue;
>>> "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
>>> Author: Bill Fenner <fe...@gm...>
>>> Date:   Wed Dec 20 21:52:10 2017 +0000
>>>
>>>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
>>> agent
>>>
>>>     With the patch in 1214, the snmp_api code assumed that if magic was
>>>     set, it was the "struct synch-state" from snmp_client.  Of course,
>>>     magic belongs to the caller, and the perl library uses it
>> differently,
>>>     so reaching into it is verboten.  Introduce a new callback (that
>>>     was already introduced in 5.8) to report this "retries exceeded"
>>>     state, and use it in snmp_client."
>>>
>>> I think the problem is really about shutting down the agentx connection
>>> when one(1) response is to late. I have
>>> done 2 patches (one that only write a better log message and one that
>>> removes the "bad" code.
>>> With these patches I don't get any crash. I think that 5.7.3 has this
>> issue
>>> as well, but it can not be crashed with the agentofdead code
>>>
>>> Can you please try this?
>>>
>>> Regards
>>> Anders Wallin
>>>
>>>
>>> On Wed, Apr 3, 2019 at 12:35 PM Josef Ridky <jr...@re...> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have compared net-snmp-5.7.3 and net-snmp-5.8 and I have found, that
>>>> following callbacks in snmplib/snmp_api.c causes the core dump issue:
>>>>
>>>> --- old/snmplib/snmp_api.c      2019-04-03 12:13:55.126769866 +0200
>>>> +++ new/snmplib/snmp_api.c      2019-04-03 12:15:18.353420790 +0200
>>>> @@ -6731,9 +6731,9 @@ snmp_resend_request(struct session_list
>>>>          sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
>>>>          sp->s_errno = errno;
>>>>          snmp_set_detail(strerror(errno));
>>>> -        if (rp->callback)
>>>> +/*        if (rp->callback)
>>>>              rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
>>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>>>>          return -1;
>>>>      } else {
>>>>          netsnmp_get_monotonic_clock(&now);
>>>> @@ -6743,9 +6743,9 @@ snmp_resend_request(struct session_list
>>>>          tv.tv_sec += tv.tv_usec / 1000000L;
>>>>          tv.tv_usec %= 1000000L;
>>>>          rp->expireM = tv;
>>>> -        if (rp->callback)
>>>> +/*        if (rp->callback)
>>>>              rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
>>>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>>>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>>>>      }
>>>>      return 0;
>>>>  }
>>>>
>>>> Without them, all works as expected.
>>>>
>>>> Josef Ridky
>>>> Software Engineer
>>>> Core Services Team
>>>> Red Hat Czech, s.r.o.
>>>>
>>>> ----- Original Message -----
>>>> | From: "Anders Wallin" <wal...@gm...>
>>>> | To: "Josef Ridky" <jr...@re...>
>>>> | Cc: "net-snmp-coders" <net...@li...>
>>>> | Sent: Tuesday, April 2, 2019 6:27:54 PM
>>>> | Subject: Re: Core dump with net-snmp-5.8
>>>> |
>>>> | Hi Josef,
>>>> | I can reproduce the issue using the master branch, I will take a look
>> at
>>>> it
>>>> | later tonight or tomorrow
>>>> |
>>>> | Regards
>>>> | Anders Wallin
>>>> |
>>>> |
>>>> | On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...> wrote:
>>>> |
>>>> | > Hi,
>>>> | >
>>>> | > thanks for your patch. Unfortunately, even when I have applied it,
>> it
>>>> | > still ends with core dump due of 'double free or corruption
>> (fasttop)'
>>>> | >
>>>> | > When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
>>>> | >
>>>> | > agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
>>>> | > snmp_agent: delegate session == 0x56207e165240
>>>> | > snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
>>>> | > agentx/master: callback resend
>>>> | > agentx/master: callback resend
>>>> | > agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
>>>> | > agentx/master: close 0x56207dfd5400, -1
>>>> | > snmp_agent: removed 40 delegated request(s) for session
>> 0x56207dfce490
>>>> | > snmp_agent: processing delegated request, asp = 0x56207e165240
>>>> | > snmp_agent: canceling next walk for asp 0x56207e165240
>>>> | > snmp_agent: REMOVE session == 0x56207e165240
>>>> | > snmp_agent: agent_session 0x56207e165240 released
>>>> | > snmp_agent: processing delegated request, asp = 0x56207e1041a0
>>>> | > snmp_agent: canceling next walk for asp 0x56207e1041a0
>>>> | > snmp_agent: REMOVE session == 0x56207e1041a0
>>>> | > snmp_agent: agent_session 0x56207e1041a0 released
>>>> | > snmp_agent: processing delegated request, asp = 0x56207e1656c0
>>>> | > snmp_agent: canceling next walk for asp 0x56207e1656c0
>>>> | > snmp_agent: REMOVE session == 0x56207e1656c0
>>>> | > snmp_agent: agent_session 0x56207e1656c0 released
>>>> | > snmp_agent: processing delegated request, asp = 0x56207e11af40
>>>> | > snmp_agent: canceling next walk for asp 0x56207e11af40
>>>> | > snmp_agent: REMOVE session == 0x56207e11af40
>>>> | > snmp_agent: agent_session 0x56207e11af40 released
>>>> | > snmp_agent: processing delegated request, asp = 0x56207e118f00
>>>> | > snmp_agent: canceling next walk for asp 0x56207e118f00
>>>> | > snmp_agent: REMOVE session == 0x56207e118f00
>>>> | > snmp_agent: agent_session 0x56207e118f00 released
>>>> | > snmp_agent: processing delegated request, asp = 0x56207e11b540
>>>> | > snmp_agent: canceling next walk for asp 0x56207e11b540
>>>> | > snmp_agent: REMOVE session == 0x56207e11b540
>>>> | > snmp_agent: agent_session 0x56207e11b540 released
>>>> | > snmp_agent: processing delegated request, asp = 0x56207e11bd00
>>>> | > snmp_agent: canceling next walk for asp 0x56207e11bd00
>>>> | > snmp_agent: REMOVE session == 0x56207e11bd00
>>>> | > snmp_agent: agent_session 0x56207e11bd00 released
>>>> | > agentx/master: Continue removing delegated subsession reqests
>>>> | > agentx/master: close transport
>>>> | > snmp_agent: REMOVE session == 0x56207dfd5400
>>>> | > agentx/master: response too late on session 0x56207dfd5400
>>>> | > agentx/master: response too late on session 0x56207dfd5400
>>>> | > double free or corruption (fasttop)
>>>> | > Aborted (core dumped)
>>>> | >
>>>> | >
>>>> | > What's interesting, when I run it with -DALL it pass (at least for
>>>> several
>>>> | > rounds).
>>>> | > It looks like some strange race condition.
>>>> | >
>>>> | > Regards
>>>> | >
>>>> | > Josef Ridky
>>>> | > Software Engineer
>>>> | > Core Services Team
>>>> | > Red Hat Czech, s.r.o.
>>>> | >
>>>> | > ----- Original Message -----
>>>> | > | From: "Anders Wallin" <wal...@gm...>
>>>> | > | To: "Josef Ridky" <jr...@re...>
>>>> | > | Cc: "net-snmp-coders" <net...@li...>
>>>> | > | Sent: Tuesday, April 2, 2019 1:46:40 PM
>>>> | > | Subject: Re: Core dump with net-snmp-5.8
>>>> | > |
>>>> | > | Hi Josef,
>>>> | > |
>>>> | > | I think it's the same issue as
>>>> | > https://sourceforge.net/p/net-snmp/bugs/2914/
>>>> | > | (where I also posted the solution)
>>>> | > | Regards
>>>> | > | Anders Wallin
>>>> | > |
>>>> | > |
>>>> | > | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...>
>>>> wrote:
>>>> | > |
>>>> | > | > Hi,
>>>> | > | >
>>>> | > | > recently, I have hit to an issue in net-snmp-5.8, that is
>>>> connected to
>>>> | > the
>>>> | > | > bug report [1].
>>>> | > | >
>>>> | > | > When I tried to run agentofdeath test from [1], snmpd daemon
>> will
>>>> crash
>>>> | > | > with malloc(): smallbin double linked list corrupted or double
>>>> free()
>>>> | > issue
>>>> | > | > and dumps core (see bellow).
>>>> | > | > From log file, I can identified one issue with "Unknown
>> operation".
>>>> | > | >
>>>> | > | > This issue is in the agentx_got_response function
>>>> | > | > (agent/mibgroup/agentx/master.c). There isn't implemented action
>>>> for
>>>> | > | > NETSNMP_CALLBACK_OP_RESEND (defined in
>>>> | > | > include/net-snmp/library/snmp_api.h).
>>>> | > | > As result "Unknown operation 6 in agentx_got_response" is shown
>> in
>>>> log
>>>> | > | > file.
>>>> | > | >
>>>> | > | > /var/log/messages
>>>> | > | > -------------------------------
>>>> | > | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in
>>>> | > | > agentx_got_response
>>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in
>>>> | > | > agentx_got_response
>>>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin
>> double
>>>> | > linked
>>>> | > | > list corrupted
>>>> | > | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core Dump
>>>> (PID
>>>> | > | > 13652/UID 0).
>>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main
>> process
>>>> | > exited,
>>>> | > | > code=dumped, status=6/ABRT
>>>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed with
>>>> result
>>>> | > | > 'core-dump'.
>>>> | > | > -------------------------------
>>>> | > | >
>>>> | > | > The "Unknown operation" callback is caused by newly added piece
>> of
>>>> | > code in
>>>> | > | > snmplib/snmp_api.c:
>>>> | > | >
>>>> | > | >  static int
>>>> | > | >  snmp_resend_request(struct session_list *slp,
>> netsnmp_request_list
>>>> | > *rp,
>>>> | > | >  int incr_retries)
>>>> | > | >  {
>>>> | > | >
>>>> | > | > ...
>>>> | > | >
>>>> | > | >          tv.tv_sec += tv.tv_usec / 1000000L;
>>>> | > | >          tv.tv_usec %= 1000000L;
>>>> | > | >          rp->expireM = tv;
>>>> | > | > +        if (rp->callback)
>>>> | > | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
>>>> | > | > +                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>>>> | > | >      }
>>>> | > | >      return 0;
>>>> | > | >  }
>>>> | > | >
>>>> | > | >
>>>> | > | > When I tried to remove it, it just stop complaining about
>>>> operation 6,
>>>> | > but
>>>> | > | > the core dump is still present.
>>>> | > | >
>>>> | > | > May I ask you for help with this issue? Do you have any idea,
>> what
>>>> | > causing
>>>> | > | > this issue in 5.8 and how to fix it?
>>>> | > | > I know, that Jan Safranek has fixed this for 5.7 by commit [2],
>>>> but it
>>>> | > | > looks like something other has changed and this issue is current
>>>> again.
>>>> | > | >
>>>> | > | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
>>>> | > | > [2]
>>>> | > | >
>>>> | >
>>>>
>> https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
>>>> | > | >
>>>> | > | > Regards
>>>> | > | >
>>>> | > | > Josef Ridky
>>>> | > | > Software Engineer
>>>> | > | > Core Services Team
>>>> | > | > Red Hat Czech, s.r.o.
>>>> | > | >
>>>> | > | >
>>>> | > | >
>>>> | > | > _______________________________________________
>>>> | > | > Net-snmp-coders mailing list
>>>> | > | > Net...@li...
>>>> | > | > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>>>> | > | >
>>>> | > |
>>>> | >
>>>> |
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Net-snmp-coders mailing list
>>> Net...@li...
>>> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>>>
>>
> 

Re: Core dump with net-snmp-5.8

[Anders W. <wal...@gm...>]

Hi Masa,

looks like it solves the problem reported by Josef, BUT it breaks DTLSUDP.
I run the tests w/o analyzing why.
To reproduce the issue I did the following using net-snmp master branch,
plus these patches
39485c6f2 - snmplib/snmp_api: Remove the request on the session when the
sending is failed (10 minutes ago) <Masayoshi Mizuma>
06a4d52d8 - agentx: logging to late responses (5 days ago) <Anders Wallin>
a420d87d3 - BUG2914: Agent master needs to treat resend as normal (5 days
ago) <Anders Wallin>
eaad09d04 - (origin/master, origin/HEAD, master) Merge branch
'V5-8-patches' (9 weeks ago) <Bart Van Assche>

$ ./configure --prefix=/usr \
                --with-persistent-directory=/var/lib/net-snmp \
                --with-mib-modules='smux tlstm-mib tsm-mib examples/example
examples/notification' \
                --with-security-modules="tsm" \
                --with-transports="TLSTCP DTLSUDP" \
                --enable-shared \
                --with-defaults \
                --enable-ipv6 \
                --with-cflags="-g -O2" \
                --without-elf

$ make install
$ cd testing
$ ./RUNFULLTESTS -g tls
DTLS-UDP user certificate tests .......................... 41/?
 This hangs forever in "41" with snmpd.log saying....
......
2019-04-08 16:29:11
2019-04-08 16:29:11
Received 0 byte packet from DTLSUDP: unknown
2019-04-08 16:29:11
2019-04-08 16:29:13
Received 0 byte packet from DTLSUDP: unknown
2019-04-08 16:29:13
2019-04-08 16:29:15
Received 0 byte packet from DTLSUDP: unknown
2019-04-08 16:29:15
2019-04-08 16:29:15 tls verification failure: ok=0 ctx=0x55ee625b4170
depth=0 err=18:self signed certificate
2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
2019-04-08 16:29:15  TLS error: SSL_read: rc=-1, sslerror = 1
(SSL_ERROR_SSL)
2019-04-08 16:29:15  TLS Error: certificate verify failed
2019-04-08 16:29:15 ---- End of OpenSSL Errors ----
2019-04-08 16:29:15 ---- OpenSSL Related Errors: ----
2019-04-08 16:29:15 TLS error: SSL_read: rc=-1, sslerror = 5
(SSL_ERROR_SYSCALL): system_error=0 (Success)
2019-04-08 16:29:15 TLS Error: (null)
2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
(SSL_ERROR_SYSCALL): system_error=0 (Success)
2019-04-08 16:29:16 TLS Error: (null)
2019-04-08 16:29:16 ---- OpenSSL Related Errors: ----
2019-04-08 16:29:16 TLS error: SSL_read: rc=-1, sslerror = 5
(SSL_ERROR_SYSCALL): system_error=0 (Success)
2019-04-08 16:29:16 TLS Error: (null)

With the fix suggested på Josef I don't see the DTLSUDP problem, but maybe
there are other problems.

Regards
Anders Wallin

PS. thx for adding commit info to a420d87d3, I updated the patch with your
commit comments


On Mon, Apr 8, 2019 at 3:27 PM Masayoshi Mizuma <msy...@gm...>
wrote:

> Hi Josef,
>
> I attach two patches to fix the memory inconsistency if the request is
> resend and timed out.
> Could you try them?
>
> - 0001-agentx-master-Return-when-NETSNMP_CALLBACK_OP_RESEND.patch
>
>   This patch was posted by Anders, and I tried to add the description.
>   This patch fixes the missing NETSNMP_CALLBACK_OP_RESEND callback.
>
> - 0002-snmplib-snmp_api-Remove-the-request-on-the-session-w.patch
>
>   This patch fixes the race between NETSNMP_CALLBACK_OP_SEND_FAILED
>   and NETSNMP_CALLBACK_OP_TIMED_OUT callback. If the request is failed,
>   then remove the request from the internal session.
>
> Thanks,
> Masa
>
> On 4/3/19 9:34 AM, Anders Wallin wrote:
> > The introduction of that code fixes another issue;
> > "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
> > Author: Bill Fenner <fe...@gm...>
> > Date:   Wed Dec 20 21:52:10 2017 +0000
> >
> >     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
> > agent
> >
> >     With the patch in 1214, the snmp_api code assumed that if magic was
> >     set, it was the "struct synch-state" from snmp_client.  Of course,
> >     magic belongs to the caller, and the perl library uses it
> differently,
> >     so reaching into it is verboten.  Introduce a new callback (that
> >     was already introduced in 5.8) to report this "retries exceeded"
> >     state, and use it in snmp_client."
> >
> > I think the problem is really about shutting down the agentx connection
> > when one(1) response is to late. I have
> > done 2 patches (one that only write a better log message and one that
> > removes the "bad" code.
> > With these patches I don't get any crash. I think that 5.7.3 has this
> issue
> > as well, but it can not be crashed with the agentofdead code
> >
> > Can you please try this?
> >
> > Regards
> > Anders Wallin
> >
> >
> > On Wed, Apr 3, 2019 at 12:35 PM Josef Ridky <jr...@re...> wrote:
> >
> >> Hi,
> >>
> >> I have compared net-snmp-5.7.3 and net-snmp-5.8 and I have found, that
> >> following callbacks in snmplib/snmp_api.c causes the core dump issue:
> >>
> >> --- old/snmplib/snmp_api.c      2019-04-03 12:13:55.126769866 +0200
> >> +++ new/snmplib/snmp_api.c      2019-04-03 12:15:18.353420790 +0200
> >> @@ -6731,9 +6731,9 @@ snmp_resend_request(struct session_list
> >>          sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
> >>          sp->s_errno = errno;
> >>          snmp_set_detail(strerror(errno));
> >> -        if (rp->callback)
> >> +/*        if (rp->callback)
> >>              rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
> >> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
> >> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
> >>          return -1;
> >>      } else {
> >>          netsnmp_get_monotonic_clock(&now);
> >> @@ -6743,9 +6743,9 @@ snmp_resend_request(struct session_list
> >>          tv.tv_sec += tv.tv_usec / 1000000L;
> >>          tv.tv_usec %= 1000000L;
> >>          rp->expireM = tv;
> >> -        if (rp->callback)
> >> +/*        if (rp->callback)
> >>              rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
> >> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
> >> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
> >>      }
> >>      return 0;
> >>  }
> >>
> >> Without them, all works as expected.
> >>
> >> Josef Ridky
> >> Software Engineer
> >> Core Services Team
> >> Red Hat Czech, s.r.o.
> >>
> >> ----- Original Message -----
> >> | From: "Anders Wallin" <wal...@gm...>
> >> | To: "Josef Ridky" <jr...@re...>
> >> | Cc: "net-snmp-coders" <net...@li...>
> >> | Sent: Tuesday, April 2, 2019 6:27:54 PM
> >> | Subject: Re: Core dump with net-snmp-5.8
> >> |
> >> | Hi Josef,
> >> | I can reproduce the issue using the master branch, I will take a look
> at
> >> it
> >> | later tonight or tomorrow
> >> |
> >> | Regards
> >> | Anders Wallin
> >> |
> >> |
> >> | On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...> wrote:
> >> |
> >> | > Hi,
> >> | >
> >> | > thanks for your patch. Unfortunately, even when I have applied it,
> it
> >> | > still ends with core dump due of 'double free or corruption
> (fasttop)'
> >> | >
> >> | > When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
> >> | >
> >> | > agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
> >> | > snmp_agent: delegate session == 0x56207e165240
> >> | > snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
> >> | > agentx/master: callback resend
> >> | > agentx/master: callback resend
> >> | > agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
> >> | > agentx/master: close 0x56207dfd5400, -1
> >> | > snmp_agent: removed 40 delegated request(s) for session
> 0x56207dfce490
> >> | > snmp_agent: processing delegated request, asp = 0x56207e165240
> >> | > snmp_agent: canceling next walk for asp 0x56207e165240
> >> | > snmp_agent: REMOVE session == 0x56207e165240
> >> | > snmp_agent: agent_session 0x56207e165240 released
> >> | > snmp_agent: processing delegated request, asp = 0x56207e1041a0
> >> | > snmp_agent: canceling next walk for asp 0x56207e1041a0
> >> | > snmp_agent: REMOVE session == 0x56207e1041a0
> >> | > snmp_agent: agent_session 0x56207e1041a0 released
> >> | > snmp_agent: processing delegated request, asp = 0x56207e1656c0
> >> | > snmp_agent: canceling next walk for asp 0x56207e1656c0
> >> | > snmp_agent: REMOVE session == 0x56207e1656c0
> >> | > snmp_agent: agent_session 0x56207e1656c0 released
> >> | > snmp_agent: processing delegated request, asp = 0x56207e11af40
> >> | > snmp_agent: canceling next walk for asp 0x56207e11af40
> >> | > snmp_agent: REMOVE session == 0x56207e11af40
> >> | > snmp_agent: agent_session 0x56207e11af40 released
> >> | > snmp_agent: processing delegated request, asp = 0x56207e118f00
> >> | > snmp_agent: canceling next walk for asp 0x56207e118f00
> >> | > snmp_agent: REMOVE session == 0x56207e118f00
> >> | > snmp_agent: agent_session 0x56207e118f00 released
> >> | > snmp_agent: processing delegated request, asp = 0x56207e11b540
> >> | > snmp_agent: canceling next walk for asp 0x56207e11b540
> >> | > snmp_agent: REMOVE session == 0x56207e11b540
> >> | > snmp_agent: agent_session 0x56207e11b540 released
> >> | > snmp_agent: processing delegated request, asp = 0x56207e11bd00
> >> | > snmp_agent: canceling next walk for asp 0x56207e11bd00
> >> | > snmp_agent: REMOVE session == 0x56207e11bd00
> >> | > snmp_agent: agent_session 0x56207e11bd00 released
> >> | > agentx/master: Continue removing delegated subsession reqests
> >> | > agentx/master: close transport
> >> | > snmp_agent: REMOVE session == 0x56207dfd5400
> >> | > agentx/master: response too late on session 0x56207dfd5400
> >> | > agentx/master: response too late on session 0x56207dfd5400
> >> | > double free or corruption (fasttop)
> >> | > Aborted (core dumped)
> >> | >
> >> | >
> >> | > What's interesting, when I run it with -DALL it pass (at least for
> >> several
> >> | > rounds).
> >> | > It looks like some strange race condition.
> >> | >
> >> | > Regards
> >> | >
> >> | > Josef Ridky
> >> | > Software Engineer
> >> | > Core Services Team
> >> | > Red Hat Czech, s.r.o.
> >> | >
> >> | > ----- Original Message -----
> >> | > | From: "Anders Wallin" <wal...@gm...>
> >> | > | To: "Josef Ridky" <jr...@re...>
> >> | > | Cc: "net-snmp-coders" <net...@li...>
> >> | > | Sent: Tuesday, April 2, 2019 1:46:40 PM
> >> | > | Subject: Re: Core dump with net-snmp-5.8
> >> | > |
> >> | > | Hi Josef,
> >> | > |
> >> | > | I think it's the same issue as
> >> | > https://sourceforge.net/p/net-snmp/bugs/2914/
> >> | > | (where I also posted the solution)
> >> | > | Regards
> >> | > | Anders Wallin
> >> | > |
> >> | > |
> >> | > | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...>
> >> wrote:
> >> | > |
> >> | > | > Hi,
> >> | > | >
> >> | > | > recently, I have hit to an issue in net-snmp-5.8, that is
> >> connected to
> >> | > the
> >> | > | > bug report [1].
> >> | > | >
> >> | > | > When I tried to run agentofdeath test from [1], snmpd daemon
> will
> >> crash
> >> | > | > with malloc(): smallbin double linked list corrupted or double
> >> free()
> >> | > issue
> >> | > | > and dumps core (see bellow).
> >> | > | > From log file, I can identified one issue with "Unknown
> operation".
> >> | > | >
> >> | > | > This issue is in the agentx_got_response function
> >> | > | > (agent/mibgroup/agentx/master.c). There isn't implemented action
> >> for
> >> | > | > NETSNMP_CALLBACK_OP_RESEND (defined in
> >> | > | > include/net-snmp/library/snmp_api.h).
> >> | > | > As result "Unknown operation 6 in agentx_got_response" is shown
> in
> >> log
> >> | > | > file.
> >> | > | >
> >> | > | > /var/log/messages
> >> | > | > -------------------------------
> >> | > | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in
> >> | > | > agentx_got_response
> >> | > | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in
> >> | > | > agentx_got_response
> >> | > | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin
> double
> >> | > linked
> >> | > | > list corrupted
> >> | > | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core Dump
> >> (PID
> >> | > | > 13652/UID 0).
> >> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main
> process
> >> | > exited,
> >> | > | > code=dumped, status=6/ABRT
> >> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed with
> >> result
> >> | > | > 'core-dump'.
> >> | > | > -------------------------------
> >> | > | >
> >> | > | > The "Unknown operation" callback is caused by newly added piece
> of
> >> | > code in
> >> | > | > snmplib/snmp_api.c:
> >> | > | >
> >> | > | >  static int
> >> | > | >  snmp_resend_request(struct session_list *slp,
> netsnmp_request_list
> >> | > *rp,
> >> | > | >  int incr_retries)
> >> | > | >  {
> >> | > | >
> >> | > | > ...
> >> | > | >
> >> | > | >          tv.tv_sec += tv.tv_usec / 1000000L;
> >> | > | >          tv.tv_usec %= 1000000L;
> >> | > | >          rp->expireM = tv;
> >> | > | > +        if (rp->callback)
> >> | > | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
> >> | > | > +                         rp->pdu->reqid, rp->pdu, rp->cb_data);
> >> | > | >      }
> >> | > | >      return 0;
> >> | > | >  }
> >> | > | >
> >> | > | >
> >> | > | > When I tried to remove it, it just stop complaining about
> >> operation 6,
> >> | > but
> >> | > | > the core dump is still present.
> >> | > | >
> >> | > | > May I ask you for help with this issue? Do you have any idea,
> what
> >> | > causing
> >> | > | > this issue in 5.8 and how to fix it?
> >> | > | > I know, that Jan Safranek has fixed this for 5.7 by commit [2],
> >> but it
> >> | > | > looks like something other has changed and this issue is current
> >> again.
> >> | > | >
> >> | > | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
> >> | > | > [2]
> >> | > | >
> >> | >
> >>
> https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
> >> | > | >
> >> | > | > Regards
> >> | > | >
> >> | > | > Josef Ridky
> >> | > | > Software Engineer
> >> | > | > Core Services Team
> >> | > | > Red Hat Czech, s.r.o.
> >> | > | >
> >> | > | >
> >> | > | >
> >> | > | > _______________________________________________
> >> | > | > Net-snmp-coders mailing list
> >> | > | > Net...@li...
> >> | > | > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
> >> | > | >
> >> | > |
> >> | >
> >> |
> >>
> >
> >
> >
> > _______________________________________________
> > Net-snmp-coders mailing list
> > Net...@li...
> > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
> >
>

Re: Core dump with net-snmp-5.8

[Masayoshi M. <msy...@gm...>]

Hi Josef,

I attach two patches to fix the memory inconsistency if the request is
resend and timed out. 
Could you try them?

- 0001-agentx-master-Return-when-NETSNMP_CALLBACK_OP_RESEND.patch

  This patch was posted by Anders, and I tried to add the description.
  This patch fixes the missing NETSNMP_CALLBACK_OP_RESEND callback.

- 0002-snmplib-snmp_api-Remove-the-request-on-the-session-w.patch

  This patch fixes the race between NETSNMP_CALLBACK_OP_SEND_FAILED 
  and NETSNMP_CALLBACK_OP_TIMED_OUT callback. If the request is failed,
  then remove the request from the internal session.

Thanks,
Masa

On 4/3/19 9:34 AM, Anders Wallin wrote:
> The introduction of that code fixes another issue;
> "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
> Author: Bill Fenner <fe...@gm...>
> Date:   Wed Dec 20 21:52:10 2017 +0000
> 
>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
> agent
> 
>     With the patch in 1214, the snmp_api code assumed that if magic was
>     set, it was the "struct synch-state" from snmp_client.  Of course,
>     magic belongs to the caller, and the perl library uses it differently,
>     so reaching into it is verboten.  Introduce a new callback (that
>     was already introduced in 5.8) to report this "retries exceeded"
>     state, and use it in snmp_client."
> 
> I think the problem is really about shutting down the agentx connection
> when one(1) response is to late. I have
> done 2 patches (one that only write a better log message and one that
> removes the "bad" code.
> With these patches I don't get any crash. I think that 5.7.3 has this issue
> as well, but it can not be crashed with the agentofdead code
> 
> Can you please try this?
> 
> Regards
> Anders Wallin
> 
> 
> On Wed, Apr 3, 2019 at 12:35 PM Josef Ridky <jr...@re...> wrote:
> 
>> Hi,
>>
>> I have compared net-snmp-5.7.3 and net-snmp-5.8 and I have found, that
>> following callbacks in snmplib/snmp_api.c causes the core dump issue:
>>
>> --- old/snmplib/snmp_api.c      2019-04-03 12:13:55.126769866 +0200
>> +++ new/snmplib/snmp_api.c      2019-04-03 12:15:18.353420790 +0200
>> @@ -6731,9 +6731,9 @@ snmp_resend_request(struct session_list
>>          sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
>>          sp->s_errno = errno;
>>          snmp_set_detail(strerror(errno));
>> -        if (rp->callback)
>> +/*        if (rp->callback)
>>              rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>>          return -1;
>>      } else {
>>          netsnmp_get_monotonic_clock(&now);
>> @@ -6743,9 +6743,9 @@ snmp_resend_request(struct session_list
>>          tv.tv_sec += tv.tv_usec / 1000000L;
>>          tv.tv_usec %= 1000000L;
>>          rp->expireM = tv;
>> -        if (rp->callback)
>> +/*        if (rp->callback)
>>              rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
>> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>>      }
>>      return 0;
>>  }
>>
>> Without them, all works as expected.
>>
>> Josef Ridky
>> Software Engineer
>> Core Services Team
>> Red Hat Czech, s.r.o.
>>
>> ----- Original Message -----
>> | From: "Anders Wallin" <wal...@gm...>
>> | To: "Josef Ridky" <jr...@re...>
>> | Cc: "net-snmp-coders" <net...@li...>
>> | Sent: Tuesday, April 2, 2019 6:27:54 PM
>> | Subject: Re: Core dump with net-snmp-5.8
>> |
>> | Hi Josef,
>> | I can reproduce the issue using the master branch, I will take a look at
>> it
>> | later tonight or tomorrow
>> |
>> | Regards
>> | Anders Wallin
>> |
>> |
>> | On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...> wrote:
>> |
>> | > Hi,
>> | >
>> | > thanks for your patch. Unfortunately, even when I have applied it, it
>> | > still ends with core dump due of 'double free or corruption (fasttop)'
>> | >
>> | > When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
>> | >
>> | > agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
>> | > snmp_agent: delegate session == 0x56207e165240
>> | > snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
>> | > agentx/master: callback resend
>> | > agentx/master: callback resend
>> | > agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
>> | > agentx/master: close 0x56207dfd5400, -1
>> | > snmp_agent: removed 40 delegated request(s) for session 0x56207dfce490
>> | > snmp_agent: processing delegated request, asp = 0x56207e165240
>> | > snmp_agent: canceling next walk for asp 0x56207e165240
>> | > snmp_agent: REMOVE session == 0x56207e165240
>> | > snmp_agent: agent_session 0x56207e165240 released
>> | > snmp_agent: processing delegated request, asp = 0x56207e1041a0
>> | > snmp_agent: canceling next walk for asp 0x56207e1041a0
>> | > snmp_agent: REMOVE session == 0x56207e1041a0
>> | > snmp_agent: agent_session 0x56207e1041a0 released
>> | > snmp_agent: processing delegated request, asp = 0x56207e1656c0
>> | > snmp_agent: canceling next walk for asp 0x56207e1656c0
>> | > snmp_agent: REMOVE session == 0x56207e1656c0
>> | > snmp_agent: agent_session 0x56207e1656c0 released
>> | > snmp_agent: processing delegated request, asp = 0x56207e11af40
>> | > snmp_agent: canceling next walk for asp 0x56207e11af40
>> | > snmp_agent: REMOVE session == 0x56207e11af40
>> | > snmp_agent: agent_session 0x56207e11af40 released
>> | > snmp_agent: processing delegated request, asp = 0x56207e118f00
>> | > snmp_agent: canceling next walk for asp 0x56207e118f00
>> | > snmp_agent: REMOVE session == 0x56207e118f00
>> | > snmp_agent: agent_session 0x56207e118f00 released
>> | > snmp_agent: processing delegated request, asp = 0x56207e11b540
>> | > snmp_agent: canceling next walk for asp 0x56207e11b540
>> | > snmp_agent: REMOVE session == 0x56207e11b540
>> | > snmp_agent: agent_session 0x56207e11b540 released
>> | > snmp_agent: processing delegated request, asp = 0x56207e11bd00
>> | > snmp_agent: canceling next walk for asp 0x56207e11bd00
>> | > snmp_agent: REMOVE session == 0x56207e11bd00
>> | > snmp_agent: agent_session 0x56207e11bd00 released
>> | > agentx/master: Continue removing delegated subsession reqests
>> | > agentx/master: close transport
>> | > snmp_agent: REMOVE session == 0x56207dfd5400
>> | > agentx/master: response too late on session 0x56207dfd5400
>> | > agentx/master: response too late on session 0x56207dfd5400
>> | > double free or corruption (fasttop)
>> | > Aborted (core dumped)
>> | >
>> | >
>> | > What's interesting, when I run it with -DALL it pass (at least for
>> several
>> | > rounds).
>> | > It looks like some strange race condition.
>> | >
>> | > Regards
>> | >
>> | > Josef Ridky
>> | > Software Engineer
>> | > Core Services Team
>> | > Red Hat Czech, s.r.o.
>> | >
>> | > ----- Original Message -----
>> | > | From: "Anders Wallin" <wal...@gm...>
>> | > | To: "Josef Ridky" <jr...@re...>
>> | > | Cc: "net-snmp-coders" <net...@li...>
>> | > | Sent: Tuesday, April 2, 2019 1:46:40 PM
>> | > | Subject: Re: Core dump with net-snmp-5.8
>> | > |
>> | > | Hi Josef,
>> | > |
>> | > | I think it's the same issue as
>> | > https://sourceforge.net/p/net-snmp/bugs/2914/
>> | > | (where I also posted the solution)
>> | > | Regards
>> | > | Anders Wallin
>> | > |
>> | > |
>> | > | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...>
>> wrote:
>> | > |
>> | > | > Hi,
>> | > | >
>> | > | > recently, I have hit to an issue in net-snmp-5.8, that is
>> connected to
>> | > the
>> | > | > bug report [1].
>> | > | >
>> | > | > When I tried to run agentofdeath test from [1], snmpd daemon will
>> crash
>> | > | > with malloc(): smallbin double linked list corrupted or double
>> free()
>> | > issue
>> | > | > and dumps core (see bellow).
>> | > | > From log file, I can identified one issue with "Unknown operation".
>> | > | >
>> | > | > This issue is in the agentx_got_response function
>> | > | > (agent/mibgroup/agentx/master.c). There isn't implemented action
>> for
>> | > | > NETSNMP_CALLBACK_OP_RESEND (defined in
>> | > | > include/net-snmp/library/snmp_api.h).
>> | > | > As result "Unknown operation 6 in agentx_got_response" is shown in
>> log
>> | > | > file.
>> | > | >
>> | > | > /var/log/messages
>> | > | > -------------------------------
>> | > | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in
>> | > | > agentx_got_response
>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in
>> | > | > agentx_got_response
>> | > | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin double
>> | > linked
>> | > | > list corrupted
>> | > | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core Dump
>> (PID
>> | > | > 13652/UID 0).
>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main process
>> | > exited,
>> | > | > code=dumped, status=6/ABRT
>> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed with
>> result
>> | > | > 'core-dump'.
>> | > | > -------------------------------
>> | > | >
>> | > | > The "Unknown operation" callback is caused by newly added piece of
>> | > code in
>> | > | > snmplib/snmp_api.c:
>> | > | >
>> | > | >  static int
>> | > | >  snmp_resend_request(struct session_list *slp, netsnmp_request_list
>> | > *rp,
>> | > | >  int incr_retries)
>> | > | >  {
>> | > | >
>> | > | > ...
>> | > | >
>> | > | >          tv.tv_sec += tv.tv_usec / 1000000L;
>> | > | >          tv.tv_usec %= 1000000L;
>> | > | >          rp->expireM = tv;
>> | > | > +        if (rp->callback)
>> | > | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
>> | > | > +                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>> | > | >      }
>> | > | >      return 0;
>> | > | >  }
>> | > | >
>> | > | >
>> | > | > When I tried to remove it, it just stop complaining about
>> operation 6,
>> | > but
>> | > | > the core dump is still present.
>> | > | >
>> | > | > May I ask you for help with this issue? Do you have any idea, what
>> | > causing
>> | > | > this issue in 5.8 and how to fix it?
>> | > | > I know, that Jan Safranek has fixed this for 5.7 by commit [2],
>> but it
>> | > | > looks like something other has changed and this issue is current
>> again.
>> | > | >
>> | > | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
>> | > | > [2]
>> | > | >
>> | >
>> https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
>> | > | >
>> | > | > Regards
>> | > | >
>> | > | > Josef Ridky
>> | > | > Software Engineer
>> | > | > Core Services Team
>> | > | > Red Hat Czech, s.r.o.
>> | > | >
>> | > | >
>> | > | >
>> | > | > _______________________________________________
>> | > | > Net-snmp-coders mailing list
>> | > | > Net...@li...
>> | > | > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>> | > | >
>> | > |
>> | >
>> |
>>
> 
> 
> 
> _______________________________________________
> Net-snmp-coders mailing list
> Net...@li...
> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
> 

SNMP V3 Authentication (MD5 + DES) failed

[Saida R. <sai...@gm...>]

Hi All,
          Thanks you very much in advance for your inputs.

I am getting the very peculiar issue, while using the net-snmp for testing
V3.
V3 authentication is failing during sending Traps as well as with SnmpWalk.
But the same issue is not observed with various other tools like
ireasoning,, mgsoft ...

Below are the details of the configuration and error messages I have
observed.
Please do the needful.

PS: Even sometimes with the same setup, V3 operations are getting success.
Server is used by me alone.

*Configuration:*

*snmptrapd.conf:*

createUser -e 0x80000F1500 testv3stuff MD5 test1234 DES test1234

authUser log,execute,net testv3stuff

authCommunity log,execute,net public

*Command For Starting the TrapDaemon: (NET-SNMP Version:  5.7.2)*
snmpwalk -v 3 -l authPriv -u testv3stuff -a MD5 -A test1234 -x DES -X
test1234 167.254.169.78:1611 .1.3.6

*Console Logs:*
                     :
trace: sc_get_properlength(): scapi.c, 138:
trace: set_enginetime(): lcd_time.c, 393:
lcd_set_enginetime: engineID 80 00 0F 15 00 : boots=0, time=0
trace: usm_get_user_from_list(): snmpusm.c, 3527:
usm: match on user testv3stuff
trace: usm_check_secLevel(): snmpusm.c, 3404:
comparex: Comparing: 1 3 SNMP-USER-BASED-SM-MIB::usmNoPrivProtocol
trace: sc_check_keyed_hash(): scapi.c, 609:
trace: sc_generate_keyed_hash(): scapi.c, 303:
trace: sc_get_properlength(): scapi.c, 138:
trace: usm_process_in_msg(): snmpusm.c, 2511:
usm: Verification failed.
Authentication failed for testv3stuff
trace: snmpv3_parse(): snmp_api.c, 3781:
dumph_recv:     ScopedPDU
trace: _snmp_parse(): snmp_api.c, 4159:
snmp_parse: Parsed SNMPv3 message (secName:testv3stuff, secLevel:authPriv):
USM authentication failure (incorrect password or key)
trace: _sess_process_packet(): snmp_api.c, 5244:
sess_process_packet: received message id#51087 reqid#0 len 483
trace: _sess_process_packet(): snmp_api.c, 5247:
sess_process_packet: parse fail
trace: _sess_read(): snmp_api.c, 5538:
                  :

-- 
Best Regards,
K Saida Rao.

RE: Core dump with net-snmp-5.8

[Matsumoto, S. <sho...@jp...>]

Hi,

> I will check, but it will be tomorrow
>I don't know how I read the code last time, I was referring to the wrong commit where the code was introduced, it was introduced with
>"commit b7b50bbac7f21a924149d03da26ff0a44b25ec60

Thank you for your response.

I share steps to reproduce the issue just to be sure.


0.      extract files from reproducer.tar.gz

1.      deploy snmpd.conf

2.      start snmpd

3.      make (to generate sub_agent)

4.      ./sub_agent &

5.      pkill -STOP -x sub_agent

6.      snmpwalk -v2c -c "public" 127.0.0.1 .1.3.6.1.4.1.99999 -Ona

Regards,
Shogo Matsumoto


From: Anders Wallin [mailto:wal...@gm...]
Sent: Thursday, April 4, 2019 8:37 PM
To: sho...@jp...<mailto:sho...@jp...>
Cc: net...@li...<mailto:net...@li...>
Subject: Re: Core dump with net-snmp-5.8

I will check, but it will be tomorrow
I don't know how I read the code last time, I was referring to the wrong commit where the code was introduced, it was introduced with
"commit b7b50bbac7f21a924149d03da26ff0a44b25ec60
Author: VMwareDev Randy <snm...@vm...<mailto:snm...@vm...>>
Date:   Mon Jun 22 22:20:43 2015 -0400

    snmp_send callback updates

    - add new NETSNMP_CALLBACK_OP_RESEND
    - add missing calls for NETSNMP_CALLBACK_OP_SEND_FAILED

    Signed-off-by: Robert Story <rs...@fr...<mailto:rs...@fr...>>"

Regards
Anders Wallin

On Thu, Apr 4, 2019 at 10:48 AM Matsumoto, Shogo <sho...@jp...<mailto:sho...@jp...>> wrote:
Hi,

The issue also occurs with the following patches.

    NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3

 0001-agentx-logging-to-late-responses.patch
 0002-agentx-do-not-shut-down-all-sessions-when-one-sessio.patch


The issue occurs with the following patch (2914) too but I found
the cause of this issue.

 https://sourceforge.net/p/net-snmp/bugs/2914/
 0001-BUG2914-Agent-master-needs-to-treat-resend-as-normal.patch


With the patch 2914, netsnmp_free_delegated_cache is called
several times for the same object as follows:

 1. snmp_resend_request calls agentx_got_response with NETSNMP_CALLBACK_OP_RESEND,
 2. agentx_got_response's NETSNMP_CALLBACK_OP_RESEND handler do nothing
 3. snmp_resend_request calls agentx_got_response with NETSNMP_CALLBACK_OP_SEND_FAILED,
 4. agentx_got_response's NETSNMP_CALLBACK_OP_SEND_FAILED handler calls netsnmp_free_delegated_cache,
 5. snmp_sess_close calls agentx_got_response with NETSNMP_CALLBACK_OP_TIMED_OUT,
 6. agentx_got_response's NETSNMP_CALLBACK_OP_TIMED_OUT handler calls netsnmp_free_delegated_cache
    (double free)

gdb
--------------------------------------------------------------------------
Breakpoint 2, snmp_resend_request (slp=slp@entry=0x564eec5df000, rp=rp@entry=0x564eec5eb160, incr_retries=1) at snmp_api.c:6747
6747                rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
(gdb) c
Continuing.

Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f2ec0) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x564eec5f2ec0) at agent_handler.c:929
#1  0x00007fab254d5363 in agentx_got_response (operation=<optimized out>, session=0x564eec4ad560, reqid=2, pdu=0x564eec5e3050, magic=<optimized out>)
    at mibgroup/agentx/master.c:262
#2  0x00007fab24c6b58f in snmp_sess_timeout (sessp=sessp@entry=0x564eec5df000) at snmp_api.c:6813
#3  0x00007fab24c6b710 in snmp_timeout () at snmp_api.c:6660
#4  0x0000564eeb4c0f58 in receive () at snmpd.c:1347
#5  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Breakpoint 1, snmp_resend_request (slp=slp@entry=0x564eec5df000, rp=rp@entry=0x564eec5f3e50, incr_retries=1) at snmp_api.c:6735
6735                rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
(gdb) c
Continuing.

Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
#1  0x00007fab254d541a in agentx_got_response (operation=3, session=0x564eec4ad560, reqid=4, pdu=0x564eec5e54a0, magic=0x564eec5f3730)
    at mibgroup/agentx/master.c:223
#2  0x00007fab24c69325 in snmp_resend_request (slp=slp@entry=0x564eec5df000, rp=rp@entry=0x564eec5f3e50, incr_retries=1) at snmp_api.c:6735
#3  0x00007fab24c6b5db in snmp_sess_timeout (sessp=sessp@entry=0x564eec5df000) at snmp_api.c:6826
#4  0x00007fab24c6b710 in snmp_timeout () at snmp_api.c:6660
#5  0x0000564eeb4c0f58 in receive () at snmpd.c:1347
#6  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
#1  0x00007fab254d541a in agentx_got_response (operation=2, session=0x564eec4ad560, reqid=4, pdu=0x564eec5e54a0, magic=0x564eec5f3730)
    at mibgroup/agentx/master.c:223
#2  0x00007fab24c69586 in snmp_sess_close (sessp=0x564eec5df000) at snmp_api.c:1975
#3  0x00007fab24c6afea in snmp_sess_select_info2_flags (sessp=0x0, numfds=0x7fff68db3694, fdset=0x7fff68db36b0, timeout=0x7fff68db36a0,
    block=0x7fff68db369c, flags=0) at snmp_api.c:6556
#4  0x0000564eeb4c0e95 in receive () at snmpd.c:1263
#5  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Program received signal SIGABRT, Aborted.
0x00007fab2335f93f in raise () from /lib64/libc.so.6
--------------------------------------------------------------------------


On the other hand, without the patch 2914 netsnmp_free_delegated_cache is called
several times for the same object as follows:

 1. snmp_resend_request calls agentx_got_response with NETSNMP_CALLBACK_OP_RESEND,
 2. agentx_got_response's "Unknown operation" handler calls netsnmp_free_delegated_cache,
 3. (retry) snmp_resend_request calls agentx_got_response AGAIN with NETSNMP_CALLBACK_OP_RESEND,
 4. agentx_got_response's "Unknown operation" handler calls netsnmp_free_delegated_cache
    (double free)

gdb
------------------------------------------------------
(gdb) b netsnmp_free_delegated_cache
Breakpoint 1 at 0x7f03bd437250: file agent_handler.c, line 929.
(gdb) c
Continuing.

Breakpoint 1, netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
#1  0x00007f03bd44a2df in agentx_got_response (operation=6, session=0x558981aa1540, reqid=12, pdu=0x558981be9370, magic=<optimized out>)
    at mibgroup/agentx/master.c:292
#2  0x00007f03bcbde0e6 in snmp_resend_request (slp=slp@entry=0x558981bd2e90, rp=rp@entry=0x558981be67a0, incr_retries=1) at snmp_api.c:6747
#3  0x00007f03bcbe05db in snmp_sess_timeout (sessp=sessp@entry=0x558981bd2e90) at snmp_api.c:6826
#4  0x00007f03bcbe0710 in snmp_timeout () at snmp_api.c:6660
#5  0x0000558980c5df58 in receive () at snmpd.c:1347
#6  0x0000558980c5d66e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Breakpoint 1, netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
#1  0x00007f03bd44a5ce in agentx_got_response (operation=6, session=0x558981aa1540, reqid=12, pdu=0x558981be9370, magic=0x558981be9110)
    at mibgroup/agentx/master.c:223
#2  0x00007f03bcbde0e6 in snmp_resend_request (slp=slp@entry=0x558981bd2e90, rp=rp@entry=0x558981be67a0, incr_retries=1) at snmp_api.c:6747
#3  0x00007f03bcbe05db in snmp_sess_timeout (sessp=sessp@entry=0x558981bd2e90) at snmp_api.c:6826
#4  0x00007f03bcbe0710 in snmp_timeout () at snmp_api.c:6660
#5  0x0000558980c5df58 in receive () at snmpd.c:1347
#6  0x0000558980c5d66e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Program received signal SIGABRT, Aborted.
------------------------------------------------------


I am not sure it's acceptable but attach a patch to fix the issue.

Regards,

Shogo Matsumoto
Fujitsu Ltd.


> The introduction of that code fixes another issue;
> "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
> Author: Bill Fenner <fenner@...<mailto:fenner@...>>
> Date:   Wed Dec 20 21:52:10 2017 +0000
>
>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
> agent
>
>     With the patch in 1214, the snmp_api code assumed that if magic was
>     set, it was the "struct synch-state" from snmp_client.  Of course,
>     magic belongs to the caller, and the perl library uses it differently,
>     so reaching into it is verboten.  Introduce a new callback (that
>     was already introduced in 5.8) to report this "retries exceeded"
>     state, and use it in snmp_client."
>
> I think the problem is really about shutting down the agentx connection
> when one(1) response is to late. I have
> done 2 patches (one that only write a better log message and one that
> removes the "bad" code.
> With these patches I don't get any crash. I think that 5.7.3 has this issue
> as well, but it can not be crashed with the agentofdead code
>
> Can you please try this?
>
> Regards
> Anders Wallin



_______________________________________________
Net-snmp-coders mailing list
Net...@li...<mailto:Net...@li...>
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

Re: Core dump with net-snmp-5.8

[Anders W. <wal...@gm...>]

I will check, but it will be tomorrow
I don't know how I read the code last time, I was referring to the wrong
commit where the code was introduced, it was introduced with
"commit b7b50bbac7f21a924149d03da26ff0a44b25ec60
Author: VMwareDev Randy <snm...@vm...>
Date:   Mon Jun 22 22:20:43 2015 -0400

    snmp_send callback updates

    - add new NETSNMP_CALLBACK_OP_RESEND
    - add missing calls for NETSNMP_CALLBACK_OP_SEND_FAILED

    Signed-off-by: Robert Story <rs...@fr...>"

Regards
Anders Wallin


On Thu, Apr 4, 2019 at 10:48 AM Matsumoto, Shogo <
sho...@jp...> wrote:

> Hi,
>
> The issue also occurs with the following patches.
>
>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
>
>  0001-agentx-logging-to-late-responses.patch
>  0002-agentx-do-not-shut-down-all-sessions-when-one-sessio.patch
>
>
> The issue occurs with the following patch (2914) too but I found
> the cause of this issue.
>
>  https://sourceforge.net/p/net-snmp/bugs/2914/
>  0001-BUG2914-Agent-master-needs-to-treat-resend-as-normal.patch
>
>
> With the patch 2914, netsnmp_free_delegated_cache is called
> several times for the same object as follows:
>
>  1. snmp_resend_request calls agentx_got_response with
> NETSNMP_CALLBACK_OP_RESEND,
>  2. agentx_got_response's NETSNMP_CALLBACK_OP_RESEND handler do nothing
>  3. snmp_resend_request calls agentx_got_response with
> NETSNMP_CALLBACK_OP_SEND_FAILED,
>  4. agentx_got_response's NETSNMP_CALLBACK_OP_SEND_FAILED handler calls
> netsnmp_free_delegated_cache,
>  5. snmp_sess_close calls agentx_got_response with
> NETSNMP_CALLBACK_OP_TIMED_OUT,
>  6. agentx_got_response's NETSNMP_CALLBACK_OP_TIMED_OUT handler calls
> netsnmp_free_delegated_cache
>     (double free)
>
> gdb
> --------------------------------------------------------------------------
> Breakpoint 2, snmp_resend_request (slp=slp@entry=0x564eec5df000,
> rp=rp@entry=0x564eec5eb160, incr_retries=1) at snmp_api.c:6747
> 6747                rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
> (gdb) c
> Continuing.
>
> Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f2ec0) at
> agent_handler.c:929
> 929     {
> (gdb) bt
> #0  netsnmp_free_delegated_cache (dcache=0x564eec5f2ec0) at
> agent_handler.c:929
> #1  0x00007fab254d5363 in agentx_got_response (operation=<optimized out>,
> session=0x564eec4ad560, reqid=2, pdu=0x564eec5e3050, magic=<optimized out>)
>     at mibgroup/agentx/master.c:262
> #2  0x00007fab24c6b58f in snmp_sess_timeout (sessp=sessp@entry=0x564eec5df000)
> at snmp_api.c:6813
> #3  0x00007fab24c6b710 in snmp_timeout () at snmp_api.c:6660
> #4  0x0000564eeb4c0f58 in receive () at snmpd.c:1347
> #5  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized
> out>) at snmpd.c:1126
> (gdb) c
> Continuing.
>
> Breakpoint 1, snmp_resend_request (slp=slp@entry=0x564eec5df000,
> rp=rp@entry=0x564eec5f3e50, incr_retries=1) at snmp_api.c:6735
> 6735                rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
> (gdb) c
> Continuing.
>
> Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at
> agent_handler.c:929
> 929     {
> (gdb) bt
> #0  netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at
> agent_handler.c:929
> #1  0x00007fab254d541a in agentx_got_response (operation=3,
> session=0x564eec4ad560, reqid=4, pdu=0x564eec5e54a0, magic=0x564eec5f3730)
>     at mibgroup/agentx/master.c:223
> #2  0x00007fab24c69325 in snmp_resend_request (slp=slp@entry=0x564eec5df000,
> rp=rp@entry=0x564eec5f3e50, incr_retries=1) at snmp_api.c:6735
> #3  0x00007fab24c6b5db in snmp_sess_timeout (sessp=sessp@entry=0x564eec5df000)
> at snmp_api.c:6826
> #4  0x00007fab24c6b710 in snmp_timeout () at snmp_api.c:6660
> #5  0x0000564eeb4c0f58 in receive () at snmpd.c:1347
> #6  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized
> out>) at snmpd.c:1126
> (gdb) c
> Continuing.
>
> Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at
> agent_handler.c:929
> 929     {
> (gdb) bt
> #0  netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at
> agent_handler.c:929
> #1  0x00007fab254d541a in agentx_got_response (operation=2,
> session=0x564eec4ad560, reqid=4, pdu=0x564eec5e54a0, magic=0x564eec5f3730)
>     at mibgroup/agentx/master.c:223
> #2  0x00007fab24c69586 in snmp_sess_close (sessp=0x564eec5df000) at
> snmp_api.c:1975
> #3  0x00007fab24c6afea in snmp_sess_select_info2_flags (sessp=0x0,
> numfds=0x7fff68db3694, fdset=0x7fff68db36b0, timeout=0x7fff68db36a0,
>     block=0x7fff68db369c, flags=0) at snmp_api.c:6556
> #4  0x0000564eeb4c0e95 in receive () at snmpd.c:1263
> #5  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized
> out>) at snmpd.c:1126
> (gdb) c
> Continuing.
>
> Program received signal SIGABRT, Aborted.
> 0x00007fab2335f93f in raise () from /lib64/libc.so.6
> --------------------------------------------------------------------------
>
>
> On the other hand, without the patch 2914 netsnmp_free_delegated_cache is
> called
> several times for the same object as follows:
>
>  1. snmp_resend_request calls agentx_got_response with
> NETSNMP_CALLBACK_OP_RESEND,
>  2. agentx_got_response's "Unknown operation" handler calls
> netsnmp_free_delegated_cache,
>  3. (retry) snmp_resend_request calls agentx_got_response AGAIN with
> NETSNMP_CALLBACK_OP_RESEND,
>  4. agentx_got_response's "Unknown operation" handler calls
> netsnmp_free_delegated_cache
>     (double free)
>
> gdb
> ------------------------------------------------------
> (gdb) b netsnmp_free_delegated_cache
> Breakpoint 1 at 0x7f03bd437250: file agent_handler.c, line 929.
> (gdb) c
> Continuing.
>
> Breakpoint 1, netsnmp_free_delegated_cache (dcache=0x558981be9110) at
> agent_handler.c:929
> 929     {
> (gdb) bt
> #0  netsnmp_free_delegated_cache (dcache=0x558981be9110) at
> agent_handler.c:929
> #1  0x00007f03bd44a2df in agentx_got_response (operation=6,
> session=0x558981aa1540, reqid=12, pdu=0x558981be9370, magic=<optimized out>)
>     at mibgroup/agentx/master.c:292
> #2  0x00007f03bcbde0e6 in snmp_resend_request (slp=slp@entry=0x558981bd2e90,
> rp=rp@entry=0x558981be67a0, incr_retries=1) at snmp_api.c:6747
> #3  0x00007f03bcbe05db in snmp_sess_timeout (sessp=sessp@entry=0x558981bd2e90)
> at snmp_api.c:6826
> #4  0x00007f03bcbe0710 in snmp_timeout () at snmp_api.c:6660
> #5  0x0000558980c5df58 in receive () at snmpd.c:1347
> #6  0x0000558980c5d66e in main (argc=<optimized out>, argv=<optimized
> out>) at snmpd.c:1126
> (gdb) c
> Continuing.
>
> Breakpoint 1, netsnmp_free_delegated_cache (dcache=0x558981be9110) at
> agent_handler.c:929
> 929     {
> (gdb) bt
> #0  netsnmp_free_delegated_cache (dcache=0x558981be9110) at
> agent_handler.c:929
> #1  0x00007f03bd44a5ce in agentx_got_response (operation=6,
> session=0x558981aa1540, reqid=12, pdu=0x558981be9370, magic=0x558981be9110)
>     at mibgroup/agentx/master.c:223
> #2  0x00007f03bcbde0e6 in snmp_resend_request (slp=slp@entry=0x558981bd2e90,
> rp=rp@entry=0x558981be67a0, incr_retries=1) at snmp_api.c:6747
> #3  0x00007f03bcbe05db in snmp_sess_timeout (sessp=sessp@entry=0x558981bd2e90)
> at snmp_api.c:6826
> #4  0x00007f03bcbe0710 in snmp_timeout () at snmp_api.c:6660
> #5  0x0000558980c5df58 in receive () at snmpd.c:1347
> #6  0x0000558980c5d66e in main (argc=<optimized out>, argv=<optimized
> out>) at snmpd.c:1126
> (gdb) c
> Continuing.
>
> Program received signal SIGABRT, Aborted.
> ------------------------------------------------------
>
>
> I am not sure it's acceptable but attach a patch to fix the issue.
>
> Regards,
>
> Shogo Matsumoto
> Fujitsu Ltd.
>
>
> > The introduction of that code fixes another issue;
> > "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
> > Author: Bill Fenner <fenner@...>
> > Date:   Wed Dec 20 21:52:10 2017 +0000
> >
> >     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
> > agent
> >
> >     With the patch in 1214, the snmp_api code assumed that if magic was
> >     set, it was the "struct synch-state" from snmp_client.  Of course,
> >     magic belongs to the caller, and the perl library uses it
> differently,
> >     so reaching into it is verboten.  Introduce a new callback (that
> >     was already introduced in 5.8) to report this "retries exceeded"
> >     state, and use it in snmp_client."
> >
> > I think the problem is really about shutting down the agentx connection
> > when one(1) response is to late. I have
> > done 2 patches (one that only write a better log message and one that
> > removes the "bad" code.
> > With these patches I don't get any crash. I think that 5.7.3 has this
> issue
> > as well, but it can not be crashed with the agentofdead code
> >
> > Can you please try this?
> >
> > Regards
> > Anders Wallin
>
>
>
> _______________________________________________
> Net-snmp-coders mailing list
> Net...@li...
> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>

Re: Core dump with net-snmp-5.8

[Matsumoto, S. <sho...@jp...>]

Hi,

The issue also occurs with the following patches.

    NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3

 0001-agentx-logging-to-late-responses.patch
 0002-agentx-do-not-shut-down-all-sessions-when-one-sessio.patch


The issue occurs with the following patch (2914) too but I found
the cause of this issue.

 https://sourceforge.net/p/net-snmp/bugs/2914/
 0001-BUG2914-Agent-master-needs-to-treat-resend-as-normal.patch


With the patch 2914, netsnmp_free_delegated_cache is called
several times for the same object as follows:

 1. snmp_resend_request calls agentx_got_response with NETSNMP_CALLBACK_OP_RESEND,
 2. agentx_got_response's NETSNMP_CALLBACK_OP_RESEND handler do nothing
 3. snmp_resend_request calls agentx_got_response with NETSNMP_CALLBACK_OP_SEND_FAILED, 
 4. agentx_got_response's NETSNMP_CALLBACK_OP_SEND_FAILED handler calls netsnmp_free_delegated_cache,
 5. snmp_sess_close calls agentx_got_response with NETSNMP_CALLBACK_OP_TIMED_OUT,
 6. agentx_got_response's NETSNMP_CALLBACK_OP_TIMED_OUT handler calls netsnmp_free_delegated_cache
    (double free)

gdb
--------------------------------------------------------------------------
Breakpoint 2, snmp_resend_request (slp=slp@entry=0x564eec5df000, rp=rp@entry=0x564eec5eb160, incr_retries=1) at snmp_api.c:6747
6747                rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
(gdb) c
Continuing.

Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f2ec0) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x564eec5f2ec0) at agent_handler.c:929
#1  0x00007fab254d5363 in agentx_got_response (operation=<optimized out>, session=0x564eec4ad560, reqid=2, pdu=0x564eec5e3050, magic=<optimized out>)
    at mibgroup/agentx/master.c:262
#2  0x00007fab24c6b58f in snmp_sess_timeout (sessp=sessp@entry=0x564eec5df000) at snmp_api.c:6813
#3  0x00007fab24c6b710 in snmp_timeout () at snmp_api.c:6660
#4  0x0000564eeb4c0f58 in receive () at snmpd.c:1347
#5  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Breakpoint 1, snmp_resend_request (slp=slp@entry=0x564eec5df000, rp=rp@entry=0x564eec5f3e50, incr_retries=1) at snmp_api.c:6735
6735                rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
(gdb) c
Continuing.

Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
#1  0x00007fab254d541a in agentx_got_response (operation=3, session=0x564eec4ad560, reqid=4, pdu=0x564eec5e54a0, magic=0x564eec5f3730)
    at mibgroup/agentx/master.c:223
#2  0x00007fab24c69325 in snmp_resend_request (slp=slp@entry=0x564eec5df000, rp=rp@entry=0x564eec5f3e50, incr_retries=1) at snmp_api.c:6735
#3  0x00007fab24c6b5db in snmp_sess_timeout (sessp=sessp@entry=0x564eec5df000) at snmp_api.c:6826
#4  0x00007fab24c6b710 in snmp_timeout () at snmp_api.c:6660
#5  0x0000564eeb4c0f58 in receive () at snmpd.c:1347
#6  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
#1  0x00007fab254d541a in agentx_got_response (operation=2, session=0x564eec4ad560, reqid=4, pdu=0x564eec5e54a0, magic=0x564eec5f3730)
    at mibgroup/agentx/master.c:223
#2  0x00007fab24c69586 in snmp_sess_close (sessp=0x564eec5df000) at snmp_api.c:1975
#3  0x00007fab24c6afea in snmp_sess_select_info2_flags (sessp=0x0, numfds=0x7fff68db3694, fdset=0x7fff68db36b0, timeout=0x7fff68db36a0,
    block=0x7fff68db369c, flags=0) at snmp_api.c:6556
#4  0x0000564eeb4c0e95 in receive () at snmpd.c:1263
#5  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Program received signal SIGABRT, Aborted.
0x00007fab2335f93f in raise () from /lib64/libc.so.6
--------------------------------------------------------------------------


On the other hand, without the patch 2914 netsnmp_free_delegated_cache is called
several times for the same object as follows:

 1. snmp_resend_request calls agentx_got_response with NETSNMP_CALLBACK_OP_RESEND,
 2. agentx_got_response's "Unknown operation" handler calls netsnmp_free_delegated_cache,
 3. (retry) snmp_resend_request calls agentx_got_response AGAIN with NETSNMP_CALLBACK_OP_RESEND,
 4. agentx_got_response's "Unknown operation" handler calls netsnmp_free_delegated_cache
    (double free)

gdb
------------------------------------------------------
(gdb) b netsnmp_free_delegated_cache
Breakpoint 1 at 0x7f03bd437250: file agent_handler.c, line 929.
(gdb) c
Continuing.

Breakpoint 1, netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
#1  0x00007f03bd44a2df in agentx_got_response (operation=6, session=0x558981aa1540, reqid=12, pdu=0x558981be9370, magic=<optimized out>)
    at mibgroup/agentx/master.c:292
#2  0x00007f03bcbde0e6 in snmp_resend_request (slp=slp@entry=0x558981bd2e90, rp=rp@entry=0x558981be67a0, incr_retries=1) at snmp_api.c:6747
#3  0x00007f03bcbe05db in snmp_sess_timeout (sessp=sessp@entry=0x558981bd2e90) at snmp_api.c:6826
#4  0x00007f03bcbe0710 in snmp_timeout () at snmp_api.c:6660
#5  0x0000558980c5df58 in receive () at snmpd.c:1347
#6  0x0000558980c5d66e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Breakpoint 1, netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
#1  0x00007f03bd44a5ce in agentx_got_response (operation=6, session=0x558981aa1540, reqid=12, pdu=0x558981be9370, magic=0x558981be9110)
    at mibgroup/agentx/master.c:223
#2  0x00007f03bcbde0e6 in snmp_resend_request (slp=slp@entry=0x558981bd2e90, rp=rp@entry=0x558981be67a0, incr_retries=1) at snmp_api.c:6747
#3  0x00007f03bcbe05db in snmp_sess_timeout (sessp=sessp@entry=0x558981bd2e90) at snmp_api.c:6826
#4  0x00007f03bcbe0710 in snmp_timeout () at snmp_api.c:6660
#5  0x0000558980c5df58 in receive () at snmpd.c:1347
#6  0x0000558980c5d66e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Program received signal SIGABRT, Aborted.
------------------------------------------------------


I am not sure it's acceptable but attach a patch to fix the issue.

Regards,

Shogo Matsumoto
Fujitsu Ltd.


> The introduction of that code fixes another issue;
> "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
> Author: Bill Fenner <fenner@...>
> Date:   Wed Dec 20 21:52:10 2017 +0000
>
>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
> agent
>
>     With the patch in 1214, the snmp_api code assumed that if magic was
>     set, it was the "struct synch-state" from snmp_client.  Of course,
>     magic belongs to the caller, and the perl library uses it differently,
>     so reaching into it is verboten.  Introduce a new callback (that
>     was already introduced in 5.8) to report this "retries exceeded"
>     state, and use it in snmp_client."
>
> I think the problem is really about shutting down the agentx connection
> when one(1) response is to late. I have
> done 2 patches (one that only write a better log message and one that
> removes the "bad" code.
> With these patches I don't get any crash. I think that 5.7.3 has this issue
> as well, but it can not be crashed with the agentofdead code
>
> Can you please try this?
>
> Regards
> Anders Wallin

Re: Core dump with net-snmp-5.8

[Anders W. <wal...@gm...>]

The introduction of that code fixes another issue;
"commit 56c30b11f3616ea4f0c38a21e08e78f050096020
Author: Bill Fenner <fe...@gm...>
Date:   Wed Dec 20 21:52:10 2017 +0000

    NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
agent

    With the patch in 1214, the snmp_api code assumed that if magic was
    set, it was the "struct synch-state" from snmp_client.  Of course,
    magic belongs to the caller, and the perl library uses it differently,
    so reaching into it is verboten.  Introduce a new callback (that
    was already introduced in 5.8) to report this "retries exceeded"
    state, and use it in snmp_client."

I think the problem is really about shutting down the agentx connection
when one(1) response is to late. I have
done 2 patches (one that only write a better log message and one that
removes the "bad" code.
With these patches I don't get any crash. I think that 5.7.3 has this issue
as well, but it can not be crashed with the agentofdead code

Can you please try this?

Regards
Anders Wallin


On Wed, Apr 3, 2019 at 12:35 PM Josef Ridky <jr...@re...> wrote:

> Hi,
>
> I have compared net-snmp-5.7.3 and net-snmp-5.8 and I have found, that
> following callbacks in snmplib/snmp_api.c causes the core dump issue:
>
> --- old/snmplib/snmp_api.c      2019-04-03 12:13:55.126769866 +0200
> +++ new/snmplib/snmp_api.c      2019-04-03 12:15:18.353420790 +0200
> @@ -6731,9 +6731,9 @@ snmp_resend_request(struct session_list
>          sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
>          sp->s_errno = errno;
>          snmp_set_detail(strerror(errno));
> -        if (rp->callback)
> +/*        if (rp->callback)
>              rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>          return -1;
>      } else {
>          netsnmp_get_monotonic_clock(&now);
> @@ -6743,9 +6743,9 @@ snmp_resend_request(struct session_list
>          tv.tv_sec += tv.tv_usec / 1000000L;
>          tv.tv_usec %= 1000000L;
>          rp->expireM = tv;
> -        if (rp->callback)
> +/*        if (rp->callback)
>              rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
> -                         rp->pdu->reqid, rp->pdu, rp->cb_data);
> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
>      }
>      return 0;
>  }
>
> Without them, all works as expected.
>
> Josef Ridky
> Software Engineer
> Core Services Team
> Red Hat Czech, s.r.o.
>
> ----- Original Message -----
> | From: "Anders Wallin" <wal...@gm...>
> | To: "Josef Ridky" <jr...@re...>
> | Cc: "net-snmp-coders" <net...@li...>
> | Sent: Tuesday, April 2, 2019 6:27:54 PM
> | Subject: Re: Core dump with net-snmp-5.8
> |
> | Hi Josef,
> | I can reproduce the issue using the master branch, I will take a look at
> it
> | later tonight or tomorrow
> |
> | Regards
> | Anders Wallin
> |
> |
> | On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...> wrote:
> |
> | > Hi,
> | >
> | > thanks for your patch. Unfortunately, even when I have applied it, it
> | > still ends with core dump due of 'double free or corruption (fasttop)'
> | >
> | > When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
> | >
> | > agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
> | > snmp_agent: delegate session == 0x56207e165240
> | > snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
> | > agentx/master: callback resend
> | > agentx/master: callback resend
> | > agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
> | > agentx/master: close 0x56207dfd5400, -1
> | > snmp_agent: removed 40 delegated request(s) for session 0x56207dfce490
> | > snmp_agent: processing delegated request, asp = 0x56207e165240
> | > snmp_agent: canceling next walk for asp 0x56207e165240
> | > snmp_agent: REMOVE session == 0x56207e165240
> | > snmp_agent: agent_session 0x56207e165240 released
> | > snmp_agent: processing delegated request, asp = 0x56207e1041a0
> | > snmp_agent: canceling next walk for asp 0x56207e1041a0
> | > snmp_agent: REMOVE session == 0x56207e1041a0
> | > snmp_agent: agent_session 0x56207e1041a0 released
> | > snmp_agent: processing delegated request, asp = 0x56207e1656c0
> | > snmp_agent: canceling next walk for asp 0x56207e1656c0
> | > snmp_agent: REMOVE session == 0x56207e1656c0
> | > snmp_agent: agent_session 0x56207e1656c0 released
> | > snmp_agent: processing delegated request, asp = 0x56207e11af40
> | > snmp_agent: canceling next walk for asp 0x56207e11af40
> | > snmp_agent: REMOVE session == 0x56207e11af40
> | > snmp_agent: agent_session 0x56207e11af40 released
> | > snmp_agent: processing delegated request, asp = 0x56207e118f00
> | > snmp_agent: canceling next walk for asp 0x56207e118f00
> | > snmp_agent: REMOVE session == 0x56207e118f00
> | > snmp_agent: agent_session 0x56207e118f00 released
> | > snmp_agent: processing delegated request, asp = 0x56207e11b540
> | > snmp_agent: canceling next walk for asp 0x56207e11b540
> | > snmp_agent: REMOVE session == 0x56207e11b540
> | > snmp_agent: agent_session 0x56207e11b540 released
> | > snmp_agent: processing delegated request, asp = 0x56207e11bd00
> | > snmp_agent: canceling next walk for asp 0x56207e11bd00
> | > snmp_agent: REMOVE session == 0x56207e11bd00
> | > snmp_agent: agent_session 0x56207e11bd00 released
> | > agentx/master: Continue removing delegated subsession reqests
> | > agentx/master: close transport
> | > snmp_agent: REMOVE session == 0x56207dfd5400
> | > agentx/master: response too late on session 0x56207dfd5400
> | > agentx/master: response too late on session 0x56207dfd5400
> | > double free or corruption (fasttop)
> | > Aborted (core dumped)
> | >
> | >
> | > What's interesting, when I run it with -DALL it pass (at least for
> several
> | > rounds).
> | > It looks like some strange race condition.
> | >
> | > Regards
> | >
> | > Josef Ridky
> | > Software Engineer
> | > Core Services Team
> | > Red Hat Czech, s.r.o.
> | >
> | > ----- Original Message -----
> | > | From: "Anders Wallin" <wal...@gm...>
> | > | To: "Josef Ridky" <jr...@re...>
> | > | Cc: "net-snmp-coders" <net...@li...>
> | > | Sent: Tuesday, April 2, 2019 1:46:40 PM
> | > | Subject: Re: Core dump with net-snmp-5.8
> | > |
> | > | Hi Josef,
> | > |
> | > | I think it's the same issue as
> | > https://sourceforge.net/p/net-snmp/bugs/2914/
> | > | (where I also posted the solution)
> | > | Regards
> | > | Anders Wallin
> | > |
> | > |
> | > | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...>
> wrote:
> | > |
> | > | > Hi,
> | > | >
> | > | > recently, I have hit to an issue in net-snmp-5.8, that is
> connected to
> | > the
> | > | > bug report [1].
> | > | >
> | > | > When I tried to run agentofdeath test from [1], snmpd daemon will
> crash
> | > | > with malloc(): smallbin double linked list corrupted or double
> free()
> | > issue
> | > | > and dumps core (see bellow).
> | > | > From log file, I can identified one issue with "Unknown operation".
> | > | >
> | > | > This issue is in the agentx_got_response function
> | > | > (agent/mibgroup/agentx/master.c). There isn't implemented action
> for
> | > | > NETSNMP_CALLBACK_OP_RESEND (defined in
> | > | > include/net-snmp/library/snmp_api.h).
> | > | > As result "Unknown operation 6 in agentx_got_response" is shown in
> log
> | > | > file.
> | > | >
> | > | > /var/log/messages
> | > | > -------------------------------
> | > | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in
> | > | > agentx_got_response
> | > | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in
> | > | > agentx_got_response
> | > | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin double
> | > linked
> | > | > list corrupted
> | > | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core Dump
> (PID
> | > | > 13652/UID 0).
> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main process
> | > exited,
> | > | > code=dumped, status=6/ABRT
> | > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed with
> result
> | > | > 'core-dump'.
> | > | > -------------------------------
> | > | >
> | > | > The "Unknown operation" callback is caused by newly added piece of
> | > code in
> | > | > snmplib/snmp_api.c:
> | > | >
> | > | >  static int
> | > | >  snmp_resend_request(struct session_list *slp, netsnmp_request_list
> | > *rp,
> | > | >  int incr_retries)
> | > | >  {
> | > | >
> | > | > ...
> | > | >
> | > | >          tv.tv_sec += tv.tv_usec / 1000000L;
> | > | >          tv.tv_usec %= 1000000L;
> | > | >          rp->expireM = tv;
> | > | > +        if (rp->callback)
> | > | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
> | > | > +                         rp->pdu->reqid, rp->pdu, rp->cb_data);
> | > | >      }
> | > | >      return 0;
> | > | >  }
> | > | >
> | > | >
> | > | > When I tried to remove it, it just stop complaining about
> operation 6,
> | > but
> | > | > the core dump is still present.
> | > | >
> | > | > May I ask you for help with this issue? Do you have any idea, what
> | > causing
> | > | > this issue in 5.8 and how to fix it?
> | > | > I know, that Jan Safranek has fixed this for 5.7 by commit [2],
> but it
> | > | > looks like something other has changed and this issue is current
> again.
> | > | >
> | > | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
> | > | > [2]
> | > | >
> | >
> https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
> | > | >
> | > | > Regards
> | > | >
> | > | > Josef Ridky
> | > | > Software Engineer
> | > | > Core Services Team
> | > | > Red Hat Czech, s.r.o.
> | > | >
> | > | >
> | > | >
> | > | > _______________________________________________
> | > | > Net-snmp-coders mailing list
> | > | > Net...@li...
> | > | > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
> | > | >
> | > |
> | >
> |
>

Re: Core dump with net-snmp-5.8

[Josef R. <jr...@re...>]

Hi,

I have compared net-snmp-5.7.3 and net-snmp-5.8 and I have found, that following callbacks in snmplib/snmp_api.c causes the core dump issue:

--- old/snmplib/snmp_api.c	2019-04-03 12:13:55.126769866 +0200
+++ new/snmplib/snmp_api.c	2019-04-03 12:15:18.353420790 +0200
@@ -6731,9 +6731,9 @@ snmp_resend_request(struct session_list
         sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
         sp->s_errno = errno;
         snmp_set_detail(strerror(errno));
-        if (rp->callback)
+/*        if (rp->callback)
             rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
-                         rp->pdu->reqid, rp->pdu, rp->cb_data);
+                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
         return -1;
     } else {
         netsnmp_get_monotonic_clock(&now);
@@ -6743,9 +6743,9 @@ snmp_resend_request(struct session_list
         tv.tv_sec += tv.tv_usec / 1000000L;
         tv.tv_usec %= 1000000L;
         rp->expireM = tv;
-        if (rp->callback)
+/*        if (rp->callback)
             rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
-                         rp->pdu->reqid, rp->pdu, rp->cb_data);
+                         rp->pdu->reqid, rp->pdu, rp->cb_data);*/
     }
     return 0;
 }

Without them, all works as expected.

Josef Ridky
Software Engineer
Core Services Team
Red Hat Czech, s.r.o.

----- Original Message -----
| From: "Anders Wallin" <wal...@gm...>
| To: "Josef Ridky" <jr...@re...>
| Cc: "net-snmp-coders" <net...@li...>
| Sent: Tuesday, April 2, 2019 6:27:54 PM
| Subject: Re: Core dump with net-snmp-5.8
| 
| Hi Josef,
| I can reproduce the issue using the master branch, I will take a look at it
| later tonight or tomorrow
| 
| Regards
| Anders Wallin
| 
| 
| On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...> wrote:
| 
| > Hi,
| >
| > thanks for your patch. Unfortunately, even when I have applied it, it
| > still ends with core dump due of 'double free or corruption (fasttop)'
| >
| > When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
| >
| > agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
| > snmp_agent: delegate session == 0x56207e165240
| > snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
| > agentx/master: callback resend
| > agentx/master: callback resend
| > agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
| > agentx/master: close 0x56207dfd5400, -1
| > snmp_agent: removed 40 delegated request(s) for session 0x56207dfce490
| > snmp_agent: processing delegated request, asp = 0x56207e165240
| > snmp_agent: canceling next walk for asp 0x56207e165240
| > snmp_agent: REMOVE session == 0x56207e165240
| > snmp_agent: agent_session 0x56207e165240 released
| > snmp_agent: processing delegated request, asp = 0x56207e1041a0
| > snmp_agent: canceling next walk for asp 0x56207e1041a0
| > snmp_agent: REMOVE session == 0x56207e1041a0
| > snmp_agent: agent_session 0x56207e1041a0 released
| > snmp_agent: processing delegated request, asp = 0x56207e1656c0
| > snmp_agent: canceling next walk for asp 0x56207e1656c0
| > snmp_agent: REMOVE session == 0x56207e1656c0
| > snmp_agent: agent_session 0x56207e1656c0 released
| > snmp_agent: processing delegated request, asp = 0x56207e11af40
| > snmp_agent: canceling next walk for asp 0x56207e11af40
| > snmp_agent: REMOVE session == 0x56207e11af40
| > snmp_agent: agent_session 0x56207e11af40 released
| > snmp_agent: processing delegated request, asp = 0x56207e118f00
| > snmp_agent: canceling next walk for asp 0x56207e118f00
| > snmp_agent: REMOVE session == 0x56207e118f00
| > snmp_agent: agent_session 0x56207e118f00 released
| > snmp_agent: processing delegated request, asp = 0x56207e11b540
| > snmp_agent: canceling next walk for asp 0x56207e11b540
| > snmp_agent: REMOVE session == 0x56207e11b540
| > snmp_agent: agent_session 0x56207e11b540 released
| > snmp_agent: processing delegated request, asp = 0x56207e11bd00
| > snmp_agent: canceling next walk for asp 0x56207e11bd00
| > snmp_agent: REMOVE session == 0x56207e11bd00
| > snmp_agent: agent_session 0x56207e11bd00 released
| > agentx/master: Continue removing delegated subsession reqests
| > agentx/master: close transport
| > snmp_agent: REMOVE session == 0x56207dfd5400
| > agentx/master: response too late on session 0x56207dfd5400
| > agentx/master: response too late on session 0x56207dfd5400
| > double free or corruption (fasttop)
| > Aborted (core dumped)
| >
| >
| > What's interesting, when I run it with -DALL it pass (at least for several
| > rounds).
| > It looks like some strange race condition.
| >
| > Regards
| >
| > Josef Ridky
| > Software Engineer
| > Core Services Team
| > Red Hat Czech, s.r.o.
| >
| > ----- Original Message -----
| > | From: "Anders Wallin" <wal...@gm...>
| > | To: "Josef Ridky" <jr...@re...>
| > | Cc: "net-snmp-coders" <net...@li...>
| > | Sent: Tuesday, April 2, 2019 1:46:40 PM
| > | Subject: Re: Core dump with net-snmp-5.8
| > |
| > | Hi Josef,
| > |
| > | I think it's the same issue as
| > https://sourceforge.net/p/net-snmp/bugs/2914/
| > | (where I also posted the solution)
| > | Regards
| > | Anders Wallin
| > |
| > |
| > | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...> wrote:
| > |
| > | > Hi,
| > | >
| > | > recently, I have hit to an issue in net-snmp-5.8, that is connected to
| > the
| > | > bug report [1].
| > | >
| > | > When I tried to run agentofdeath test from [1], snmpd daemon will crash
| > | > with malloc(): smallbin double linked list corrupted or double free()
| > issue
| > | > and dumps core (see bellow).
| > | > From log file, I can identified one issue with "Unknown operation".
| > | >
| > | > This issue is in the agentx_got_response function
| > | > (agent/mibgroup/agentx/master.c). There isn't implemented action for
| > | > NETSNMP_CALLBACK_OP_RESEND (defined in
| > | > include/net-snmp/library/snmp_api.h).
| > | > As result "Unknown operation 6 in agentx_got_response" is shown in log
| > | > file.
| > | >
| > | > /var/log/messages
| > | > -------------------------------
| > | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in
| > | > agentx_got_response
| > | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in
| > | > agentx_got_response
| > | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin double
| > linked
| > | > list corrupted
| > | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core Dump (PID
| > | > 13652/UID 0).
| > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main process
| > exited,
| > | > code=dumped, status=6/ABRT
| > | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed with result
| > | > 'core-dump'.
| > | > -------------------------------
| > | >
| > | > The "Unknown operation" callback is caused by newly added piece of
| > code in
| > | > snmplib/snmp_api.c:
| > | >
| > | >  static int
| > | >  snmp_resend_request(struct session_list *slp, netsnmp_request_list
| > *rp,
| > | >  int incr_retries)
| > | >  {
| > | >
| > | > ...
| > | >
| > | >          tv.tv_sec += tv.tv_usec / 1000000L;
| > | >          tv.tv_usec %= 1000000L;
| > | >          rp->expireM = tv;
| > | > +        if (rp->callback)
| > | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
| > | > +                         rp->pdu->reqid, rp->pdu, rp->cb_data);
| > | >      }
| > | >      return 0;
| > | >  }
| > | >
| > | >
| > | > When I tried to remove it, it just stop complaining about operation 6,
| > but
| > | > the core dump is still present.
| > | >
| > | > May I ask you for help with this issue? Do you have any idea, what
| > causing
| > | > this issue in 5.8 and how to fix it?
| > | > I know, that Jan Safranek has fixed this for 5.7 by commit [2], but it
| > | > looks like something other has changed and this issue is current again.
| > | >
| > | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
| > | > [2]
| > | >
| > https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
| > | >
| > | > Regards
| > | >
| > | > Josef Ridky
| > | > Software Engineer
| > | > Core Services Team
| > | > Red Hat Czech, s.r.o.
| > | >
| > | >
| > | >
| > | > _______________________________________________
| > | > Net-snmp-coders mailing list
| > | > Net...@li...
| > | > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
| > | >
| > |
| >
| 

Re: Core dump with net-snmp-5.8

[Anders W. <wal...@gm...>]

Hi Josef,
I can reproduce the issue using the master branch, I will take a look at it
later tonight or tomorrow

Regards
Anders Wallin


On Tue, Apr 2, 2019 at 3:42 PM Josef Ridky <jr...@re...> wrote:

> Hi,
>
> thanks for your patch. Unfortunately, even when I have applied it, it
> still ends with core dump due of 'double free or corruption (fasttop)'
>
> When I run snmpd with -Dsnmp_agent,agentx/master it ends with:
>
> agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
> snmp_agent: delegate session == 0x56207e165240
> snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
> agentx/master: callback resend
> agentx/master: callback resend
> agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
> agentx/master: close 0x56207dfd5400, -1
> snmp_agent: removed 40 delegated request(s) for session 0x56207dfce490
> snmp_agent: processing delegated request, asp = 0x56207e165240
> snmp_agent: canceling next walk for asp 0x56207e165240
> snmp_agent: REMOVE session == 0x56207e165240
> snmp_agent: agent_session 0x56207e165240 released
> snmp_agent: processing delegated request, asp = 0x56207e1041a0
> snmp_agent: canceling next walk for asp 0x56207e1041a0
> snmp_agent: REMOVE session == 0x56207e1041a0
> snmp_agent: agent_session 0x56207e1041a0 released
> snmp_agent: processing delegated request, asp = 0x56207e1656c0
> snmp_agent: canceling next walk for asp 0x56207e1656c0
> snmp_agent: REMOVE session == 0x56207e1656c0
> snmp_agent: agent_session 0x56207e1656c0 released
> snmp_agent: processing delegated request, asp = 0x56207e11af40
> snmp_agent: canceling next walk for asp 0x56207e11af40
> snmp_agent: REMOVE session == 0x56207e11af40
> snmp_agent: agent_session 0x56207e11af40 released
> snmp_agent: processing delegated request, asp = 0x56207e118f00
> snmp_agent: canceling next walk for asp 0x56207e118f00
> snmp_agent: REMOVE session == 0x56207e118f00
> snmp_agent: agent_session 0x56207e118f00 released
> snmp_agent: processing delegated request, asp = 0x56207e11b540
> snmp_agent: canceling next walk for asp 0x56207e11b540
> snmp_agent: REMOVE session == 0x56207e11b540
> snmp_agent: agent_session 0x56207e11b540 released
> snmp_agent: processing delegated request, asp = 0x56207e11bd00
> snmp_agent: canceling next walk for asp 0x56207e11bd00
> snmp_agent: REMOVE session == 0x56207e11bd00
> snmp_agent: agent_session 0x56207e11bd00 released
> agentx/master: Continue removing delegated subsession reqests
> agentx/master: close transport
> snmp_agent: REMOVE session == 0x56207dfd5400
> agentx/master: response too late on session 0x56207dfd5400
> agentx/master: response too late on session 0x56207dfd5400
> double free or corruption (fasttop)
> Aborted (core dumped)
>
>
> What's interesting, when I run it with -DALL it pass (at least for several
> rounds).
> It looks like some strange race condition.
>
> Regards
>
> Josef Ridky
> Software Engineer
> Core Services Team
> Red Hat Czech, s.r.o.
>
> ----- Original Message -----
> | From: "Anders Wallin" <wal...@gm...>
> | To: "Josef Ridky" <jr...@re...>
> | Cc: "net-snmp-coders" <net...@li...>
> | Sent: Tuesday, April 2, 2019 1:46:40 PM
> | Subject: Re: Core dump with net-snmp-5.8
> |
> | Hi Josef,
> |
> | I think it's the same issue as
> https://sourceforge.net/p/net-snmp/bugs/2914/
> | (where I also posted the solution)
> | Regards
> | Anders Wallin
> |
> |
> | On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...> wrote:
> |
> | > Hi,
> | >
> | > recently, I have hit to an issue in net-snmp-5.8, that is connected to
> the
> | > bug report [1].
> | >
> | > When I tried to run agentofdeath test from [1], snmpd daemon will crash
> | > with malloc(): smallbin double linked list corrupted or double free()
> issue
> | > and dumps core (see bellow).
> | > From log file, I can identified one issue with "Unknown operation".
> | >
> | > This issue is in the agentx_got_response function
> | > (agent/mibgroup/agentx/master.c). There isn't implemented action for
> | > NETSNMP_CALLBACK_OP_RESEND (defined in
> | > include/net-snmp/library/snmp_api.h).
> | > As result "Unknown operation 6 in agentx_got_response" is shown in log
> | > file.
> | >
> | > /var/log/messages
> | > -------------------------------
> | > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in
> | > agentx_got_response
> | > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in
> | > agentx_got_response
> | > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin double
> linked
> | > list corrupted
> | > Mar 28 06:52:43 localhost systemd[1]: Started Process Core Dump (PID
> | > 13652/UID 0).
> | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main process
> exited,
> | > code=dumped, status=6/ABRT
> | > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed with result
> | > 'core-dump'.
> | > -------------------------------
> | >
> | > The "Unknown operation" callback is caused by newly added piece of
> code in
> | > snmplib/snmp_api.c:
> | >
> | >  static int
> | >  snmp_resend_request(struct session_list *slp, netsnmp_request_list
> *rp,
> | >  int incr_retries)
> | >  {
> | >
> | > ...
> | >
> | >          tv.tv_sec += tv.tv_usec / 1000000L;
> | >          tv.tv_usec %= 1000000L;
> | >          rp->expireM = tv;
> | > +        if (rp->callback)
> | > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
> | > +                         rp->pdu->reqid, rp->pdu, rp->cb_data);
> | >      }
> | >      return 0;
> | >  }
> | >
> | >
> | > When I tried to remove it, it just stop complaining about operation 6,
> but
> | > the core dump is still present.
> | >
> | > May I ask you for help with this issue? Do you have any idea, what
> causing
> | > this issue in 5.8 and how to fix it?
> | > I know, that Jan Safranek has fixed this for 5.7 by commit [2], but it
> | > looks like something other has changed and this issue is current again.
> | >
> | > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
> | > [2]
> | >
> https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
> | >
> | > Regards
> | >
> | > Josef Ridky
> | > Software Engineer
> | > Core Services Team
> | > Red Hat Czech, s.r.o.
> | >
> | >
> | >
> | > _______________________________________________
> | > Net-snmp-coders mailing list
> | > Net...@li...
> | > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
> | >
> |
>

Re: Core dump with net-snmp-5.8

[Josef R. <jr...@re...>]

Hi,

thanks for your patch. Unfortunately, even when I have applied it, it still ends with core dump due of 'double free or corruption (fasttop)'

When I run snmpd with -Dsnmp_agent,agentx/master it ends with:

agentx/master: sending pdu (req=0x1d4,trans=0x1d3,sess=0x5)
snmp_agent: delegate session == 0x56207e165240
snmp_agent: end of handle_snmp_packet, asp = 0x56207e165240
agentx/master: callback resend
agentx/master: callback resend
agentx/master: timeout on session 0x56207dfd5400 req=0x1c9
agentx/master: close 0x56207dfd5400, -1
snmp_agent: removed 40 delegated request(s) for session 0x56207dfce490
snmp_agent: processing delegated request, asp = 0x56207e165240
snmp_agent: canceling next walk for asp 0x56207e165240
snmp_agent: REMOVE session == 0x56207e165240
snmp_agent: agent_session 0x56207e165240 released
snmp_agent: processing delegated request, asp = 0x56207e1041a0
snmp_agent: canceling next walk for asp 0x56207e1041a0
snmp_agent: REMOVE session == 0x56207e1041a0
snmp_agent: agent_session 0x56207e1041a0 released
snmp_agent: processing delegated request, asp = 0x56207e1656c0
snmp_agent: canceling next walk for asp 0x56207e1656c0
snmp_agent: REMOVE session == 0x56207e1656c0
snmp_agent: agent_session 0x56207e1656c0 released
snmp_agent: processing delegated request, asp = 0x56207e11af40
snmp_agent: canceling next walk for asp 0x56207e11af40
snmp_agent: REMOVE session == 0x56207e11af40
snmp_agent: agent_session 0x56207e11af40 released
snmp_agent: processing delegated request, asp = 0x56207e118f00
snmp_agent: canceling next walk for asp 0x56207e118f00
snmp_agent: REMOVE session == 0x56207e118f00
snmp_agent: agent_session 0x56207e118f00 released
snmp_agent: processing delegated request, asp = 0x56207e11b540
snmp_agent: canceling next walk for asp 0x56207e11b540
snmp_agent: REMOVE session == 0x56207e11b540
snmp_agent: agent_session 0x56207e11b540 released
snmp_agent: processing delegated request, asp = 0x56207e11bd00
snmp_agent: canceling next walk for asp 0x56207e11bd00
snmp_agent: REMOVE session == 0x56207e11bd00
snmp_agent: agent_session 0x56207e11bd00 released
agentx/master: Continue removing delegated subsession reqests
agentx/master: close transport
snmp_agent: REMOVE session == 0x56207dfd5400
agentx/master: response too late on session 0x56207dfd5400
agentx/master: response too late on session 0x56207dfd5400
double free or corruption (fasttop)
Aborted (core dumped)


What's interesting, when I run it with -DALL it pass (at least for several rounds).
It looks like some strange race condition.

Regards

Josef Ridky
Software Engineer
Core Services Team
Red Hat Czech, s.r.o.

----- Original Message -----
| From: "Anders Wallin" <wal...@gm...>
| To: "Josef Ridky" <jr...@re...>
| Cc: "net-snmp-coders" <net...@li...>
| Sent: Tuesday, April 2, 2019 1:46:40 PM
| Subject: Re: Core dump with net-snmp-5.8
| 
| Hi Josef,
| 
| I think it's the same issue as https://sourceforge.net/p/net-snmp/bugs/2914/
| (where I also posted the solution)
| Regards
| Anders Wallin
| 
| 
| On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...> wrote:
| 
| > Hi,
| >
| > recently, I have hit to an issue in net-snmp-5.8, that is connected to the
| > bug report [1].
| >
| > When I tried to run agentofdeath test from [1], snmpd daemon will crash
| > with malloc(): smallbin double linked list corrupted or double free() issue
| > and dumps core (see bellow).
| > From log file, I can identified one issue with "Unknown operation".
| >
| > This issue is in the agentx_got_response function
| > (agent/mibgroup/agentx/master.c). There isn't implemented action for
| > NETSNMP_CALLBACK_OP_RESEND (defined in
| > include/net-snmp/library/snmp_api.h).
| > As result "Unknown operation 6 in agentx_got_response" is shown in log
| > file.
| >
| > /var/log/messages
| > -------------------------------
| > Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in
| > agentx_got_response
| > Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in
| > agentx_got_response
| > Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin double linked
| > list corrupted
| > Mar 28 06:52:43 localhost systemd[1]: Started Process Core Dump (PID
| > 13652/UID 0).
| > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main process exited,
| > code=dumped, status=6/ABRT
| > Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed with result
| > 'core-dump'.
| > -------------------------------
| >
| > The "Unknown operation" callback is caused by newly added piece of code in
| > snmplib/snmp_api.c:
| >
| >  static int
| >  snmp_resend_request(struct session_list *slp, netsnmp_request_list *rp,
| >  int incr_retries)
| >  {
| >
| > ...
| >
| >          tv.tv_sec += tv.tv_usec / 1000000L;
| >          tv.tv_usec %= 1000000L;
| >          rp->expireM = tv;
| > +        if (rp->callback)
| > +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
| > +                         rp->pdu->reqid, rp->pdu, rp->cb_data);
| >      }
| >      return 0;
| >  }
| >
| >
| > When I tried to remove it, it just stop complaining about operation 6, but
| > the core dump is still present.
| >
| > May I ask you for help with this issue? Do you have any idea, what causing
| > this issue in 5.8 and how to fix it?
| > I know, that Jan Safranek has fixed this for 5.7 by commit [2], but it
| > looks like something other has changed and this issue is current again.
| >
| > [1] https://sourceforge.net/p/net-snmp/bugs/2411/
| > [2]
| > https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
| >
| > Regards
| >
| > Josef Ridky
| > Software Engineer
| > Core Services Team
| > Red Hat Czech, s.r.o.
| >
| >
| >
| > _______________________________________________
| > Net-snmp-coders mailing list
| > Net...@li...
| > https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
| >
| 

Re: Core dump with net-snmp-5.8

[Anders W. <wal...@gm...>]

Hi Josef,

I think it's the same issue as https://sourceforge.net/p/net-snmp/bugs/2914/
(where I also posted the solution)
Regards
Anders Wallin


On Tue, Apr 2, 2019 at 12:43 PM Josef Ridky <jr...@re...> wrote:

> Hi,
>
> recently, I have hit to an issue in net-snmp-5.8, that is connected to the
> bug report [1].
>
> When I tried to run agentofdeath test from [1], snmpd daemon will crash
> with malloc(): smallbin double linked list corrupted or double free() issue
> and dumps core (see bellow).
> From log file, I can identified one issue with "Unknown operation".
>
> This issue is in the agentx_got_response function
> (agent/mibgroup/agentx/master.c). There isn't implemented action for
> NETSNMP_CALLBACK_OP_RESEND (defined in include/net-snmp/library/snmp_api.h).
> As result "Unknown operation 6 in agentx_got_response" is shown in log
> file.
>
> /var/log/messages
> -------------------------------
> Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in
> agentx_got_response
> Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in
> agentx_got_response
> Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin double linked
> list corrupted
> Mar 28 06:52:43 localhost systemd[1]: Started Process Core Dump (PID
> 13652/UID 0).
> Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main process exited,
> code=dumped, status=6/ABRT
> Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed with result
> 'core-dump'.
> -------------------------------
>
> The "Unknown operation" callback is caused by newly added piece of code in
> snmplib/snmp_api.c:
>
>  static int
>  snmp_resend_request(struct session_list *slp, netsnmp_request_list *rp,
>  int incr_retries)
>  {
>
> ...
>
>          tv.tv_sec += tv.tv_usec / 1000000L;
>          tv.tv_usec %= 1000000L;
>          rp->expireM = tv;
> +        if (rp->callback)
> +            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
> +                         rp->pdu->reqid, rp->pdu, rp->cb_data);
>      }
>      return 0;
>  }
>
>
> When I tried to remove it, it just stop complaining about operation 6, but
> the core dump is still present.
>
> May I ask you for help with this issue? Do you have any idea, what causing
> this issue in 5.8 and how to fix it?
> I know, that Jan Safranek has fixed this for 5.7 by commit [2], but it
> looks like something other has changed and this issue is current again.
>
> [1] https://sourceforge.net/p/net-snmp/bugs/2411/
> [2]
> https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df
>
> Regards
>
> Josef Ridky
> Software Engineer
> Core Services Team
> Red Hat Czech, s.r.o.
>
>
>
> _______________________________________________
> Net-snmp-coders mailing list
> Net...@li...
> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
>

Core dump with net-snmp-5.8

[Josef R. <jr...@re...>]

Hi,

recently, I have hit to an issue in net-snmp-5.8, that is connected to the bug report [1].

When I tried to run agentofdeath test from [1], snmpd daemon will crash with malloc(): smallbin double linked list corrupted or double free() issue and dumps core (see bellow).
>From log file, I can identified one issue with "Unknown operation".

This issue is in the agentx_got_response function (agent/mibgroup/agentx/master.c). There isn't implemented action for NETSNMP_CALLBACK_OP_RESEND (defined in include/net-snmp/library/snmp_api.h).
As result "Unknown operation 6 in agentx_got_response" is shown in log file.

/var/log/messages
-------------------------------
Mar 28 06:52:42 localhost snmpd[12073]: Unknown operation 6 in agentx_got_response
Mar 28 06:52:43 localhost snmpd[12073]: Unknown operation 6 in agentx_got_response
Mar 28 06:52:43 localhost snmpd[12073]: malloc(): smallbin double linked list corrupted
Mar 28 06:52:43 localhost systemd[1]: Started Process Core Dump (PID 13652/UID 0).
Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Main process exited, code=dumped, status=6/ABRT
Mar 28 06:52:48 localhost systemd[1]: snmpd.service: Failed with result 'core-dump'.
-------------------------------

The "Unknown operation" callback is caused by newly added piece of code in snmplib/snmp_api.c:

 static int
 snmp_resend_request(struct session_list *slp, netsnmp_request_list *rp,
 int incr_retries)
 {

...

         tv.tv_sec += tv.tv_usec / 1000000L;
         tv.tv_usec %= 1000000L;
         rp->expireM = tv;
+        if (rp->callback)
+            rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
+                         rp->pdu->reqid, rp->pdu, rp->cb_data);
     }
     return 0;
 }


When I tried to remove it, it just stop complaining about operation 6, but the core dump is still present.

May I ask you for help with this issue? Do you have any idea, what causing this issue in 5.8 and how to fix it? 
I know, that Jan Safranek has fixed this for 5.7 by commit [2], but it looks like something other has changed and this issue is current again. 

[1] https://sourceforge.net/p/net-snmp/bugs/2411/
[2] https://github.com/net-snmp/net-snmp/commit/793d596838ff7cb48a73b675d62897c56c9e62df

Regards

Josef Ridky
Software Engineer
Core Services Team
Red Hat Czech, s.r.o.

Re: SNMP Configure Script Issue: When I include a header file in include/net-snmp/library/snmp_logging.h

[prabu v. <pra...@gm...>]

Dear All,

As per my understanding with respect to Autoconf, I've made the following
changes and still observed the same fatal errors with less number.

*Changes Done with respect to configure.d:*

diff --git a/configure.d/config_modules_lib b/configure.d/config_modules_lib
index bb69daa..cffed6c 100644
--- a/configure.d/config_modules_lib
+++ b/configure.d/config_modules_lib
@@ -99,6 +99,9 @@ if test ! -d snmplib/transports ; then
   mkdir snmplib/transports
 fi

+echo "CPPFLAGS = "$CPPFLAGS
+
 #
 # Do transport module processing.
 #
diff --git a/configure.d/config_project_paths b/configure.d/
config_project_paths
index 9690c84..4878e1e 100644
--- a/configure.d/config_project_paths
+++ b/configure.d/config_project_paths
@@ -5,6 +5,8 @@
 ##
 #########################################

+CPPFLAGS="$CPPFLAGS -I$(pwd)/../../inc"
+
 ##
 #   Prefix paths:
 ##

*After making changes, did reconfiguration using the following command:*

$ autoreconf

*The output logs of ./configure: *













*checking ipv6 stack type... "linux-glibc, yes, using libc"checking for
platform-specific source...  CPPFLAGS =
-I/home/anand/workspace/modules/snmp/net-snmp-5.8.pre2/../../incchecking
for and configuring transport modules to use... In file included from
./include/net-snmp/output_api.h:181:0,                 from
./include/net-snmp/library/snmp_client.h:32,                 from
./include/net-snmp/varbind_api.h:102,                 from
./include/net-snmp/library/snmp_api.h:33,                 from
./include/net-snmp/definitions.h:23,                 from
./include/net-snmp/types.h:462,                 from
include/net-snmp/library/snmpUDPIPv6Domain.h:11,                 from
module_tmp_header.h:160:./include/net-snmp/library/snmp_logging.h:8:23:
fatal error: my_macros.h: No such file or directorycompilation terminated.*

Before the reconfiguration(autoreconf), I observed around 12 fatal errors
which have been reduced to 2 errors only. Please correct me if I'm going in
the wrong direction and help me to resolve this issue!

*Note:* The my_macros.h file is available in the
/home/anand/workspace/modules/snmp/net-snmp-5.8.pre2/../../inc directory
and in this scenario I did not make use of the ./configure option
--with-cflags.

Thanks in advance!

On Fri, Mar 22, 2019 at 3:50 PM prabu varadharajan <
pra...@gm...> wrote:

> Thanks for the quick response, Magnus.
>
> Please find the compilation options below and let me know if anything else
> required.
>
> libtool: compile:  gcc -I../include -I. -I../snmplib -I/usr/include/libnl3
> -I/home/anand/workspace/modules/snmp/explore/net-snmp-5.8.pre2/../inc/
> -DNETSNMP_ENABLE_IPV6 -fno-strict-aliasing -DNETSNMP_REMOVE_U64
> -I/home/anand/workspace/modules/snmp/explore/net-snmp-5.8.pre2/../inc/
> -Ulinux -Dlinux=linux -c transports/snmpUDPIPv6Domain.c  -fPIC -DPIC -o
> transports/.libs/snmpUDPIPv6Domain.o
>
> For additional reference please find the configure options as well.
>
> *./configure --without-openssl --with-default-snmp-version=2
> --with-sys-contact=contact --with-sys-location=location
> --with-logfile=/var/log/snmpd.log --with-persistent-directory=/var/net-snmp
> --disable-manuals --disable-scripts --disable-mibs --without-perl-modules
> --disable-embedded-perl --enable-mini-agent --enable-fast-install
> --enable-ipv6 --with-transports=UDPIPv6
> --with-cflags=-I/home/anand/workspace/modules/snmp/explore/net-snmp-5.8.pre2/../inc/*
>
>
>
>
> On Fri, Mar 22, 2019 at 3:22 PM Magnus Fromreide <ma...@ly...>
> wrote:
>
>> On Fri, Mar 22, 2019 at 01:15:52PM +0530, prabu varadharajan wrote:
>> > Dear All,
>> >
>> > Following is my project source code hierarchy,
>> > + src
>> >     |
>> >     +---- snmp
>> >     |
>> >     +---- my_app
>> >     |
>> >     +---- inc
>> >             |
>> >             +---- local_snmp_macros.h
>> >             +---- my_macros.h
>> >
>> > I have declared few functions and macros in both local_snmp_macros.h and
>> > my_macros.h files which are being used by both snmp and my_app
>> > applications. As of now I'm copying the header files before ./configure
>> and
>> > compilation from src/inc/ to src/snmp/agent/mibgroup and removing those
>> on
>> > clean which is not recommended. So now I would like to avoid this
>> copying
>> > overhead and include from src/inc itself.
>> >
>> > For this I have updated ./configure --with-cflags=-I$(pwd)/../inc and
>> able
>> > to build the application successfully but I'm getting following fatal
>> errors.
>> > In this case also I'm observing some* weird behaviour* like I have
>> > included my_macros.h
>> > and local_snmp_macros.h files in
>> > src/snmp/include/net-snmp/library/snmp_logging.h whereas I'm getting
>> fatal
>> > errors for the *first one*(my_macros.h) only. I have tried exploring
>> > Makefile.* and configure script as well but I could not resolve this
>> one.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > *In file included from
>> > ./include/net-snmp/output_api.h:181:0,                 from
>> > ./include/net-snmp/library/snmp_client.h:32,                 from
>> > ./include/net-snmp/varbind_api.h:102,                 from
>> > ./include/net-snmp/library/snmp_api.h:33,                 from
>> > ./include/net-snmp/definitions.h:23,                 from
>> > ./include/net-snmp/types.h:462,                 from
>> > include/net-snmp/library/snmpUDPIPv6Domain.h:11,                 from
>> > module_tmp_header.h:164:./include/net-snmp/library/snmp_logging.h:7:23:
>> > fatal error: my_macros.h: No such file or directorycompilation
>> terminated.*
>> >
>> > *File include view for reference:*
>> > #ifndef SNMP_LOGGING_H
>> > #define SNMP_LOGGING_H
>> >
>> > #include <net-snmp/types.h>
>> > #include <net-snmp/output_api.h>
>> > #include "dummy.h"
>> > #include <my_macros.h>
>> > #include <local_snmp_macros.h>
>> >
>> > As per my understanding, I suspect the *configure* script only which
>> could
>> > have some bug. Please correct me if I'm wrong and help me to overcome
>> from
>> > this error.
>> >
>> > *Note: *I have observed same behaviour in noth Net-SNMP-5.5 and
>> > Net-SNMP-5.8.pre2 as well.
>> >
>> > Please feel free to ask more details if required that I can provide to
>> > resolve this issue. Or let me know if any reference link if this issue
>> is
>> > already reported and resolved.
>>
>> Could you please show the compile line before the error message (something
>> like gcc blah blah something.c -o something.o)
>>
>
>
> --
> With Best Regards,
> Anandaprabu V <https://www.linkedin.com/in/anandaprabu-v-10867671/>
> VVDN Technologies Pvt. Ltd <http://www.vvdntech.com/>, Chennai
> Cell : +91 9500650885 | Skype : prabuvaradharajan
>


-- 
With Best Regards,
Anandaprabu V <https://www.linkedin.com/in/anandaprabu-v-10867671/>
VVDN Technologies Pvt. Ltd <http://www.vvdntech.com/>, Chennai
Cell : +91 9500650885 | Skype : prabuvaradharajan

Re: SNMP Configure Script Issue: When I include a header file in include/net-snmp/library/snmp_logging.h

[prabu v. <pra...@gm...>]

Thanks for the quick response, Magnus.

Please find the compilation options below and let me know if anything else
required.

libtool: compile:  gcc -I../include -I. -I../snmplib -I/usr/include/libnl3
-I/home/anand/workspace/modules/snmp/explore/net-snmp-5.8.pre2/../inc/
-DNETSNMP_ENABLE_IPV6 -fno-strict-aliasing -DNETSNMP_REMOVE_U64
-I/home/anand/workspace/modules/snmp/explore/net-snmp-5.8.pre2/../inc/
-Ulinux -Dlinux=linux -c transports/snmpUDPIPv6Domain.c  -fPIC -DPIC -o
transports/.libs/snmpUDPIPv6Domain.o

For additional reference please find the configure options as well.

*./configure --without-openssl --with-default-snmp-version=2
--with-sys-contact=contact --with-sys-location=location
--with-logfile=/var/log/snmpd.log --with-persistent-directory=/var/net-snmp
--disable-manuals --disable-scripts --disable-mibs --without-perl-modules
--disable-embedded-perl --enable-mini-agent --enable-fast-install
--enable-ipv6 --with-transports=UDPIPv6
--with-cflags=-I/home/anand/workspace/modules/snmp/explore/net-snmp-5.8.pre2/../inc/*




On Fri, Mar 22, 2019 at 3:22 PM Magnus Fromreide <ma...@ly...>
wrote:

> On Fri, Mar 22, 2019 at 01:15:52PM +0530, prabu varadharajan wrote:
> > Dear All,
> >
> > Following is my project source code hierarchy,
> > + src
> >     |
> >     +---- snmp
> >     |
> >     +---- my_app
> >     |
> >     +---- inc
> >             |
> >             +---- local_snmp_macros.h
> >             +---- my_macros.h
> >
> > I have declared few functions and macros in both local_snmp_macros.h and
> > my_macros.h files which are being used by both snmp and my_app
> > applications. As of now I'm copying the header files before ./configure
> and
> > compilation from src/inc/ to src/snmp/agent/mibgroup and removing those
> on
> > clean which is not recommended. So now I would like to avoid this copying
> > overhead and include from src/inc itself.
> >
> > For this I have updated ./configure --with-cflags=-I$(pwd)/../inc and
> able
> > to build the application successfully but I'm getting following fatal
> errors.
> > In this case also I'm observing some* weird behaviour* like I have
> > included my_macros.h
> > and local_snmp_macros.h files in
> > src/snmp/include/net-snmp/library/snmp_logging.h whereas I'm getting
> fatal
> > errors for the *first one*(my_macros.h) only. I have tried exploring
> > Makefile.* and configure script as well but I could not resolve this one.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *In file included from
> > ./include/net-snmp/output_api.h:181:0,                 from
> > ./include/net-snmp/library/snmp_client.h:32,                 from
> > ./include/net-snmp/varbind_api.h:102,                 from
> > ./include/net-snmp/library/snmp_api.h:33,                 from
> > ./include/net-snmp/definitions.h:23,                 from
> > ./include/net-snmp/types.h:462,                 from
> > include/net-snmp/library/snmpUDPIPv6Domain.h:11,                 from
> > module_tmp_header.h:164:./include/net-snmp/library/snmp_logging.h:7:23:
> > fatal error: my_macros.h: No such file or directorycompilation
> terminated.*
> >
> > *File include view for reference:*
> > #ifndef SNMP_LOGGING_H
> > #define SNMP_LOGGING_H
> >
> > #include <net-snmp/types.h>
> > #include <net-snmp/output_api.h>
> > #include "dummy.h"
> > #include <my_macros.h>
> > #include <local_snmp_macros.h>
> >
> > As per my understanding, I suspect the *configure* script only which
> could
> > have some bug. Please correct me if I'm wrong and help me to overcome
> from
> > this error.
> >
> > *Note: *I have observed same behaviour in noth Net-SNMP-5.5 and
> > Net-SNMP-5.8.pre2 as well.
> >
> > Please feel free to ask more details if required that I can provide to
> > resolve this issue. Or let me know if any reference link if this issue is
> > already reported and resolved.
>
> Could you please show the compile line before the error message (something
> like gcc blah blah something.c -o something.o)
>


-- 
With Best Regards,
Anandaprabu V <https://www.linkedin.com/in/anandaprabu-v-10867671/>
VVDN Technologies Pvt. Ltd <http://www.vvdntech.com/>, Chennai
Cell : +91 9500650885 | Skype : prabuvaradharajan

Re: SNMP Configure Script Issue: When I include a header file in include/net-snmp/library/snmp_logging.h

[Magnus F. <ma...@ly...>]

On Fri, Mar 22, 2019 at 01:15:52PM +0530, prabu varadharajan wrote:
> Dear All,
> 
> Following is my project source code hierarchy,
> + src
>     |
>     +---- snmp
>     |
>     +---- my_app
>     |
>     +---- inc
>             |
>             +---- local_snmp_macros.h
>             +---- my_macros.h
> 
> I have declared few functions and macros in both local_snmp_macros.h and
> my_macros.h files which are being used by both snmp and my_app
> applications. As of now I'm copying the header files before ./configure and
> compilation from src/inc/ to src/snmp/agent/mibgroup and removing those on
> clean which is not recommended. So now I would like to avoid this copying
> overhead and include from src/inc itself.
> 
> For this I have updated ./configure --with-cflags=-I$(pwd)/../inc and able
> to build the application successfully but I'm getting following fatal errors.
> In this case also I'm observing some* weird behaviour* like I have
> included my_macros.h
> and local_snmp_macros.h files in
> src/snmp/include/net-snmp/library/snmp_logging.h whereas I'm getting fatal
> errors for the *first one*(my_macros.h) only. I have tried exploring
> Makefile.* and configure script as well but I could not resolve this one.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> *In file included from
> ./include/net-snmp/output_api.h:181:0,                 from
> ./include/net-snmp/library/snmp_client.h:32,                 from
> ./include/net-snmp/varbind_api.h:102,                 from
> ./include/net-snmp/library/snmp_api.h:33,                 from
> ./include/net-snmp/definitions.h:23,                 from
> ./include/net-snmp/types.h:462,                 from
> include/net-snmp/library/snmpUDPIPv6Domain.h:11,                 from
> module_tmp_header.h:164:./include/net-snmp/library/snmp_logging.h:7:23:
> fatal error: my_macros.h: No such file or directorycompilation terminated.*
> 
> *File include view for reference:*
> #ifndef SNMP_LOGGING_H
> #define SNMP_LOGGING_H
> 
> #include <net-snmp/types.h>
> #include <net-snmp/output_api.h>
> #include "dummy.h"
> #include <my_macros.h>
> #include <local_snmp_macros.h>
> 
> As per my understanding, I suspect the *configure* script only which could
> have some bug. Please correct me if I'm wrong and help me to overcome from
> this error.
> 
> *Note: *I have observed same behaviour in noth Net-SNMP-5.5 and
> Net-SNMP-5.8.pre2 as well.
> 
> Please feel free to ask more details if required that I can provide to
> resolve this issue. Or let me know if any reference link if this issue is
> already reported and resolved.

Could you please show the compile line before the error message (something
like gcc blah blah something.c -o something.o)

SNMP Configure Script Issue: When I include a header file in include/net-snmp/library/snmp_logging.h

[prabu v. <pra...@gm...>]

Dear All,

Following is my project source code hierarchy,
+ src
    |
    +---- snmp
    |
    +---- my_app
    |
    +---- inc
            |
            +---- local_snmp_macros.h
            +---- my_macros.h

I have declared few functions and macros in both local_snmp_macros.h and
my_macros.h files which are being used by both snmp and my_app
applications. As of now I'm copying the header files before ./configure and
compilation from src/inc/ to src/snmp/agent/mibgroup and removing those on
clean which is not recommended. So now I would like to avoid this copying
overhead and include from src/inc itself.

For this I have updated ./configure --with-cflags=-I$(pwd)/../inc and able
to build the application successfully but I'm getting following fatal errors.
In this case also I'm observing some* weird behaviour* like I have
included my_macros.h
and local_snmp_macros.h files in
src/snmp/include/net-snmp/library/snmp_logging.h whereas I'm getting fatal
errors for the *first one*(my_macros.h) only. I have tried exploring
Makefile.* and configure script as well but I could not resolve this one.










*In file included from
./include/net-snmp/output_api.h:181:0,                 from
./include/net-snmp/library/snmp_client.h:32,                 from
./include/net-snmp/varbind_api.h:102,                 from
./include/net-snmp/library/snmp_api.h:33,                 from
./include/net-snmp/definitions.h:23,                 from
./include/net-snmp/types.h:462,                 from
include/net-snmp/library/snmpUDPIPv6Domain.h:11,                 from
module_tmp_header.h:164:./include/net-snmp/library/snmp_logging.h:7:23:
fatal error: my_macros.h: No such file or directorycompilation terminated.*

*File include view for reference:*
#ifndef SNMP_LOGGING_H
#define SNMP_LOGGING_H

#include <net-snmp/types.h>
#include <net-snmp/output_api.h>
#include "dummy.h"
#include <my_macros.h>
#include <local_snmp_macros.h>

As per my understanding, I suspect the *configure* script only which could
have some bug. Please correct me if I'm wrong and help me to overcome from
this error.

*Note: *I have observed same behaviour in noth Net-SNMP-5.5 and
Net-SNMP-5.8.pre2 as well.

Please feel free to ask more details if required that I can provide to
resolve this issue. Or let me know if any reference link if this issue is
already reported and resolved.

-- 
With Best Regards,
Anandaprabu V <https://www.linkedin.com/in/anandaprabu-v-10867671/>
VVDN Technologies Pvt. Ltd <http://www.vvdntech.com/>, Chennai
Cell : +91 9500650885 | Skype : prabuvaradharajan

QNX 7

[Ian C <mc...@ya...>]

I'm trying to configure a cross-compilation on Ubuntu for QNX 7. Does anyone work what configure options to use?All the ones I've attempted so far end up with:
configure: error: cannot run C compiled programs
Ive tried:--with-cc=qcc --build=QNX --build=QNX --host=QNX
but really those are just (bad) guesses.
Any help would be awesome.
thanks,Ian

Re: Getting linker error for Net-snmp. Need help

[Magnus F. <ma...@ly...>]

On Wed, Feb 27, 2019 at 09:23:40AM +0000, Kir...@de... wrote:
> Please let me know the fix for the same.

Please let us know what you did before the make.
How did you invoke configure?
Did you run in a clean checkout?

> # make
> making all in /usr/pkg/src/net-snmp-5.8/snmplib
> making all in /usr/pkg/src/net-snmp-5.8/agent
> making all in /usr/pkg/src/net-snmp-5.8/agent/helpers
> making all in /usr/pkg/src/net-snmp-5.8/agent/mibgroup
> /bin/sh ../libtool --mode=link gcc -g -O2 -DNETSNMP_ENABLE_IPV6 -fno-strict-aliasing -DNETSNMP_REMOVE_U64 -g -O2 -Unetbsdelf7 -Dnetbsdelf7=netbsdelf7 -o snmpd snmpd.lo  libnetsnmpagent.la libnetsnmpmibs.la ../snmplib/libnetsnmp.la -lelf -lm
> libtool: link: gcc -g -O2 -DNETSNMP_ENABLE_IPV6 -fno-strict-aliasing -DNETSNMP_REMOVE_U64 -g -O2 -Unetbsdelf7 -Dnetbsdelf7=netbsdelf7 -o .libs/snmpd .libs/snmpd.o  ./.libs/libnetsnmpagent.so ./.libs/libnetsnmpmibs.so /usr/pkg/src/net-snmp-5.8/agent/.libs/libnetsnmpagent.so /usr/pkg/src/net-snmp-5.8/snmplib/.libs/libnetsnmp.so -lkvm ../snmplib/.libs/libnetsnmp.so -lcrypto -lelf -lm  -Wl,-rpath -Wl,/usr/local/lib
> ./.libs/libnetsnmpmibs.so: undefined reference to `netbsd_read_ip_stat'
> ./.libs/libnetsnmpmibs.so: undefined reference to `netbsd_read_icmp_stat'
> ./.libs/libnetsnmpmibs.so: undefined reference to `netbsd_read_icmp6_stat'
> ./.libs/libnetsnmpmibs.so: undefined reference to `netbsd_read_icmp_msg_stat'
> ./.libs/libnetsnmpmibs.so: undefined reference to `netbsd_read_icmp6_msg_stat'
> ./.libs/libnetsnmpmibs.so: undefined reference to `netbsd_read_tcp_stat'
> ./.libs/libnetsnmpmibs.so: undefined reference to `netbsd_read_udp_stat'
> 
> Regards,
> Kiran


> _______________________________________________
> Net-snmp-coders mailing list
> Net...@li...
> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders