Menu

#1034 5.x agentx memory leak

closed
agent (1105)
4
2014-10-16
2004-03-15
Anonymous
No

I've a steady memory leak using sorted-array container
in 5.0.9, 5.1,
5.1.1-pre2, 5.1.1-V5-1-patches.

The behavior is exhibited when performing a SET request
on a column in a row
(either actually succeding in changing the column or
getting rejected). I
thought it was a problem with the double-free issue
(reported earlier on this
mailing list, and I did check the code to make sure the
fix was in), but this
seems like a bigger fish.

STEPS TO REPRODUCE THIS BEHAVIOR:

1). Download the 'example-daemon code' from:
http://www.net-snmp.org/tutorial-5/toolkit/demon/example-demon.c
And the 'Makefile' as well
(http://www.net-snmp.org/tutorial-5/toolkit/demoapp/Makefile)

2). Generate the sub-agent code:

\#mib2c -c mib2c.array-user.conf netSnmpIETFWGTable

Edit the include file to uncomment second index.
 104 //\#define netSnmpIETFWGTable\_IDX2

also in the source code, edit

netSnmpIETFWGTable_extract_index function
to point the last index value to NULL.

242     var\_nsIETFWGName.next\_variable = NULL;

and in netSnmpIETFWGTable\_set\_reserve1 function change

the check
to only check the type, not the length (if you don't
change the
check for length the sub-agent will expect a string of
*exactly* 65534 octets
long).

441             rc = netsnmp\_check\_vb\_type\(var,

ASN_OCTET_STR);
442 break;

3). Replace in the downloaded files, the string
"nstAgentSubagentObject" with
"netSnmpIETFWGTable".

cat Makefile

s/nstAgentSubagentObject/netSnmpIETFWGTable/g > a;
mv a Makefile
cat example-demon.c
s/nstAgentSubagentObject/netSnmpIETFWGTable/g > a;
mv a example-demon.c

4). Compile the code.
'make'
Ignore the warnings or just comment out the offending
lines.

5). Configure your SNMPd daemon snmpd.conf to include
write access and
AgentX support, such as:

rwcommunity  public 127.0.0.1
master  agentx

For simplicity, write those in your local directory

snmpd.conf file.

6). Start the SNMP daemon in one console

snmpd -c ./snmpd.conf -C -f -V -L

\(-c ./snmpd.conf points to the file created in step 5\).

7). In another console start the sub-agent:

./example-daemon

8). In yet another console start writing to one column
exactly the same
value in a loop:

$while \(true\); do snmpset -v2c -c public localhost

nsIETFWGChair1.\"bob\" =
"BLAH"; ps -aux | grep example | grep -v grep; done

And see how the memory grows.
You should see something like this:

NET-SNMP-EXAMPLES-MIB::nsIETFWGChair1."bob" = STRING:
"BLAH"
root 16789 8.2 0.2 6520 2080 pts/7 S 13:37
0:00 ./example-demon
NET-SNMP-EXAMPLES-MIB::nsIETFWGChair1."bob" = STRING:
"BLAH"
root 16789 8.2 0.2 6524 2084 pts/7 S 13:37
0:00 ./example-demon
NET-SNMP-EXAMPLES-MIB::nsIETFWGChair1."bob" = STRING:
"BLAH"
root 16789 6.0 0.2 6528 2088 pts/7 S 13:37
0:00 ./example-demon

9). For a better memory leak, try to set
'nsIETFWGChair2' with some value
(it will fail, b/c the length of your string is not
exactly 65534), but
the sub-agent continues to grow in size:

bash-2.05b$ while (true); do snmpset -v2c -c public
localhost
nsIETFWGChair2.\"bob\" = "BLAH"; ps -aux | grep example
| grep -v grep; done
Error in packet.
Reason: wrongLength (The set value has an illegal
length from what the agent
expects)
Failed object: NET-SNMP-EXAMPLES-MIB::nsIETFWGChair2."bob"

root 16941 3.4 0.2 6520 2076 pts/7 S 13:40
0:00 ./example-demon
Error in packet.
Reason: wrongLength (The set value has an illegal
length from what the agent
expects)
Failed object: NET-SNMP-EXAMPLES-MIB::nsIETFWGChair2."bob"

root 16941 3.4 0.2 6716 2084 pts/7 S 13:40
0:00 ./example-demon
Error in packet.
Reason: wrongLength (The set value has an illegal
length from what the agent
expects)
Failed object: NET-SNMP-EXAMPLES-MIB::nsIETFWGChair2."bob"

root 16941 3.1 0.2 6916 2096 pts/7 S 13:40
0:00 ./example-demon
Error in packet.
Reason: wrongLength (The set value has an illegal
length from what the agent
expects)
Failed object: NET-SNMP-EXAMPLES-MIB::nsIETFWGChair2."bob"

10). To make sure this was not an AgentX issue, I also
changed the
'example-daemon.c' code to compile as SNMP master agent:

 17   int agentx\_subagent=0; /\* change this if you

want to be a SNMP
master agent */

And the memory leak continued:

root 17205 4.3 0.1 6516 2052 pts/7 S 13:50
0:00 ./example-demon
root 17205 3.9 0.1 6712 2060 pts/7 S 13:50
0:00 ./example-demon
root 17205 3.9 0.2 6908 2068 pts/7 S 13:50
0:00 ./example-demon
root 17205 3.5 0.2 7104 2076 pts/7 S 13:50
0:00 ./example-demon
root 17205 3.2 0.2 7300 2084 pts/7 S 13:50
0:00 ./example-demon
root 17205 3.2 0.2 7496 2092 pts/7 S 13:50
0:00 ./example-demon
root 17205 3.0 0.2 7692 2100 pts/7 S 13:50
0:00 ./example-demon
root 17205 3.0 0.2 7888 2108 pts/7 S 13:50
0:00 ./example-demon
root 17205 2.7 0.2 8084 2116 pts/7 S 13:50
0:00 ./example-demon
root 17205 2.7 0.2 8280 2124 pts/7 S 13:50
0:00 ./example-demon

Discussion

  • Robert Story

    Robert Story - 2004-03-15

    Logged In: YES
    user_id=76148

    9) has been fixed in cvs for all 5.x releases.

     
  • Robert Story

    Robert Story - 2004-03-16

    Logged In: YES
    user_id=76148

    the leak seems to be the result of a change to fix a
    double-free (see log msg for agent/snmp_agent.c version
    1.190). Since a double free can result in a crash, a memory
    leak is preferrable.

    If you remove the NULL assignment to asp->pdu->variables,
    the leak should go away, but a crash may result. If you have
    a test bed, or non-critical machine, you could use this
    patch to try and duplicate the double-free problem. If that
    can be found and fixed, then the leak can be fixed too.

     
  • Robert Story

    Robert Story - 2004-03-22

    Logged In: YES
    user_id=76148

    To clarify the workaround for those who wish to risk the
    crash to fix the memory leak:

    in the function save_set_cache() in agent/snmp_agent.c,
    comment out the line 'asp->pdu->variables = NULL;'.

    If you do so, and find a way to reporduce the double-free,
    please let us know asap so the leak can be fixed.

     
  • Bernhard Penz

    Bernhard Penz - 2004-03-24

    Logged In: YES
    user_id=450794

    Somewhere during the SET processing, memory is
    allocated/cloned but isn't freed. Thats what purify tells me on
    the memory leak (the => points at the location of the leak in
    the call stack):

    copy\_varlist   \[snmp\_client.c:399\]
                 \* clone the next variable. Cleanup if alloc fails
                 \*/
                newvar = \(netsnmp\_variable\_list \*\)
     =>             malloc\(sizeof\(netsnmp\_variable\_list\)\);
                if \(snmp\_clone\_var\(var, newvar\)\) \{
                    if \(newvar\)
                        free\(\(char \*\) newvar\);
    copy\_pdu\_vars  \[snmp\_client.c:469\]
            if \(pdu->flags & UCD\_MSG\_FLAG\_FORCE\_PDU\_COPY\)
                copied = 1;             /\* We're interested
    

    in 'empty' responses too */

     =>     newpdu->variables = \_copy\_varlist\(var, drop\_idx,
    

    copy_count);
    if (newpdu->variables)
    copied = 1;

    clone\_pdu      \[snmp\_client.c:508\]
        \{
            netsnmp\_pdu    \*newpdu;
            newpdu = \_clone\_pdu\_header\(pdu\);
     =>     newpdu = \_copy\_pdu\_vars\(pdu, newpdu, drop\_err,
    

    0, 10000); /* skip none, copy all */

            return newpdu;
        \}
    snmp\_clone\_pdu \[snmp\_client.c:535\]
        netsnmp\_pdu    \*
        snmp\_clone\_pdu\(netsnmp\_pdu \*pdu\)
        \{
     =>     return \_clone\_pdu\(pdu, 0\);  /\* copies all variables
    

    */
    }

    init\_agent\_snmp\_session \[snmp\_agent.c:1117\]
    
            DEBUGMSGTL\(\("snmp\_agent","agent\_sesion %08p
    

    created\n", asp));
    asp->session = session;
    => asp->pdu = snmp_clone_pdu(pdu);
    asp->orig_pdu = snmp_clone_pdu(pdu);
    asp->rw = READ;
    asp->exact = TRUE;
    handle_snmp_packet [snmp_agent.c:1609]
    }

            if \(magic == NULL\) \{
     =>         asp = init\_agent\_snmp\_session\(session, pdu\);
                status = SNMP\_ERR\_NOERROR;
            \} else \{
                asp = \(netsnmp\_agent\_session \*\) magic;
    sess\_process\_packet \[snmp\_api.c:5118\]
               \*/
              handled = 1;
              sp->callback
    

    (NETSNMP_CALLBACK_OP_RECEIVED_MESSAGE,
    => sp, pdu->reqid, pdu, sp->callback_magic);
    /*
    * MTR snmp_res_unlock(MT_LIBRARY_ID,
    MT_LIB_SESSION);
    */
    sess_read [snmp_api.c:5516]
    return rc;
    } else {
    rc = _sess_process_packet(sessp, sp, isp,
    transport, opaque,
    => olength, rxbuf, length);
    SNMP_FREE(rxbuf);
    return rc;
    }
    snmp_sess_read [snmp_api.c:5534]
    netsnmp_session *pss;
    int rc;

     =>     rc = \_sess\_read\(sessp, fdset\);
            psl = \(struct session\_list \*\) sessp;
            pss = psl->session;
            if \(rc && pss->s\_snmp\_errno\) \{
    

    Asp->pdu->variables get cloned in the processing of a SET
    request, and because they are set to null in the
    save_set_cache, they are never freed. But freeing them
    before setting to null also results in a crash. So, the correct
    place to free would be at the end of a SET request. This
    would require some hack (e.g. a new member in asp like
    cloned_set_variables) since asp.requests is also set to null
    which are only freed a the very end of a SET request.

     

Log in to post a comment.