Memory Leak while parsing

Help
Abhilash
2012-08-24
2013-06-12
  • Abhilash
    Abhilash
    2012-08-24

    Below is my sample code:

    #include<iostream.h>
    #include<stdio.h>
    #include<string.h>
    #include "libjson.h"
    #include "JSONNode.h"

    int id;
    int errorCode;
    char message;

    void ParseJSON(JSONNODE *n);
    void parse(const char *msg);
    int main(int argv, char* argc)
    {
       parse("{\"id\":1,\"error\":{\"code\":-20103,\"message\":\"Unknown session 1.2.3.5 (for all subscribers)\"}}");
       json_free_all();
    }

    void parse(const char *msg)
    {
       JSONNODE *n=json_parse(msg);
       ParseJSON(n);
       json_clear(n);
       json_free_all();
    }

    void ParseJSON(JSONNODE *n)
    {
        if (n == NULL){
            printf("Invalid JSON Node\n");
            return;
        }
        JSONNODE_ITERATOR i=json_begin(n);
        for(;i < json_end(n);i++)
        {
           if (*i == NULL){
                printf("Invalid JSON Node\n");
                return;
            }

            // recursively call ourselves to dig deeper into the tree
            if (json_type(*i) == JSON_ARRAY || json_type(*i) == JSON_NODE){
                ParseJSON(*i);
            }

            // get the node name and value as a string
            json_char *node_name = json_name(*i);

            // find out where to store the values
            if (strcmp(node_name, "id") == 0){
                id = json_as_int(*i);
                cout<<"id:"<<id;
            }
            else if (strcmp(node_name, "code") == 0){
                errorCode = json_as_int(*i);
                cout<<"errorCode:"<<errorCode;
            }
            else if (strcmp(node_name, "message") == 0){
               json_char *node_value = json_as_string(*i);
               strcpy(message, node_value);
               json_free(node_value);
                cout<<"message:"<<message;
            }
            cout<<" ";

            // cleanup and increment the iterator
            json_free(node_name);
            cout<<"\n";

        }
    }

    I am able to parse the msg, but valgrind shows memory leak of 240 bytes.

    Valgrind Output:

    ==31193== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 19 from 1)
    ==31193== malloc/free: in use at exit: 540 bytes in 24 blocks.
    ==31193== malloc/free: 224 allocs, 200 frees, 7,233 bytes allocated.
    ==31193== For counts of detected errors, rerun with: -v
    ==31193== searching for pointers to 24 not-freed blocks.
    ==31193== checked 100,168 bytes.
    ==31193==
    ==31193== 20 bytes in 1 blocks are definitely lost in loss record 1 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x40274FA: internalJSONNode::getDtorCounter() (internalJSONNode.h:83)
    ==31193==    by 0x40210EE: internalJSONNode::~internalJSONNode() (internalJSONNode.cpp:151)
    ==31193==    by 0x4021222: internalJSONNode::deleteInternal(internalJSONNode*) (internalJSONNode.cpp:677)
    ==31193==    by 0x402968E: JSONNode::decRef() (JSONNode.h:826)
    ==31193==    by 0x40296DB: JSONNode::~JSONNode() (JSONNode.h:615)
    ==31193==    by 0x402FAC8: JSONNode::deleteJSONNode(JSONNode*) (JSONNode.cpp:311)
    ==31193==    by 0x402B192: jsonChildren::deleteAll() (JSONChildren.cpp:76)
    ==31193==    by 0x404489B: jsonChildren::clear() (JSONChildren.h:111)
    ==31193==    by 0x4044A1E: JSONNode::clear() (JSONNode.h:633)
    ==31193==    by 0x403C602: json_clear (libjson.cpp:449)
    ==31193==
    ==31193==
    ==31193== 42 (20 direct, 22 indirect) bytes in 1 blocks are definitely lost in loss record 2 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x4028576: json_auto<char>::getCtorCounter() (JSONMemory.h:142)
    ==31193==    by 0x403389F: json_auto<char>::json_auto() (JSONMemory.h:143)
    ==31193==    by 0x4037626: JSONWorker::parse(std::string const&) (JSONWorker.cpp:12)
    ==31193==    by 0x403E54E: json_parse (libjson.cpp:83)
    ==31193==    by 0x8049242: parse(char const*) (post.cpp:21)
    ==31193==    by 0x8049280: main (post.cpp:15)
    ==31193==
    ==31193==
    ==31193== 49 (20 direct, 29 indirect) bytes in 1 blocks are definitely lost in loss record 3 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x40273C0: internalJSONNode::getCtorCounter() (internalJSONNode.h:83)
    ==31193==    by 0x4021582: internalJSONNode::internalJSONNode(std::string const&) (internalJSONNode.cpp:47)
    ==31193==    by 0x402172D: internalJSONNode::newInternal(std::string const&) (internalJSONNode.cpp:698)
    ==31193==    by 0x403840E: JSONNode::JSONNode(std::string const&) (JSONNode.h:539)
    ==31193==    by 0x40372BF: JSONWorker::_parse_unformatted(char const*, char const*) (JSONWorker.cpp:67)
    ==31193==    by 0x403766E: JSONWorker::parse(std::string const&) (JSONWorker.cpp:15)
    ==31193==    by 0x403E54E: json_parse (libjson.cpp:83)
    ==31193==    by 0x8049242: parse(char const*) (post.cpp:21)
    ==31193==    by 0x8049280: main (post.cpp:15)
    ==31193==
    ==31193==
    ==31193== 45 (20 direct, 25 indirect) bytes in 1 blocks are definitely lost in loss record 4 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x4027C56: jsonChildren::getCtorCounter() (JSONChildren.h:39)
    ==31193==    by 0x4027D36: jsonChildren::jsonChildren() (JSONChildren.h:42)
    ==31193==    by 0x4027D8E: jsonChildren::newChildren() (JSONChildren.h:264)
    ==31193==    by 0x40215C8: internalJSONNode::internalJSONNode(std::string const&) (internalJSONNode.cpp:51)
    ==31193==    by 0x402172D: internalJSONNode::newInternal(std::string const&) (internalJSONNode.cpp:698)
    ==31193==    by 0x403840E: JSONNode::JSONNode(std::string const&) (JSONNode.h:539)
    ==31193==    by 0x40372BF: JSONWorker::_parse_unformatted(char const*, char const*) (JSONWorker.cpp:67)
    ==31193==    by 0x403766E: JSONWorker::parse(std::string const&) (JSONWorker.cpp:15)
    ==31193==    by 0x403E54E: json_parse (libjson.cpp:83)
    ==31193==    by 0x8049242: parse(char const*) (post.cpp:21)
    ==31193==
    ==31193==
    ==31193== 41 (20 direct, 21 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x4028E0C: JSONNode::getCtorCounter() (JSONNode.h:137)
    ==31193==    by 0x403841A: JSONNode::JSONNode(std::string const&) (JSONNode.h:540)
    ==31193==    by 0x40372BF: JSONWorker::_parse_unformatted(char const*, char const*) (JSONWorker.cpp:67)
    ==31193==    by 0x403766E: JSONWorker::parse(std::string const&) (JSONWorker.cpp:15)
    ==31193==    by 0x403E54E: json_parse (libjson.cpp:83)
    ==31193==    by 0x8049242: parse(char const*) (post.cpp:21)
    ==31193==    by 0x8049280: main (post.cpp:15)
    ==31193==
    ==31193==
    ==31193== 41 (20 direct, 21 indirect) bytes in 1 blocks are definitely lost in loss record 6 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x4028A5E: JSONNode::getDtorCounter() (JSONNode.h:137)
    ==31193==    by 0x40296E0: JSONNode::~JSONNode() (JSONNode.h:616)
    ==31193==    by 0x403730C: JSONWorker::_parse_unformatted(char const*, char const*) (JSONWorker.cpp:69)
    ==31193==    by 0x403766E: JSONWorker::parse(std::string const&) (JSONWorker.cpp:15)
    ==31193==    by 0x403E54E: json_parse (libjson.cpp:83)
    ==31193==    by 0x8049242: parse(char const*) (post.cpp:21)
    ==31193==    by 0x8049280: main (post.cpp:15)
    ==31193==
    ==31193==
    ==31193== 42 (20 direct, 22 indirect) bytes in 1 blocks are definitely lost in loss record 7 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x40286B0: json_auto<char>::getDtorCounter() (JSONMemory.h:142)
    ==31193==    by 0x402972E: json_auto<char>::~json_auto() (JSONMemory.h:148)
    ==31193==    by 0x403767C: JSONWorker::parse(std::string const&) (JSONWorker.cpp:15)
    ==31193==    by 0x403E54E: json_parse (libjson.cpp:83)
    ==31193==    by 0x8049242: parse(char const*) (post.cpp:21)
    ==31193==    by 0x8049280: main (post.cpp:15)
    ==31193==
    ==31193==
    ==31193== 49 (20 direct, 29 indirect) bytes in 1 blocks are definitely lost in loss record 8 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x4040B38: auto_expand_node::getCtorCounter() (JSONMemory.h:100)
    ==31193==    by 0x4040BF9: auto_expand_node::auto_expand_node() (JSONMemory.h:101)
    ==31193==    by 0x4040C6A: jsonSingletonNODE_HANDLER::jsonSingletonNODE_HANDLER() (JSONGlobals.h:76)
    ==31193==    by 0x4040CB5: jsonSingletonNODE_HANDLER::getValue() (JSONGlobals.h:76)
    ==31193==    by 0x403E563: json_parse (libjson.cpp:83)
    ==31193==    by 0x8049242: parse(char const*) (post.cpp:21)
    ==31193==    by 0x8049280: main (post.cpp:15)
    ==31193==
    ==31193==
    ==31193== 44 (20 direct, 24 indirect) bytes in 1 blocks are definitely lost in loss record 9 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x404113A: auto_expand::getCtorCounter() (JSONMemory.h:81)
    ==31193==    by 0x40411FB: auto_expand::auto_expand() (JSONMemory.h:82)
    ==31193==    by 0x404126C: jsonSingletonSTRING_HANDLER::jsonSingletonSTRING_HANDLER() (JSONGlobals.h:75)
    ==31193==    by 0x40412B7: jsonSingletonSTRING_HANDLER::getValue() (JSONGlobals.h:75)
    ==31193==    by 0x4044E88: toCString(std::string const&) (libjson.cpp:35)
    ==31193==    by 0x403CCC3: json_name (libjson.cpp:334)
    ==31193==    by 0x8049070: ParseJSON(void*) (post.cpp:47)
    ==31193==    by 0x8049250: parse(char const*) (post.cpp:22)
    ==31193==    by 0x8049280: main (post.cpp:15)
    ==31193==
    ==31193==
    ==31193== 29 bytes in 1 blocks are possibly lost in loss record 10 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0xCD4A1A: std::string::_Rep::_S_create(unsigned, unsigned, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.8)
    ==31193==    by 0xCD5814: (within /usr/lib/libstdc++.so.6.0.8)
    ==31193==    by 0xCD5A26: std::string::string(char const*, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.8)
    ==31193==    by 0x40274C7: internalJSONNode::getDtorCounter() (internalJSONNode.h:83)
    ==31193==    by 0x40210EE: internalJSONNode::~internalJSONNode() (internalJSONNode.cpp:151)
    ==31193==    by 0x4021222: internalJSONNode::deleteInternal(internalJSONNode*) (internalJSONNode.cpp:677)
    ==31193==    by 0x402968E: JSONNode::decRef() (JSONNode.h:826)
    ==31193==    by 0x40296DB: JSONNode::~JSONNode() (JSONNode.h:615)
    ==31193==    by 0x402FAC8: JSONNode::deleteJSONNode(JSONNode*) (JSONNode.cpp:311)
    ==31193==    by 0x402B192: jsonChildren::deleteAll() (JSONChildren.cpp:76)
    ==31193==    by 0x404489B: jsonChildren::clear() (JSONChildren.h:111)
    ==31193==
    ==31193==
    ==31193== 45 (20 direct, 25 indirect) bytes in 1 blocks are definitely lost in loss record 11 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x40278A8: jsonChildren::getDtorCounter() (JSONChildren.h:39)
    ==31193==    by 0x402992A: jsonChildren::~jsonChildren() (JSONChildren.h:57)
    ==31193==    by 0x4025AD5: jsonChildren::deleteChildren(jsonChildren*) (JSONChildren.h:256)
    ==31193==    by 0x4021114: internalJSONNode::~internalJSONNode() (internalJSONNode.cpp:155)
    ==31193==    by 0x4021222: internalJSONNode::deleteInternal(internalJSONNode*) (internalJSONNode.cpp:677)
    ==31193==    by 0x402968E: JSONNode::decRef() (JSONNode.h:826)
    ==31193==    by 0x40296DB: JSONNode::~JSONNode() (JSONNode.h:615)
    ==31193==    by 0x402FAC8: JSONNode::deleteJSONNode(JSONNode*) (JSONNode.cpp:311)
    ==31193==    by 0x402B192: jsonChildren::deleteAll() (JSONChildren.cpp:76)
    ==31193==    by 0x404489B: jsonChildren::clear() (JSONChildren.h:111)
    ==31193==
    ==31193==
    ==31193== 44 (20 direct, 24 indirect) bytes in 1 blocks are definitely lost in loss record 12 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x404138E: auto_expand::getDtorCounter() (JSONMemory.h:81)
    ==31193==    by 0x4044EE7: auto_expand::~auto_expand() (JSONMemory.h:83)
    ==31193==    by 0x4044F70: jsonSingletonSTRING_HANDLER::~jsonSingletonSTRING_HANDLER() (JSONGlobals.h:75)
    ==31193==    by 0x403D017: __tcf_8 (JSONGlobals.h:75)
    ==31193==    by 0x4677E7F8: exit (in /lib/libc-2.5.so)
    ==31193==    by 0x46768DF3: (below main) (in /lib/libc-2.5.so)
    ==31193==
    ==31193==
    ==31193== 49 (20 direct, 29 indirect) bytes in 1 blocks are definitely lost in loss record 13 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0x40272B7: JSONStats::setCallbacks(unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), unsigned& (*)(), std::string const&) (JSONStats.h:65)
    ==31193==    by 0x4040D8C: auto_expand_node::getDtorCounter() (JSONMemory.h:100)
    ==31193==    by 0x4043B47: auto_expand_node::~auto_expand_node() (JSONMemory.h:102)
    ==31193==    by 0x4043BD0: jsonSingletonNODE_HANDLER::~jsonSingletonNODE_HANDLER() (JSONGlobals.h:76)
    ==31193==    by 0x403A6DF: __tcf_9 (JSONGlobals.h:76)
    ==31193==    by 0x4677E7F8: exit (in /lib/libc-2.5.so)
    ==31193==    by 0x46768DF3: (below main) (in /lib/libc-2.5.so)
    ==31193==
    ==31193==
    ==31193== 271 bytes in 11 blocks are indirectly lost in loss record 14 of 14
    ==31193==    at 0x4005B65: operator new(unsigned) (vg_replace_malloc.c:163)
    ==31193==    by 0xCD4A1A: std::string::_Rep::_S_create(unsigned, unsigned, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.8)
    ==31193==    by 0xCD5814: (within /usr/lib/libstdc++.so.6.0.8)
    ==31193==    by 0xCD5A26: std::string::string(char const*, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.8)
    ==31193==    by 0x4028543: json_auto<char>::getCtorCounter() (JSONMemory.h:142)
    ==31193==    by 0x403389F: json_auto<char>::json_auto() (JSONMemory.h:143)
    ==31193==    by 0x4037626: JSONWorker::parse(std::string const&) (JSONWorker.cpp:12)
    ==31193==    by 0x403E54E: json_parse (libjson.cpp:83)
    ==31193==    by 0x8049242: parse(char const*) (post.cpp:21)
    ==31193==    by 0x8049280: main (post.cpp:15)
    ==31193==
    ==31193== LEAK SUMMARY:
    ==31193==    definitely lost: 240 bytes in 12 blocks.
    ==31193==    indirectly lost: 271 bytes in 11 blocks.
    ==31193==      possibly lost: 29 bytes in 1 blocks.
    ==31193==    still reachable: 0 bytes in 0 blocks.
    ==31193==         suppressed: 0 bytes in 0 blocks.

    My JSONOptions.h file:

    #ifndef JSON_OPTIONS_H
    #define JSON_OPTIONS_H
    #define JSON_LIBRARY
    #define JSON_SAFE
    #define JSON_LESS_MEMORY
    #define JSON_REF_COUNT
    #define JSON_BINARY
    #define JSON_EXPOSE_BASE64
    #define JSON_ITERATORS
    #define JSON_STREAM
    #define JSON_MEMORY_MANAGE
    #define JSON_WRITE_PRIORITY MED
    #define JSON_READ_PRIORITY HIGH
    #define JSON_ESCAPE_WRITES
    #define JSON_COMMENTS
    #define JSON_VALIDATE
    #define JSON_CASE_INSENSITIVE_FUNCTIONS
    #define JSON_DEPRECATED_FUNCTIONS
    #define JSON_CASTABLE
    #define JSON_SECURITY_MAX_NEST_LEVEL 128
    #define JSON_SECURITY_MAX_STRING_LENGTH 33554432
    #define JSON_SECURITY_MAX_STREAM_OBJECTS 128
    #endif

     
  • I see a lot of JSONGlobals in the valgrind output, these aren't leaked, they are simply static variables.

     
  • I have recently added a feature to allow globals to be cleaned up as well, I will release it soon.

     
  • Abhilash
    Abhilash
    2012-08-24

    Thanks for the quick reply.
    Was wondering whether I missed out calling any cleanup functions.

     

  • Anonymous
    2012-08-27

    Yes, I'm experienced with the same problem (memory leak detected) when calling json_clear, json_delete,

    FYI: I'm using Visual Studio 2012 and compile it on x86.

     
  • Is the "leak" a constant amount, or does it grow depending on the number of times you parse?  I've seen a lot of people complain about small amounts of RAM, but they've always been traced down to static globals that are created as they are needed and not cleaned up until the library shuts down.  I also have boatloads of tests specifically for checking for leaks.

     
  • Abhilash
    Abhilash
    2012-08-28

    Its constant for multiple message parsing.