Re: [Jsoncpp-devel] Example code for Jsoncpp library

[Anath B. M. <a_b...@ya...>]

Hi Baptiste,

Thank you for your reply. 

Sorry for this confusion. To elaborate my point (use case), I want to write/read 
each 4K as a separate JSON structure/object that is enclosed within {}. For 
example, each 4K JSON object would be stored as one entity/blob. This entity 
will need to be a composite array (multiple arrays) for my use case enclosing 
it with [] i.e. ["value1", "value2", "value3"].

I was thinking to store each 4K blob in an in-memory db in JSON for easier 
composition/decomposition of the data, but this JSON takes a lot of extra space 
due to its inherent tags like {}, [], which I referred as a separator. For 1M 
records, the total memory size for my use case is then 4GB. However, my use case 
could go even higher, e.g. 64G. So, I was also looking into the JsonCpp library 
to see if there is any data compression capability added to it, but could not 
find anything. 


You are probably right that JSON is probably not a good choice for such a large 
in-memory storage. Any further thoughts?

Thanks,
Anath B. Mandal




________________________________
From: Baptiste Lepilleur <bap...@gm...>
To: Anath Bandhu Mandal <a_b...@ya...>
Cc: jsoncpp-devel <jso...@li...>; blep 
<bl...@us...>
Sent: Thu, 16 September, 2010 13:49:41
Subject: Re: Example code for Jsoncpp library


2010/9/15 Anath Bandhu Mandal <a_b...@ya...>

Hi
>
>I have been using the jsoncpp library, but having difficulties to understand how 
>to effectively read/write data using Json::Value objects. You have give one 
>example on http://jsoncpp.sourceforge.net/index.html, which does not explore 
>much of the classes on how are they should be used. For example, I want to store 
>about 4K of data with separator in one array, but this array should be added 
>for a million records for read/write. However, I don't find an effective to way 
>utilize this library due to limited example code.
>
>Could you please provide some example code on how to effective utilize the 
>member functions for Json::Value, Json::Reader, and Json::Writer please?

You can look at the code of the jsontestrunner that is used for testing 
reader/writer (e.g. it reads a json file, and rewrite it in json, as well as a 
custom format).

That being said, I don't think jsoncpp is a good choice of library for your use 
case: if I understood correctly (not sure what you mean by data with separator), 
you want to parse 4K * 1M values. Because the parser will load the whole json 
file in memory, a lot of memory is going to be wasted (I'm not even sure you 
will be able to fit the Json::Value tree in memory after conversion: 
4K*1M=4*10**9 => means about 4GB of memory required if each value needed only 1 
byte!). Writing should not be an issue as you can use the stream based writer 
that allow incremental writing to file. 

Baptiste.


>
>Thanks,
>AB Mandal
>