Menu

Stream writer

2007-06-11
2013-04-22
  • Christopher Dunn

    Writing a large file would be much more efficient if the writer operated on a stream rather than on a string.

    Currently, the entire file must exist in-memory as a string before it can be written to disk.  And since strings are memcpy'd as they are lengthened, an extra copy of the file exists in-memory while it is being generated.  So writing a 1GB JSON file requires 2GB of memory in addition to the memory required to store the jsoncpp datastructures (which can be another 1GB or so).

    If instead the writer operated on streams, the user could supply a stringstream to get a string when desired.

    At my company, we have to switch away from jsoncpp to alternative storage methods when the filesize exceeds several hundred MB because of the huge memory requirements.

     
    • Christopher Dunn

      I am adding StyledStreamWriter.

       

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.