SourceForge has been redesigned. Learn more.
Close

processing large files bloats filesystem cache

2014-06-15
2014-06-21
  • Reinhard Nißl

    Reinhard Nißl - 2014-06-15

    Hi,

    when 7zipping for example a database (2 files, total size ~40 GB, compressed ~ 4GB), the windows file system cache is bloated with data from those 3 files, although nobody will need it after compression. Due to this bloat, other more relevant data got dropped from the cache, resulting in a performance degrade for applications that need them.

    What do you think about adding the flag FILE_FLAG_NO_BUFFERING to every CreateFile() call?

    BTW: I couldn't examine the source of 7.33alpha yet, in case this flag has already been added.

    Bye.

     
    • igg

      igg - 2014-06-21

      The FILE_FLAG_NO_BUFFERING flag unfortunately imposes significant restrictions on how the file can be accessed, so it may require big(ger) changes in the code.

      Also, you can't generally say the the compressed data won't be used after the compression - maybe it is so in your case, but may not be true for others.

       

Log in to post a comment.