#9 issues with many small files

[david brown]

I found a sequence of problems with many small files of the same size. The attached .tgz file should demonstrate the issues.
1) the program crashed after an allocation failure; I reduced the buffer size of the cache held per file where the file is smaller than the cache;
2) once the allocation failure was addressed, the program failed to open files - complaining of too many open files. I modified file.cpp to close the underlying file when the file is read into the cache.

I am NOT 100% sure that these approaches are correct.

[david brown]

test case

[david brown]

i extracted the tar file to retest.. and the errors I saw situation is not immediately reproducible - sorry. I guess it is a mix of the changes I had to make to get the file small enough to attach.
I am attempting to reproduce now..

[david brown]

i can get the attached tar file to cause 'too many open files' errors by combining the extracted files and other larger files in my home directory. However there is no way that these files can fit in the upload limit. I might be able to write a script to generate data that tickles this bug.

I do not have a clear picture of what the cause is for the out of memory issue, i will look further tomorrow.