So I got my accumulator application working, and I loved it.

Then I went to apply it to a large collection of files and ran into memory problems.

My accumulator is non-streaming (though I may be able to create a streamable version).

I assume one of two things is happening:
The existence of the accumulator prevents the documents from being discarded.
The documents are being successfully discarded, but the memory from the accumulator itself is not being released.

Is this expected behavior? Is there a way around it?



"A false conclusion, once arrived at and widely accepted is not dislodged easily, and the less it is understood, the more tenaciously it is held." - Cantor's Law of Preservation of Ignorance.