Re: [Exist-open] Heap Memory Problem with eXist behaviour

[<rma...@se...>]

Hi,

Wolfgang Meier wrote:
> Hi,
>
>>   Our problem is that eXist gradually takes heap memory and used 
>> memory is
>> increased more and more... And it is supposed that when a collection is
>> removed, used memory goes free a time later...
>
> Yes, though some things will remain in memory, e.g. the page caches
> and the collection caches. Anyway, we need some more information to
> give an advice.
>
> Which eXist version are you using? Are you working on an embedded or a
> remote database? Do you use XUpdate/XQuery update extensions to modify
> documents or are you just replacing them?
    We are using eXist 1.0.1 on an embedded mode in our application. We 
don't  modify documents. Our processing is based on:
    The application progressivelly generates a big amount of threads and 
each thread creates a collection and works with three or four documents. 
When each
thread finishes its work, it removes only the collection. So this 
proccesing implies a lof of create/remove operations on collections and 
create/query operations on documents.

>
>>   Jmap utility is used to watch heap memory after some execution 
>> time. An
>> example of histogram of classes is the following:
>>
>> Object Histogram:
>>
>> Size            Count   Class description
>> -------------------------------------------------------
>> 91962880        718460  org.exist.dom.DocumentImpl
>> 60633680        850782  char[]
>> 55273064        733191  int[]
>> 49442456        1437214 long[]
>> 24257568        758049  java.util.TreeMap$Entry
>> 22990720        718460  org.exist.storage.lock.MultiReadReentrantLock
>> 20354880        848120  java.lang.String
>> 17806800        741950  org.exist.security.Permission
>
> Well, it looks like you have at least 718460 documents in your
> database. Is that correct? If yes, eXist certainly needs quite an
> amount of memory to manage those documents. The Permission and the
> Lock objects belong to the document object. How many collections do
> you have for your 718460 documents?
In this heap photo, the application only has 4 collections with 16 
documents (4 x collection).

In accordance with the processing described above, the application 
creates a collection, stores four or five documents, makes some querys 
and finally remove the collection. If the remove operation is only 
applied on the collection, the documents stored in it are internally 
removed? or it is necessary to remove the stored documents in a 
collection and then the collection to clean up correctly?


>
> Wolfgang
>
Cheers,
Raymond