[Exist-development] delete operations extremely slow for large collection

[Robby P. <rob...@ci...>]

Hi all,

Is it normal behaviour that a REST delete for 1 xml document in a collection from ~17.000 documents takes several minutes?

Below my stacktrace...  As you can see it takes about 2 minutes and 17 seconds before i  get back a response.

Might this be related to misconfiguration?  Is this due to the size of the collection? And are there ways to improve performance because this is unacceptable for my needs.

Kind regards,
Robby Pelssers

----------------------------------------------------
30 Sep 2010 17:53:29,616 [P1-8] DEBUG (RESTServer.java [doDelete]:1012) - removing document /db/released/ChemicalContent/NZX20A.xml
30 Sep 2010 17:53:29,632 [P1-8] DEBUG (RESTServer.java [doDelete]:1012) - removing document /db/released/ChemicalContent/NZX20A.xml
30 Sep 2010 17:53:29,632 [P1-8] INFO  (NativeBroker.java [removeXMLResource]:2222) - Removing document NZX20A.xml (18488) ...
30 Sep 2010 17:55:43,447 [P1-8] DEBUG (NativeTextEngine.java [dropIndex]:1072) - Removing 0 tokens
30 Sep 2010 17:55:43,447 [P1-8] DEBUG (NativeTextEngine.java [dropIndex]:1072) - Removing 0 tokens
30 Sep 2010 17:55:43,447 [P1-8] DEBUG (NativeTextEngine.java [dropIndex]:1072) - Removing 0 tokens
30 Sep 2010 17:55:43,447 [P1-8] DEBUG (NativeBroker.java [removeXMLResource]:2229) - removeDocument() - removing dom
30 Sep 2010 17:55:43,509 [P1-8] DEBUG (NativeBroker.java [removeResourceMetadata]:2326) - Removing resource metadata for 18488
30 Sep 2010 17:55:47,322 [P1-8] DEBUG (NGramIndex.java [sync]:86) - SYNC NGRAM
30 Sep 2010 17:55:47,322 [P1-8] INFO  (NativeBroker.java [sync]:3191) - Memory: 126,912K total; 506,816K max; 68,712K free
30 Sep 2010 17:55:47,322 [P1-8] INFO  (BTree.java [printStatistics]:2348) - dom.dbx INDEX Buffers occupation : 100% (1639 out of 1639) Cache efficiency
: 98%
30 Sep 2010 17:55:47,338 [P1-8] INFO  (DOMFile.java [printStatistics]:1434) - dom.dbx DATA Buffers occupation : 98% (252 out of 256) Cache efficiency :
99%
30 Sep 2010 17:55:47,338 [P1-8] INFO  (BTree.java [printStatistics]:2348) - collections.dbx INDEX Buffers occupation : 100% (96 out of 96) Cache efficie
ncy : 100%
30 Sep 2010 17:55:47,338 [P1-8] INFO  (BFile.java [printStatistics]:427) - collections.dbx DATA Buffers occupation : 99% (194 out of 195) Cache efficien
cy : 100%
30 Sep 2010 17:55:47,338 [P1-8] INFO  (BTree.java [printStatistics]:2348) - elements.dbx INDEX Buffers occupation : 8% (5 out of 64) Cache efficiency :
100%
30 Sep 2010 17:55:47,338 [P1-8] INFO  (BFile.java [printStatistics]:427) - elements.dbx DATA Buffers occupation : 100% (736 out of 737) Cache efficiency
 : 96%
30 Sep 2010 17:55:47,338 [P1-8] INFO  (BTree.java [printStatistics]:2348) - values.dbx INDEX Buffers occupation : 100% (64 out of 64) Cache efficiency :
 34%
30 Sep 2010 17:55:47,338 [P1-8] INFO  (BFile.java [printStatistics]:427) - values.dbx DATA Buffers occupation : 0% (0 out of 64) Cache efficiency : 94%

30 Sep 2010 17:55:47,338 [P1-8] DEBUG (Journal.java [checkpoint]:318) - Checkpoint reached
30 Sep 2010 17:55:47,338 [P1-8] DEBUG (NotificationService.java [debug]:76) - Registered UpdateListeners:
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] DEBUG (NGramIndex.java [sync]:86) - SYNC NGRAM
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] INFO  (NativeBroker.java [sync]:3191) - Memory: 126,912K total; 506,816K max; 68,083K free
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] INFO  (BTree.java [printStatistics]:2348) - dom.dbx INDEX Buffers occupation : 100% (1639 out
of 1639) Cache efficiency : 98%
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] INFO  (DOMFile.java [printStatistics]:1434) - dom.dbx DATA Buffers occupation : 98% (252 out o
f 256) Cache efficiency : 99%
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] INFO  (BTree.java [printStatistics]:2348) - collections.dbx INDEX Buffers occupation : 100% (9
6 out of 96) Cache efficiency : 100%
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] INFO  (BFile.java [printStatistics]:427) - collections.dbx DATA Buffers occupation : 99% (194
out of 195) Cache efficiency : 100%
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] INFO  (BTree.java [printStatistics]:2348) - elements.dbx INDEX Buffers occupation : 8% (5 out
of 64) Cache efficiency : 100%
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] INFO  (BFile.java [printStatistics]:427) - elements.dbx DATA Buffers occupation : 100% (736 ou
t of 737) Cache efficiency : 96%
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] INFO  (BTree.java [printStatistics]:2348) - values.dbx INDEX Buffers occupation : 100% (64 out
 of 64) Cache efficiency : 34%
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] INFO  (BFile.java [printStatistics]:427) - values.dbx DATA Buffers occupation : 0% (0 out of 6
4) Cache efficiency : 94%
30 Sep 2010 17:55:47,588 [exist_QuartzScheduler_Worker-3] DEBUG (Journal.java [checkpoint]:318) - Checkpoint reached