Re: [Exist-open] eXist-db repair and reindex process taking a very long time

[Lizzi, V. <Vin...@ta...>]

Hello eXist-db community,

By comparing the folders data\lucene and data\range to a previous backup, going by the number of files and total size of these folders, it looks like the reindexing process is about 50% done. The reindexing process has been running since August 13 and is using over 100 Gb of memory.

Is there any way to start eXist-db and allow it to go through its recovery process but stop it from reindexing database files?  I'm wondering if that could be a way to get the database operational again, and then I could manually run xmldb:reindex().

Thanks,
Vincent

_____________________________________________
Vincent M. Lizzi
Head of Information Standards | Taylor & Francis Group
vin...@ta...<mailto:vin...@ta...>



Information Classification: General
From: Lizzi, Vincent
Sent: Tuesday, September 3, 2024 11:36 AM
To: Exist-open <exi...@li...>
Subject: eXist-db repair and reindex process taking a very long time

Hello eXist-db community,

I've been monitoring an eXist-db database that is going through its automated recovery process, and am wondering if there is any way to get more information about its progress and how soon the process will finish.

This eXist-db has full text indexing and range indexes configured on several large collections. The EC2 server on which eXist-db is hosted had an outage. When eXist-db was restarted its automatic repair process began. That was about 3 weeks ago. Through Windows Resource Monitor I can see that the eXist-db process is reading from dob.dbx and structure.dbx and writing to structure.dbx and writing to files in the "lucene" and "range" folders, and the process is using about 6% of CPU consistently, and memory usage has increased gradually to over 100 GB. The last line in exist.log is still:

2024-08-13 17:02:59,710 [main] INFO  (NativeBroker.java [repair]:3692) - Reindexing database files ...

Is there any way to find out more what the process is doing, estimate when it will finish, or release any bottlenecks, without interrupting the process?

Thanks,
Vincent

______________________________________________
Vincent M. Lizzi
Head of Information Standards | Taylor & Francis Group
530 Walnut St., Suite 850, Philadelphia, PA 19106
E-Mail: vin...@ta...<mailto:vin...@ta...>
Web: www.tandfonline.com<http://www.tandfonline.com>

Taylor & Francis is a trading name of Informa UK Limited,
registered in England under no. 1072954

"Everything should be made as simple as possible, but not simpler."