Menu
▾
▴
Re: [Exist-open] Fwd: Problem files after a crash
|
From: Joe W. <jo...@gm...> - 2021-10-04 17:50:10
|
Hi Nick,
Great, now that you have a backup, could you please open an issue and, if
possible, attach the backup there? We would also benefit from knowing what
query fails.
Joe
On Tue, Sep 28, 2021 at 10:36 PM Nick Sincaglia <nsi...@nu...>
wrote:
> I found another example of this issue today. I had a file which was
> located here:
>
> /db/apps/core/message-store/data/DEP-42/20210912/193872978031-20210912232055722.xml
>
> My scripts would fail when they tried to process it. I took a backup of
> the database. Then I opened the file in Oxygen, clicked the Format and
> Index icon and then saved it. My scripts processed this file without any
> problem.
>
> I have the database backup and would love to get some help in trying to
> explain what is wrong with this file or the indexes that is causing this
> problem.
>
> Nick
>
> On 9/27/21 9:07 AM, Nick Sincaglia wrote:
>
> Joe,
> I have experienced this issue on various different versions of eXist-db. I
> don't think it is specific to the version I am using.
>
> I have a variety of different eXist-db versions running for different
> clients. I usually try to get a new client using the latest version of
> eXist-db when I first start working with them and unless there is a good
> reason to upgrade, I usually keep them on that version for a while. Every
> new version of eXist-db comes with the potential risk that something that
> was working might break in the new version. So, I am reluctant to upgrade
> unless it there is a clear benefit.
>
> The issue I described is caused by the initial crash. It is just not clear
> to me why the re-index does not address the issue. It is hard to find these
> files as well. Usually I find them because the system is throwing an
> unexplained error. It would be helpful to be able to detect these files
> after a crash to proactively identify them before they cause the system to
> error in unexplained ways.
>
> Nick
>
> On 9/26/21 8:13 PM, Joe Wicentowski wrote:
>
> Hi Nick,
>
> I don't have any suggestions, except perhaps one: Have you considered
> upgrading to eXist 5? It's got a lot of fixes to locking and has been very
> stable for us in production.
>
> Joe
>
> On Sun, Sep 26, 2021 at 5:57 PM Nick Sincaglia <nsi...@nu...>
> wrote:
>
>> I brought this up on the last community call but I wanted to send the
>> e-mail to see if the wider community audience might have some suggestions.
>>
>> I had my eXist-db v4.5 crash a number of weeks ago. I restarted the
>> system, it ran through the data integrity check and everything seemed to be
>> OK. However, later, I noticed one of the files in my system that was not
>> indexed correctly. I could tell this because my file processing script
>> could find the file in the system but I was getting an error that did not
>> make any sense.
>>
>> So, what I decided to do next was to reindex the entire database
>> (xmldb:reindex('/db')). I tried running my script again and I still got the
>> same error. So, next I located the file in the system, opened up the file
>> and made a minor change to the XML file (The small change I made was I
>> added a white space to the end of the file, but I could have made any
>> change.) Then I saved the file. After that, I ran my script and everything
>> worked as expected.
>>
>> What I believe I understand about this issue is that there is something
>> that happens when you save a file, which forces a re-index of a file, which
>> is different then if you run xmldb:reindex(). Also, I have experimented
>> with downloading a collection of documents, then deleting the collection in
>> the database and then uploading the collection to the database using
>> WebDAV. This technique also does not solve the issue. There is something
>> about opening the file, changing it and then saving it that resolves this
>> issue. I was posting this because I see this issue every 6 months or so.
>> Whenever, I am completely stumped by a script not running properly, I have
>> learned to locate the file, modify it in a minor way and resave it and very
>> frequently it resolves my issue.
>>
>> When I brought this up on the community call, I was given 2 suggestions.
>> First, they thought maybe the XML files had a BOM character in them. This
>> is not the case for me because we are not working with any Windows servers.
>> I have seen this BOM character issue before and so I can confidently say,
>> this is not related to a BOM character.
>>
>> The second suggestion was that maybe our index was indexing an attribute
>> instead of a tag. I checked my index and I don't think this is the case. My
>> index file is below.
>> <collection xmlns="http://exist-db.org/collection-config/1.0"
>> <http://exist-db.org/collection-config/1.0>>
>> <index>
>> <!-- Disable the old full text index -->
>> <fulltext default="none" attributes="false"/>
>> <!-- New full text index based on Lucene indexes for full names
>> and song titles -->
>> <lucene>
>> <analyzer
>> class="org.apache.lucene.analysis.standard.StandardAnalyzer"/>
>> <analyzer id="ws"
>> class="org.apache.lucene.analysis.WhitespaceAnalyzer"/>
>> </lucene>
>> <!-- Range indexes for fast ID lookups -->
>> <range>
>> <create qname="next-activity" type="xs:string"/>
>> <create qname="data-exchange-id" type="xs:string"/>
>> <create qname="data-origin-party-id" type="xs:string"/>
>> <create qname="data-destination-party-id" type="xs:string"/>
>> <create qname="release-id" type="xs:string"/>
>> <create qname="search" type="xs:string"/>
>> <create qname="delivery-type" type="xs:string"/>
>> <create qname="queue-date-time" type="xs:string"/>
>> <create qname="delivery-date-time" type="xs:string"/>
>> <create qname="workflow-status" type="xs:string"/>
>> <create qname="event-timestamp" type="xs:dateTime"/>
>> </range>
>> </index>
>> </collection>
>>
>> Any thoughts or suggestions are welcome!
>>
>> Nick
>>
>> --
>> Nick Sincaglia
>> President/Founder
>> NueMeta, LLC
>> Digital Media & Technology
>> Phone: +1-6...@nu... http://www.nuemeta.com
>> Skype: nsincaglia
>>
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
> --
> Sent from my iPhone
>
>
> --
> Nick Sincaglia
> President/Founder
> NueMeta, LLC
> Digital Media & Technology
> Phone: +1-6...@nu... http://www.nuemeta.com
> Skype: nsincaglia
>
>
> --
> Nick Sincaglia
> President/Founder
> NueMeta, LLC
> Digital Media & Technology
> Phone: +1-6...@nu... http://www.nuemeta.com
> Skype: nsincaglia
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
|
