Menu
▾
▴
Re: [Exist-open] Fwd: Problem files after a crash
|
From: <ai...@un...> - 2021-09-29 05:45:18
|
I can support Nick's findings.
One of our resources shows the same or similar behaviour.
Occasionally, a resource cannot be displayed (browser loads the page
(betterform) without error but 'shows' unbound fields).
The file can be loaded in eXide and looks perfectly normal.
After staring at it und saving the file loads and displays as expected.
Editing is not needed,
Simple Re-indexing, however, does not help.
I didn't try to access the file via scripting.
As an amateur I imagined that the 'memory representation' of the
resource in eXistDB is altered in some way??
I didn't report an issue because I could not isolate a test case and
we are using an 'ancient' eXistDb v4.5 with betterform.
reagrds
Peter
Ubuntu 18, eXistDb 4.5, java8_181
Zitat von Nick Sincaglia <nsi...@nu...>:
> I found another example of this issue today. I had a file which was
> located here:
> /db/apps/core/message-store/data/DEP-42/20210912/193872978031-20210912232055722.xml
>
> My scripts would fail when they tried to process it. I took a backup
> of the database. Then I opened the file in Oxygen, clicked the
> Format and Index icon and then saved it. My scripts processed this
> file without any problem.
>
> I have the database backup and would love to get some help in trying
> to explain what is wrong with this file or the indexes that is
> causing this problem.
>
> Nick
>
> On 9/27/21 9:07 AM, Nick Sincaglia wrote:
>> Joe,
>> I have experienced this issue on various different versions of
>> eXist-db. I don't think it is specific to the version I am using.
>>
>> I have a variety of different eXist-db versions running for
>> different clients. I usually try to get a new client using the
>> latest version of eXist-db when I first start working with them and
>> unless there is a good reason to upgrade, I usually keep them on
>> that version for a while. Every new version of eXist-db comes with
>> the potential risk that something that was working might break in
>> the new version. So, I am reluctant to upgrade unless it there is a
>> clear benefit.
>>
>> The issue I described is caused by the initial crash. It is just
>> not clear to me why the re-index does not address the issue. It is
>> hard to find these files as well. Usually I find them because the
>> system is throwing an unexplained error. It would be helpful to be
>> able to detect these files after a crash to proactively identify
>> them before they cause the system to error in unexplained ways.
>>
>> Nick
>>
>> On 9/26/21 8:13 PM, Joe Wicentowski wrote:
>>> Hi Nick,
>>>
>>> I don't have any suggestions, except perhaps one: Have you
>>> considered upgrading to eXist 5? It's got a lot of fixes to
>>> locking and has been very stable for us in production.
>>>
>>> Joe
>>>
>>> On Sun, Sep 26, 2021 at 5:57 PM Nick Sincaglia
>>> <nsi...@nu... <mailto:nsi...@nu...>> wrote:
>>>
>>> I brought this up on the last community call but I wanted to send
>>> the e-mail to see if the wider community audience might have some
>>> suggestions.
>>>
>>> I had my eXist-db v4.5 crash a number of weeks ago. I restarted
>>> the system, it ran through the data integrity check and
>>> everything seemed to be OK. However, later, I noticed one of the
>>> files in my system that was not indexed correctly. I could tell
>>> this because my file processing script could find the file in the
>>> system but I was getting an error that did not make any sense.
>>>
>>> So, what I decided to do next was to reindex the entire database
>>> (xmldb:reindex('/db')). I tried running my script again and I
>>> still got the same error. So, next I located the file in the
>>> system, opened up the file and made a minor change to the XML
>>> file (The small change I made was I added a white space to the
>>> end of the file, but I could have made any change.) Then I saved
>>> the file. After that, I ran my script and everything worked as
>>> expected.
>>>
>>> What I believe I understand about this issue is that there is
>>> something that happens when you save a file, which forces a
>>> re-index of a file, which is different then if you run
>>> xmldb:reindex(). Also, I have experimented with downloading a
>>> collection of documents, then deleting the collection in the
>>> database and then uploading the collection to the database using
>>> WebDAV. This technique also does not solve the issue. There is
>>> something about opening the file, changing it and then saving it
>>> that resolves this issue. I was posting this because I see this
>>> issue every 6 months or so. Whenever, I am completely stumped by
>>> a script not running properly, I have learned to locate the file,
>>> modify it in a minor way and resave it and very frequently it
>>> resolves my issue.
>>>
>>> When I brought this up on the community call, I was given 2
>>> suggestions. First, they thought maybe the XML files had a BOM
>>> character in them. This is not the case for me because we are not
>>> working with any Windows servers. I have seen this BOM character
>>> issue before and so I can confidently say, this is not related to
>>> a BOM character.
>>>
>>> The second suggestion was that maybe our index was indexing an
>>> attribute instead of a tag. I checked my index and I don't think
>>> this is the case. My index file is below.
>>> <collection xmlns="http://exist-db.org/collection-config/1.0"
>>> <http://exist-db.org/collection-config/1.0>>
>>> <index>
>>> <!-- Disable the old full text index -->
>>> <fulltext default="none" attributes="false"/>
>>> <!-- New full text index based on Lucene indexes for full
>>> names and song titles -->
>>> <lucene>
>>> <analyzer
>>> class="org.apache.lucene.analysis.standard.StandardAnalyzer"/>
>>> <analyzer id="ws"
>>> class="org.apache.lucene.analysis.WhitespaceAnalyzer"/>
>>> </lucene>
>>> <!-- Range indexes for fast ID lookups -->
>>> <range>
>>> <create qname="next-activity" type="xs:string"/>
>>> <create qname="data-exchange-id" type="xs:string"/>
>>> <create qname="data-origin-party-id" type="xs:string"/>
>>> <create qname="data-destination-party-id"
>>> type="xs:string"/>
>>> <create qname="release-id" type="xs:string"/>
>>> <create qname="search" type="xs:string"/>
>>> <create qname="delivery-type" type="xs:string"/>
>>> <create qname="queue-date-time" type="xs:string"/>
>>> <create qname="delivery-date-time" type="xs:string"/>
>>> <create qname="workflow-status" type="xs:string"/>
>>> <create qname="event-timestamp" type="xs:dateTime"/>
>>> </range>
>>> </index>
>>> </collection>
>>>
>>> Any thoughts or suggestions are welcome!
>>>
>>> Nick
>>>
>>> -- Nick Sincaglia
>>> President/Founder
>>> NueMeta, LLC
>>> Digital Media & Technology
>>> Phone: +1-630-303-7035
>>> nsi...@nu... <mailto:nsi...@nu...>
>>> http://www.nuemeta.com <http://www.nuemeta.com>
>>> Skype: nsincaglia
>>>
>>> _______________________________________________
>>> Exist-open mailing list
>>> Exi...@li...
>>> <mailto:Exi...@li...>
>>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>> <https://lists.sourceforge.net/lists/listinfo/exist-open>
>>>
>>> --
>>> Sent from my iPhone
>>
>> --
>> Nick Sincaglia
>> President/Founder
>> NueMeta, LLC
>> Digital Media & Technology
>> Phone: +1-630-303-7035
>> nsi...@nu... http://www.nuemeta.com
>> Skype: nsincaglia
|
