From: Casey J. <cas...@jo...> - 2012-07-27 15:12:45
|
Ok, we have some idea's as to why this might be happening. Can you please describe the process you run on your files in more detail? For instance: - How many documents/collections does it run over? - Is the code Java or XQuery? - Did you see a relationship between the files which this code was run on and the effected files? - Are there any other operations which access a large amount of files? If you can share the code or parts of the code that would be very helpful. Cheers, Casey On Thu, Jul 26, 2012 at 5:41 PM, Jens Østergaard Petersen <oe...@gm... > wrote: > Hi Casey, > > I have looked a little into a recent backup (Dulip and I work at the same > place). > > I know of no crashes with the present installation, but we have > a __lost_and_found__ in the backup with a lot a of empty __contents__.xml, > as well as 72 empty records that come from (have same names as files from) > various collections (most from collection B mentioned below). All our files > are named with UUIDs, therefore we can track them even if they don't stay > put. > > The 40 empty (zero bytes) records that were placed wrongly in collection A > all came from collection B. (Collection B has more than 10,000 records, if > that is relevant.) There are no other collections with empty records in > their backup. > > We noticed the 40 empty records when trying to import collection A - empty > records are of course not well-formed, so the Java client protested > ("premature end of file", I think it was). We wanted to import the latest > backup since we were not able to access the collection using the admin > client or oXygen. I assume that this was because it contained empty > records. The collection responded well to searches, but we did not test all > 900+ records, so probably the 40 missing ones were - missing. > > All files with empty counterparts had ACEs - except one, which was in a > collection the parent of which had an ACE, if that is relevant. > > It is not like 40 extra (empty) records were aded to collection A. 40 > records were missing from collection A, as I can see from comparing > backups. The missing records do not have the same names as the empty > records - as I mentioned above, the files are named with UUIDs - and we > have not run out of them yet! > > None of the 40 records, missing or extra, were listed in > the __contents__.xml file in the backup of collection A. It contained > only <resource> entries for the full extant files. > > So 40 records disappeared from collection A and 40 records were copied > from collection B into A, but emptied of contents. Does this make sense? > Why the "neatness" of 40/40? > > Cheers - all the same, > > Jens > > On Jul 26, 2012, at 7:55 PM, Casey Jordan wrote: > > Dulip, > > Can you describe the type of operations which were occurring around the > time you noticed this? > > > Log output, output from consistency check or errors you get when trying to > start the database would help. > > Likely one of the core team will need to give you further instruction, > however we have had similar issues lately so it would be useful to > correlate. > > Thanks, > > Casey > > On Thu, Jul 26, 2012 at 1:52 PM, Withanage, Dulip < > wit...@as...> wrote: > >> Hi Casey, >> I have >> java -Xms2128m -Xmx5002m >> Revision: 16852 >> >> I have one more thing that perhaps will be helpful to find the cause. >> Some of the files were moved to a different collection, leaving a zero >> byte file. I can assure that I did not try to manually move them. >> In the newer location , the dfiles are present. >> I can make you some files available , if you like to have a thorough look. >> >> Best, >> Dulip >> >> >> >> >> >> >> --------------------------------------- >> Dulip Withanage >> Senior Software developer. >> Karl Jaspers Center >> University of Heidelberg >> Germany >> ________________________________________ >> From: Casey Jordan [cas...@jo...] >> Sent: Thursday, July 26, 2012 5:53 PM >> To: Withanage, Dulip >> Cc: exi...@li... >> Subject: Re: [Exist-open] eXist database collection corrupt / inconsistent >> >> Can you please provide eXist version, as well as what memory or >> configuration settings you are using that are different from the default. >> >> Thanks, >> >> Casey >> >> On Thu, Jul 26, 2012 at 5:20 AM, Dulip Withanage < >> wit...@as...<mailto: >> wit...@as...>> wrote: >> Hi all , >> >> My eXist database has a collection, which is not accessible with java >> client , web interface or external programs like oxygen. >> Although I can access its files using, xquery commands. >> >> This happened to me a long time ago and I got the impression that the >> JVM did not have enough memory for this to occur. Although now it has >> been assured that the JVM is set enough, this has happened again. I >> see nothing in the logs. >> >> Can somebody help give me tip, where and how to debug this problem? >> >> >> Dulip >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Exist-open mailing list >> Exi...@li...<mailto:Exi...@li...> >> https://lists.sourceforge.net/lists/listinfo/exist-open >> >> >> >> -- >> -- >> Casey Jordan >> easyDITA a product of Jorsek LLC >> "CaseyDJordan" on LinkedIn, Twitter & Facebook >> (585) 348 7399 >> easydita.com<http://easydita.com> >> >> >> This message is intended only for the use of the Addressee(s) and may >> contain information that is privileged, confidential, and/or exempt from >> disclosure under applicable law. If you are not the intended recipient, >> please be advised that any disclosure copying, distribution, or use of >> the information contained herein is prohibited. If you have received >> this communication in error, please destroy all copies of the message, >> whether in electronic or hard copy format, as well as attachments, and >> immediately contact the sender by replying to this e-mail or by phone. >> Thank you. >> >> > > > -- > -- > Casey Jordan > easyDITA a product of Jorsek LLC > "CaseyDJordan" on LinkedIn, Twitter & Facebook > (585) 348 7399 > easydita.com > > > This message is intended only for the use of the Addressee(s) and may > contain information that is privileged, confidential, and/or exempt from > disclosure under applicable law. If you are not the intended recipient, > please be advised that any disclosure copying, distribution, or use of > the information contained herein is prohibited. If you have received > this communication in error, please destroy all copies of the message, > whether in electronic or hard copy format, as well as attachments, and > immediately contact the sender by replying to this e-mail or by phone. > Thank you. > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. > http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ > Exist-open mailing list > Exi...@li... > https://lists.sourceforge.net/lists/listinfo/exist-open > > > -- -- Casey Jordan easyDITA a product of Jorsek LLC "CaseyDJordan" on LinkedIn, Twitter & Facebook (585) 348 7399 easydita.com This message is intended only for the use of the Addressee(s) and may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, please be advised that any disclosure copying, distribution, or use of the information contained herein is prohibited. If you have received this communication in error, please destroy all copies of the message, whether in electronic or hard copy format, as well as attachments, and immediately contact the sender by replying to this e-mail or by phone. Thank you. |