Error after Unexpected shutdown with 2GB Database

Help
Marc
2012-12-26
2013-09-26
  • Marc

    Marc - 2012-12-26

    Hello, I have read an older thread regarding Error com.db4o.ext.db4oexception after an unexpected shutdown. Log Detail: An error has occurred. See error log for more details.
    'C:\Documents and Settings\Marc.rssowl2.metadata.plugins\org.rssowl.core\rssowl.db' is closed. close() was called or open() failed.
    I read as well the advice presented in the FAQ. Problem is there is no backup file of my current 2GB database worth months if not one year of information. RSSOwl is able to start and displays the feeds in the feed list tab with their corresponding unread news count, however is unable to open the feeds in the editor nor saved searches. There's an rssowl.db.onlinebak.weekly file but as judging from earlier posts and the file size (1,5 GB) is not exactly a backup. I'm trying to keep up with the christmas spirit...

    --
    Full Log "Could not open the editor: An exception was thrown during initialization"

    com.db4o.ext.Db4oException: 'C:\Documents and Settings\Marc.rssowl2.metadata.plugins\org.rssowl.core\rssowl.db' is closed. close() was called or open() failed.
    at com.db4o.internal.Exceptions4.throwRuntimeException(Unknown Source)
    at com.db4o.internal.Exceptions4.throwRuntimeException(Unknown Source)
    at com.db4o.internal.Exceptions4.throwRuntimeException(Unknown Source)
    at com.db4o.internal.PartialObjectContainer.checkClosed(Unknown Source)
    at com.db4o.internal.PartialObjectContainer.checkTransaction(Unknown Source)
    at com.db4o.internal.PartialObjectContainer.query(Unknown Source)
    at com.db4o.internal.PartialObjectContainer.query(Unknown Source)
    at org.rssowl.core.internal.persist.dao.PreferencesDAOImpl.loadAll(PreferencesDAOImpl.java:120)
    at org.rssowl.core.internal.persist.dao.PreferencesDAOImpl.load(PreferencesDAOImpl.java:112)
    at org.rssowl.core.internal.persist.pref.GlobalScope.load(GlobalScope.java:151)
    at org.rssowl.core.internal.persist.pref.GlobalScope.getInteger(GlobalScope.java:168)
    at org.rssowl.core.internal.persist.pref.EntityScope.getInteger(EntityScope.java:108)
    at org.rssowl.ui.internal.editors.feed.FeedView.loadSettings(FeedView.java:796)
    at org.rssowl.ui.internal.editors.feed.FeedView.init(FeedView.java:465)
    at org.eclipse.ui.internal.EditorManager.createSite(EditorManager.java:799)
    at org.eclipse.ui.internal.EditorReference.createPartHelper(EditorReference.java:643)
    at org.eclipse.ui.internal.EditorReference.createPart(EditorReference.java:428)
    at org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:594)
    at org.eclipse.ui.internal.PartPane.setVisible(PartPane.java:306)
    at org.eclipse.ui.internal.presentations.PresentablePart.setVisible(PresentablePart.java:180)
    at org.eclipse.ui.internal.presentations.util.PresentablePartFolder.select(PresentablePartFolder.java:270)
    at org.eclipse.ui.internal.presentations.util.LeftToRightTabOrder.select(LeftToRightTabOrder.java:65)
    at org.eclipse.ui.internal.presentations.util.TabbedStackPresentation.selectPart(TabbedStackPresentation.java:473)
    at org.eclipse.ui.internal.PartStack.refreshPresentationSelection(PartStack.java:1256)
    at org.eclipse.ui.internal.PartStack.setSelection(PartStack.java:1209)
    at org.eclipse.ui.internal.PartStack.showPart(PartStack.java:1608)
    at org.eclipse.ui.internal.PartStack.add(PartStack.java:499)
    at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:103)
    at org.eclipse.ui.internal.PartStack.add(PartStack.java:485)
    at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:112)
    at org.eclipse.ui.internal.EditorSashContainer.addEditor(EditorSashContainer.java:63)
    at org.eclipse.ui.internal.EditorAreaHelper.addToLayout(EditorAreaHelper.java:217)
    at org.eclipse.ui.internal.EditorAreaHelper.addEditor(EditorAreaHelper.java:207)
    at org.eclipse.ui.internal.EditorManager.createEditorTab(EditorManager.java:779)
    at org.eclipse.ui.internal.EditorManager.openEditorFromDescriptor(EditorManager.java:678)
    at org.eclipse.ui.internal.EditorManager.openEditor(EditorManager.java:639)
    at org.eclipse.ui.internal.WorkbenchPage.busyOpenEditorBatched(WorkbenchPage.java:2817)
    at org.eclipse.ui.internal.WorkbenchPage.busyOpenEditor(WorkbenchPage.java:2729)
    at org.eclipse.ui.internal.WorkbenchPage.access$11(WorkbenchPage.java:2721)
    at org.eclipse.ui.internal.WorkbenchPage$10.run(WorkbenchPage.java:2673)
    at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
    at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2668)
    at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2652)
    at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2643)
    at org.rssowl.ui.internal.OwlUI.internalOpenInFeedView(OwlUI.java:1867)
    at org.rssowl.ui.internal.OwlUI.openInFeedView(OwlUI.java:1780)
    at org.rssowl.ui.internal.OwlUI.openInFeedView(OwlUI.java:1766)
    at org.rssowl.ui.internal.OwlUI.openInFeedView(OwlUI.java:1754)
    at org.rssowl.ui.internal.OwlUI.openInFeedView(OwlUI.java:1743)
    at org.rssowl.ui.internal.actions.OpenAction$1.run(OpenAction.java:63)
    at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
    at org.rssowl.ui.internal.actions.OpenAction.run(OpenAction.java:59)
    at org.eclipse.jface.action.Action.runWithEvent(Action.java:498)
    at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:583)
    at org.eclipse.jface.action.ActionContributionItem.access$2(ActionContributionItem.java:500)
    at org.eclipse.jface.action.ActionContributionItem$5.handleEvent(ActionContributionItem.java:411)
    at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
    at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1003)
    at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3823)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3422)
    at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2384)
    at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2348)
    at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2200)
    at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:495)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:288)
    at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:490)
    at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
    at org.rssowl.ui.internal.Application.start(Application.java:119)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:193)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:386)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1236)

    Edit: Added com.db4o.ext.Db4oException Log

     
    Last edit: Marc 2012-12-27
  • Marc

    Marc - 2012-12-27

    Is it possible to manually repair the database? If is not overkill can I get some guidance?
    Also is it contempled to add a repair function to RSS Owl?

     
  • sg2002

    sg2002 - 2012-12-27

    Hello. I have a script that should help you, it's just not compiled right now, wait a few days, I'm gonna post it.

     
  • Marc

    Marc - 2012-12-31

    That'd be great! When you post it please link it here so that I can try it right away / receive the notification.

     
    Last edit: Marc 2012-12-31
  • sg2002

    sg2002 - 2013-01-02

    Hello. Here it is.

    This jar moves data from one rssowl database to another. Just feeds and items in them(and their read states), not folders or views.

    To use it just call it like this:
    java -Xmx3072m -jar feedtransfer.jar d:/brokendb/rssowl.db d:/cleandb/rssowl.db

    First argument is the database from which you want to export feeds, the second is the destination, which in your case should be a clean database.

    -Xmx3072m is needed since you have such a large database and I didn't optimize my code property. :-) Also you should be running a 64bit machine and 64-bit java on it.

    This script is really-really slow and in your case would probably take quite a few hours to finish. Dunno, probably somewhere around five.

     
  • Marc

    Marc - 2013-01-03

    Thanks Sg2002! I don't have any 64-bit installation at the time, could it work on a 32-bit one / with 4GB RAM?
    Another question, as I had been running RSSOwl using PortableApps's Portable Java, should/can I run the jar file using "jPortable Launcher"
    or can I run it by editing RSSOwl.ini in the executable folder?

    I have attempted the following:
    Running it from jPortable following this indications:
    http://portableapps.com/node/30058
    I tried adding these lines to JavaPortableLauncher\Data\JavaPortableLauncher.ini

    [Launch]
    ProgramExecutable=javaw.exe

    CommandLineArguments=-client -Xmx3072m "C:\Documents and Settings\Marc\.rssowl2.metadata.plugins\org.rssowl.core\rssowl.db" V:\rssowl.db

    Running it via RSSOwl adding these lines to RSSOwl.ini
    -vmargs
    -Xms15m
    -Xmx3072m
    -jar C:\rssowl-2.1.2.windows\rssowl\feedtransfer.jar C:\Documents and Settings\Marc\.rssowl2.metadata.plugins\org.rssowl.core\rssowl.db V:\rssowl.db
    -Dosgi.requiredJavaVersion=1.5
    -Djava.util.Arrays.useLegacyMergeSort=true

    As your example used "/" instead of"\" to specify paths, but the example in PortableApps used the later I tried with both.
    I apologize for the noobish questions. Most likely I'm making a mistake in the syntax, or it just can't be run in 32-bit systems, or either way I tried can't properly run the script and I have to install Java. :that emoticon that hits himself in the head:

     
  • sg2002

    sg2002 - 2013-01-03

    Hello Mark.

    Don't know about portable Java, gonna maybe try it later.

    You probably shouldn't use RSSOwl.ini.

    Just run it from the command line.

    The script should work for 32-bit systems, but it's really RAM intensive(reads the whole db first). And the memory limit for 32-bit Java on Windows is around 1500m, so it's not enough for you db.

    Also the problem may be not in your database, but in RSSOwl itself, since in my case it became really unstable just after maybe a 600mb's db, so it was probably crashing all the time for you and it should also be terribly slow. The only solution to that is running a custom 64-bit build of RSSOwl and that also requires a 64-bit machine.

     
  • Marc

    Marc - 2013-01-24

    Hello Sg2002 and thanks for clearing things out. It took me a while to get me started but I finally set a dual boot custom windows 7 64-bits and run the script via the command line.
    After it finished reading the feeds it started reporting "C:\Users\Marc\Desktop\rssowl.db is closed. close<> was called or open<> fail.

     
  • Marc

    Marc - 2013-01-30

    Do I'm making a noobish/obvious mistake? It justs puzzless me that the script apparently "shares" the same problem with RSSowl sort of.
    I'm guessing here but, the corrupt portion of the database the one that produces the error if is the portion of the database that was being written/read/updated, shouldn't it account for only one feed, or a couple perhaps? If so, couldn't the corrupt portion be isolated; identify the feed it belongs to and delete the whole feed data from the database in order to sanitize it? Whether there is a solution or not thanks Sg2002 for all the help.

     
  • Marc

    Marc - 2013-03-04

    Alright, so I started from scratch hoping that eventually I may be able to regain access to my old database. After two moths with a database of 700 MB the same error is displayed after a restart of the application. Seriously an error that has been reported by more than one user, that makes the entire database inaccesible and which can be caused by a mere power outage or application restart, add to that that the database is so complex that there's no way for the users to export it. Damn this is the kind of issues that should be taken care. As long as this goes it's not possible to rely on RSSOwl.

     
  • Marc

    Marc - 2013-05-07

    After last update I tried again loading the 2GB database. This time using -Xmx1450m and -DR32 using PortableApps's Java Portable x64. I run the cleanup tool, and afterwards I got access back to the database and was able to browse feeds/news without any db4o errors.
    Not sure if related or not (a separate bug?), labeled items don't show up in a label search (label is)/label saved search. Newly added labeled items do show up on the other hand. So the the quick fix here is to re-add/label all labeled items. There are some other weird results in saved searches returning results from non specified feeds or without speficied search keywords, but I can finally access the database so that's the main deal.

    @sg2002 Can your script be used to merge two databases? Most likely not but...
    I'm not sure though if RSSOwl can handle a 3,5 GB database? I've upgraded both RAM and arquitecture so that shouldn't be a problem, but for example if I specify -Xmx>1500m RSSOwl returns an error.

    Also some thoughts, If I moved all news to archive would it reduce load/perform/work better with larger databases? My issue with archiving them is that is then separated from the feed.

     
    Last edit: Marc 2013-05-07
  • sg2002

    sg2002 - 2013-05-08

    Hello. Sorry I was away for a while plus didn't get any notifications from this thread.

    I can probably make you a script that merges two databases.

    The problem with RSSOwl is that it's still a 32bit application. And on Windows 32 bit java is limited to less than 1500m. That's why you're getting those exceptions.

    I can probably post 64-bit RSSOwl build, but it has some minor stability problems.

    3,5 is probably possible, if you have enough RAM.

     
  • Marc

    Marc - 2013-05-08

    Thanks Sg2002, No problem. I think it would be worth trying. From what I have read there are many users that have databases over 2GB.It is only a matter of time till 1500m is not enough?

    "3,5 is probably possible, if you have enough RAM."
    Will 4GB do?

     
  • sg2002

    sg2002 - 2013-05-09

    I think it all depends on how you use RSSOwl. As long as you don't have extra-large saved searches\folders that you wish to view, you should be fine.

    My experience is as folows: I have a 1GiB database with about 200k unread items. When loaded RSSOwl uses about 700m of memory. I also have an extra-large saves search that is about 150k of items. If I try to open it using some form of classic view - RSSOwl crashes, since it has to show all the feed titles in the list. So, my only option is using newspaper view. Newspaper view works fine, but it's really memory hungry in this case. When I open my 150k view, RSSOwl immediately jumps to 2.2GiB of used memory. And that's more than 32bit RSSOwl can handle.

    Anyway, here's my earlier script updated to support better merge of RSSOwl databases. The usage should be the same as before. I've tested it with some toy samples. but with nothing really serious.

     
  • Anonymous - 2013-05-11

    There are some tips here (http://www.rssowl.org/help#item_6g) how to reduce memory. But in general, 150k news will be hard for rssowl to display. Its fine to have 150k news in the DB, but its problematic to display it at once. The newspaper view uses lazy loading of news, so it works better.

    In general I suggest to use clever saved searches and adjust display to only show news from e.g. the past week. The less news rssowl has to show at once, the better.

     
  • Marc

    Marc - 2013-05-19

    Thanks Sg2002. So far: first times I run the script using -Xmx3072m argument, and after 30min/1h the script closed itself.
    I increased the max memory to -Xmx3572m (since 1,5 GB +2 GB). The script ran then for 3 days and then finished /closed without any change to the database files. A possibility is that at the time/end of the 3 days there wasn't 3,5 GB of free space in that drive, I don't know how it works into that regard; perhaps that isn't an issue (if it moves data from the old database to the new ones without creating an additional file.
    Bottom line: I might have to re-run the script making sure there are over 3,5 GB of free space on the drive.

    I've all saved searches/most feeds limited to last 5 days as it is suggested in the FAQ/recommendation. I use mostly list display mode while newspaper view for picture rss feeds such as deviantart ones.
    Perhaps lazy mode could be used as well in other display modes?

    I wonder what happens with the base memory that RSSOwl uses (minimized with no feeds opened). The 2 GB database uses around 600k, the 1,5 GB around 500k. If both databases are merged will I end up with a big 1100k of base used memory.

     
    Last edit: Marc 2013-05-19
  • Anonymous - 2013-07-17
    Post awaiting moderation.
  • Anonymous - 2013-07-17
    Post awaiting moderation.
  • sg2002

    sg2002 - 2013-09-26

    Hmm, I think I ran into a similar problem. In my case db got corrputed with some category name becaming null, which should not really happen. Restored from backup, since I don't have time to tinker now, but if that happens again - gonna go fill a proper bug report. ;-)

    Mark, that script dies when db gets corrputed. Have you seen what was the message when it closed?

    Anyway, I'm planning to propertly rewrite it some time soon, to get better performance, since waiting long hours with those shiny new multithreaded computers is a bit lame.

    BTW, what is in gina's posts? Spam?

     
    Last edit: sg2002 2013-09-26

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks