[ omegat-Bugs-1627589 ] out of memory->quit has a bad effect on the project TM

[SourceForge.net <no...@so...>]

Bugs item #1627589, was opened at 2007-01-04 08:50
Message generated for change (Comment added) made by alex73
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=520347&aid=1627589&group_id=68187

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: CVS
Status: Open
Resolution: None
Priority: 9
Private: No
Submitted By: Jean-Christophe Helary (brandelune)
Assigned to: Nobody/Anonymous (nobody)
Summary: out of memory->quit has a bad effect on the project TM

Initial Comment:
I am using a big project to test the memory requirements.

After quitting while loading for lack of memory, I eventually killed enough applications to have the loading process last enough to reach a point where OmegaT will eventually try to update the project_save.tmx file (while proceeding with the loading of remaining TMX etc).

When OmegaT reaches that point, it seems its memory use gets over the limit and OmegaT quits without properly writing project_save.tmx.

The result is that a 3.6mb TMX is not updated, and the remaining file is a 0k project_save.tmx file.

I attach the relevant part of the log.

----------------------------------------------------------------------

>Comment By: Alex Buloichik (alex73)
Date: 2007-11-30 11:26

Message:
Logged In: YES 
user_id=458256
Originator: NO

  Seems to right way is to save TMX into some special file(like
project_save.tmx.new), and rename to project_save.tmx only after saving
finished.
  In this case we will create file "project_save.tmx.new", store data,
then rename "project_save.tmx" into "project_save.tmx.bak", then rename
"project_save.tmx.new" into "project_save.tmx".
  When OutOfMemory, or other exception was occured, we will now lose
project_save.tmx.


----------------------------------------------------------------------

Comment By: Henry Pijffers (henry_pijffers)
Date: 2007-01-09 21:17

Message:
Logged In: YES 
user_id=545103
Originator: NO

I think it's a good idea to disable automatic saving while the project is
still loading. What about manual saving though? It'll still crash OmegaT,
so should we allow that?

It'll happen less though, if Lucene gives us faster loading times, as
Maxym promised.

Another idea is to make the save method check for any exceptions. If one
is thrown, it could at least delete the new project_save.tmx and replace it
by the last backup.

----------------------------------------------------------------------

Comment By: Jean-Christophe Helary (brandelune)
Date: 2007-01-09 03:09

Message:
Logged In: YES 
user_id=915082
Originator: YES

Whatever the reason is, I think the implemented behavior could be more
user-friendly. I think there is a RFE about automatically using the backup
somewhere. In this particular case I think we need a few things:

-either have OmegaT make an estimation of the necessary memory/available
memory ration and declares before doing anything that memory won't be
sufficient and problems can occur (continue yes/no) kind of alert.

-and/or we have OmegaT automatically recognize the problem at the
following load: the current project_save.tmx is way smaller than anyone of
the backup memory and OmegaT proposes the user to select a backup memory to
replace the bad project_save.tmx.

That way we'd have a real use for backups.

In any case, there should be a security device at one point to _not_
attempt to save before the project is fully loaded. For big projects,
loading will take enough time that an automatic save will be triggered
while loading, thus adding to the memory load and leading to such problems.
If automatic save could be disabled while loading it seems we would run in
less trouble. What do you think ?

----------------------------------------------------------------------

Comment By: Henry Pijffers (henry_pijffers)
Date: 2007-01-08 23:22

Message:
Logged In: YES 
user_id=545103
Originator: NO

The log shows that the out-of-memory problem occurs while saving, i.e.
while writing to project_save.tmx. This usually leads to an empty file.

This is one of the reasons we use backups.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=520347&aid=1627589&group_id=68187