Menu
▾
▴
4 Attachments
#714 Git team project saving failure with 3.1.6
We use OmegaT 3.0.3 on Windows with Git team projects. One of our translators started using 3.1.6 to test it before upgrading the other translators.
After about one week once in the sudden all existing translation went missing because an empty project_save.tmx file was committed.
Another translator had the same project open with version 3.0.3 at the same time. The two versions were fighting with each other because the format of the TMX file was sliglthly different and lot of unecessary commits were made.
The attached omegat_failed_save.log has the relevant logs. Based on the logs it seems a "Git 'download'" failed before the the empty TMX file was committed (line 19).
15124: Info: Git 'reset' execution start (GIT_START)
15124: Info: Git 'reset' execution finished successfully (GIT_FINISH)
15124: Info: Git 'download' execution start (GIT_START)
15124: Error: Git 'download' error: Could not write file C:\OmegaT_Projects\Website_latest\omegat\._project_save.tmx830851676267507248.tmp (GIT_ERROR)
15124: FINE: GIT HEAD rev: 7a9fba68bd9a23fb67758ff68de6c1212209ca19
15124: FINE: GIT origin/master rev: 11b2fa8ad2b80f82fbfbcbd8b2fcb2c6b4dbec00
15124: FINE: GIT commonBase rev: 11b2fa8ad2b80f82fbfbcbd8b2fcb2c6b4dbec00
15124: FINE: rebaseProject: TMX head revision: 11b2fa8ad2b80f82fbfbcbd8b2fcb2c6b4dbec00
Description of the Git commit IDs:
7a9fba68bd9a23fb67758ff68de6c1212209ca19 last good commit by 3.1.6
11b2fa8ad2b80f82fbfbcbd8b2fcb2c6b4dbec00 last good commit by 3.0.3
6d84381f54a81f597ca378d9db0a83c6b98cedb0 empty TMX commit by 3.1.6
We also found about 1200 backup files with this pattern:
project_save.tmx-based_on_00101a96a3ab8926c363f9edc8586827605fe822.new
The attached backup_files.zip includes the full list of files.
There were other errors and expcetions related to saving the project (omegat_other_errors.log).
The complete log file is also attached as OmegaT.20141002.1616.zip.
Because of this issue we stopped using 3.1.6. We'd like to use the latets version because of the "Register identical translation" feature.
If there is any other information you would need to help resolving this issue please let me know. Thank you for the great work you do on OmegaT!
Richard
If two versions were fighting with each other, it shouldn't be a real problem. It will just increase counts of commits.
I could say much more with ability to see git history only, but looks like source of problem is problem of disk writing. There are many errors about that. OmegaT reports errors, save backups, .new files, but probably error handling is not ideal in case of automatic saving.
It will be good to change error handling in Omegat, but it required big discussion and big changes.
That said, I have very frequently
19741: Erreur: java.io.IOException: Error rename new file to tmx
19741: Erreur: at org.omegat.core.data.RealProject.rebaseProject(RealProject.java:989)
19741: Erreur: at org.omegat.core.data.RealProject.saveProject(RealProject.java:682)
19741: Erreur: at org.omegat.core.data.RealProject.saveProject(RealProject.java:654)
19741: Erreur: at org.omegat.core.threads.SaveThread.run(SaveThread.java:90)
(This is under Subversion, Windows 7).
It doesn't prevent working, but it's certainly not normal having that much writing errors.
It has not changed since 3.0.3, and I get the same behaviour with 3.1.7.
Didier
I'm not worried about the occasional errors if OmegaT can recover from them, but wiping out the translation memory as a result of a file writing error seems a more serious issue.
The other problem is the accumulating backup files. With 3.0.3 only 10 backup files were kept. With 3.1.6 for some reason these temporally files are left behind:
project_save.tmx-based_on_00101a96a3ab8926c363f9edc8586827605fe822.new
The user had 1200 of them in the omegat folder and may be this was the trigger for the TM wipe out incident.
Is there something I can do to help to narrow this done? I'd really like to start using 3.1.6 with the "Register identical translation" feature.
Richard
Of course. I was not contesting it, just mentioning the fact that file errors happen even in other circumstances than yours.
So far, I don't see what changes between 3.0.3 and 3.1.6 could have triggered such a different behaviour.
For completeness, did you try 3.1.6 on another machine?
Didier
I did more testing and I was able to reproduce the creation of the .new files in the 'omegat' folder with version 3.0.9 or newer.
When more users keep the same team project open each time a user makes a change and a new commit is made one or more .new files get created in the omegat folder of the project for the other user(s). Earlier versions of OmegaT (3.0.3 - 3.0.8) never left any .new files behind during my tests.
The extra commits made by competing different versions of OmegaT were only triggers of the issue.
It seems that this change in 3.0.9 introduced the problem:
Test setup:
User1 and user2 both using OmegaT 3.0.9 or newer
I attached the logs of user1 and user2, but this time I don't see any errors or exceptions.
If I can help with further testing or additional details please let me know.
Richard
Thank you, Richard.
I'm not sure that RFE#950 introduce this issue. It can affect on the ".new" files leave, but it's not a real problem.
I found very strange behavior of status() calling of jgit. Sometimes(I don't understand when and why) it reports about project_save.tmx is not modified. But command-line git reports that project_save.tmx was modified.
It looks like jgit library issue. I wrote to developers.
We can try to make workaround - just set GitRemoteRepository.isChanged() to return always true. But it requires some additional investigation.
P.S. 3.2 will not have such issue because it will compare files manually.
Can you try /trunk?
A change was made in revision 6731, which may solve your issue in most cases.
Didier
Unfortunately, revision 6731 didn't change anything in my test setup.
Each time user1 changes the project a .new file gets created with the SHA1 ID of the previous commit in the omegat folder of user2's project folder. And the same the other way around.
Would migrating to a newer JGit release be a big job? JGit is at version 3.5 which includes a large number of fixes and improvements compare to 2.1.
Richard
Don't worry about .new files. Do you see any translation lost in the 6731 ?
Could you try to use jgit 3.5(you can just rename it into 2.1 and put into your OmegaT).
We have a fairly large project and the project_save.tmx is already 6MB. It is hard to ignore that every few minutes a 6MB file gets created endlessly. When the TM was emptied out the translator had 1200 of these files. I guess the large number of files in the folder may played a roll in triggering the error. I was never able to reproduce the empty TM error.
I could give the translators some script to clean these files, but I don't think that is the right solution.
I did try JGit 3.5 and to my surprise it worked in my short test. However, the issue with the .new files remains.
Richard
The .new file issue should be unrelated to the main issue (I believe the former should be fixed in r6758).
Related to this issue, Alex removed the git isChanged() check when uploading; this is causing spurious glossary checkins on my projects. Any ideas for a workaround?
The issue should be fixed in SVN (/trunk).
Didier
I can confirm that the .new files are no longer accumulating in my test setup. Thank you for your help!
Richard
Fixed in the released version 3.1.8 of OmegaT.
Didier