#836 Changes to segments get lost in team projects randomly and silently

[Roman Mironov]

Version 4.0: Changes to segments get lost in team projects randomly and silently, whether it is a new translation or a change to an existing one. Kos suggested this could be an issue with a temporary buffer where OmegaT stores segment changes before committing them to the project_save.tmx.

[Kos Ivantsov]

It was my wild guess that somehow those segments that were stored in OmegaT buffer but not yet saved to project_save.tmx, get lost. I've noticed segments lost after using merge and split script which forces project to reload. A few segment translations that were entered before the reload just disappeared.

[Didier Briel]

I confirm I sometimes seem to lose a few segments when reloading the project with F5.

Didier

[Mario Liebisch]

Same problem here. The worst part about this is that the segments will show your translation, they just aren't saved or tallied into the statistics (they appear translated but aren't saved). If you export target documents, they'll still contain the untranslated segments.

This might be related to a previous bug I remember from the 3.x branch where your current segment's translation would be cleared if you happen to edit it while the project is synced.

However I couldn't find any pattern for this branch. It happens randomly and unpredictable. Sometimes it's just one segment out of 100, sometimes it happens with 4-5 segments in a row, no matter whether they're written by hand or just taken from fuzzy matches using Ctrl+R.

[Kos Ivantsov]

I've been trying to keep my eyes open to the issue, and it clearly happens after sync. Reloading the project just triggers the sync, and therefore this issue becomes apparent after the reload, but it may happen after any sync (autosave, manual save, F5). Does the same thing happen in Git-based team projects?

--
Kos

[Mario Liebisch]

Yes, in our case it's a git based project and to be honest it's essentially a deal breaker for us, since we can't go over the whole translation 5-10 times just to ensure everything is really there.

[Didier Briel]

Just a short message to let you know we're working on it. I managed to reproduced the issue with F5 in a controlled environment.

Didier

[Didier Briel]

Our current hypothesis is that the issue comes from parallel excecution of autosave, reload or compile (create translated documents).

Team code must not be executed in parallel because it will read/write/switch data in the same directory. I.e. one thread can say "switch to ver 1", then second thread can say "switch to ver 2", then first thread will read data as "ver 1", but really will read "ver 2".

Reload and compile can be executed from UI only, so, they cannot be started in parallel. But autosave can be started in background.

In /trunk, a timeout was implemented, so that if synchronisation is already started, it canot be started by another process before 3 minutes.

If the lock will not be released during this time, OmegaT will report an operation error.

Please try /trunk revision 9396, and report if you still have the issue.

Didier

[Mario Liebisch]

Unfortunately the bug still happens, but it's "partially" fixed, since the problem is visible immediately.

Consider the following scenario:

• Segment 20 is untranslated.
• I begin editing segment 20.
• When done I hit TAB to go to segment 21.
• As I switch to segment 21, segment 20 essentially gets reset, reverting to its untranslated state.

We've been working on the same project on two machines, so I assume it might actually be some problem when merging git changes. As far as I can tell from the logs, autosave is properly disabled before syncing and reenabled after.

Edit:
Unfortunately that was pure coincidence. Being away for 15 minutes, switching between documents and once again two random segments (out of 25) are reverted to English…

[Aaron Madlon-Kay]

I have some comments about r9396, which implements the changes Didier describes above:

1. The new Core method Core.projectLoadSaveExecute() seems overly specific given its implementation. I think Core.executeExclusively() would be more appropriate.
2. I find anonymous Runnables overly verbose and distracting. As we have moved to Java 8, I would like to change these to lambdas (or method references when possible).

Though the commit appears not to fully address the issue here, I assume it still will not be reverted. In that case if there are no objections I will make the above changes sometime soon.

[Didier Briel]

Though the commit appears not to fully address the issue here, I assume it still will not be reverted. In that case if there are no objections I will make the above changes sometime soon.

That's OK with me.

Didier

[Didier Briel]

Further issues were discovered and corrected in the background synchronisation procedure. Please try /trunk revision 9413.

Didier

[Mario Liebisch]

Nope, unfortunately still happening several times late last week (remember we're on git though).

[Didier Briel]

remember we're on git though

It shouldn't make a difference.

Didier

[Didier Briel]

Could you give us access (giving us credentials privately, of course) to a team project where we can reproduce the issue and check the logs?

Didier

[Mario Liebisch]

Oh, missed the answer; unfortunately not since it's third party content under NDA. Which logs do you mean? Stdout or any other/additional files? I surely could grab those.

As a temporary workaround I've set the auto-save to one hour on all machines. The issue disappeared, so it's clearly linked to autosaving/syncing in some way.

[Didier Briel]

Which logs do you mean?

Git logs paired with OmegaT logs.
Ideally, for these to be used, we have to know on which commit translations were lost.

Didier

[Mario Liebisch]

So to essentially identify what's been committed and lost? Can try to figure something out.

[Didier Briel]

So to essentially identify what's been committed and lost?

Yes, and on what OmegaT action (e.g., rebase, etc.).

Didier

[Mario Liebisch]

Set the interval to 3 seconds and immediately lost a segment - plus I've found at least one exception:

99682: FINE: Enable autosave 
99682: Info: Beenden der Projektspeicherung (LOG_DATAENGINE_SAVE_END)
99682: FINE: Prepare team sync 
99682: FINE: Retrieve BASE(614cfce73e332cdc35301681eb1f74cd94f57b7f) version of 'omegat/project_save.tmx' 
99682: FINE: GIT switchToVersion 614cfce73e332cdc35301681eb1f74cd94f57b7f  
99682: Info: Ereignis: Projektwechsel - "SAVE" (LOG_INFO_EVENT_PROJECT_CHANGE)
99682: FINE: Retrieve HEAD version of 'omegat/project_save.tmx' 
99682: FINE: GIT switchToVersion origin/master  
99682: FINE: Retrieve BASE(614cfce73e332cdc35301681eb1f74cd94f57b7f) version of 'glossary/glossary.txt' 
99682: FINE: GIT switchToVersion 614cfce73e332cdc35301681eb1f74cd94f57b7f  
99682: FINE: Retrieve HEAD version of 'glossary/glossary.txt' 
99682: FINE: GIT switchToVersion origin/master  
99682: FINE: Finish project save from SaveThread 
99682: FINE: Rebase team sync 
99682: Info: Rebase starten (TEAM_REBASE_START)
99682: FINE: Rebase and commit 'omegat/project_save.tmx' 
99682: FINE: local file 'omegat/project_save.tmx' was changed 
99682: Info: TMX-Datei ***\.repositories\prep\prepared283616969868229877 wird gelesen (TMXR_INFO_READING_FILE)
99682: Info: Erstellt von: OmegaT (TMXR_INFO_CREATION_TOOL)
99682: Info: Version: OmegaT-4.0.1 (TMXR_INFO_CREATION_TOOL_VERSION)
99682: Info: Segmentierungsmethode: sentence (TMXR_INFO_SEG_TYPE)
99682: Info: Quellsprache: EN-US (TMXR_INFO_SOURCE_LANG)
99682: Info: Lesen der TMX-Datei vollst�ndig (TMXR_INFO_READING_COMPLETE)
99682: FINE: remote file 'omegat/project_save.tmx' wasn't changed 
99682: FINE: only local changes - just use local file 'omegat/project_save.tmx' 
99682: Info: TMX-Datei ***\omegat\project_save.tmx wird gelesen (TMXR_INFO_READING_FILE)
99682: Info: Erstellt von: OmegaT (TMXR_INFO_CREATION_TOOL)
99682: Info: Version: OmegaT-4.0.1 (TMXR_INFO_CREATION_TOOL_VERSION)
99682: Info: Segmentierungsmethode: sentence (TMXR_INFO_SEG_TYPE)
99682: Info: Quellsprache: EN-US (TMXR_INFO_SOURCE_LANG)
99682: Info: Lesen der TMX-Datei vollst�ndig (TMXR_INFO_READING_COMPLETE)
99682: FINE: Rebase and commit 'glossary/glossary.txt' 
99682: FINE: local file 'glossary/glossary.txt' wasn't changed 
99682: FINE: remote file 'glossary/glossary.txt' wasn't changed 
99682: FINE: there are no changes 'glossary/glossary.txt' 
99682: Info: Rebase beenden (TEAM_REBASE_END)
99682: FINE: Commit team sync 
99682: Info: Ereignis: Projektwechsel - "MODIFIED" (LOG_INFO_EVENT_PROJECT_CHANGE)
99682: Info: Ereignis: Neuer Eintrag aktiviert (LOG_INFO_EVENT_ENTRY_ACTIVATED)
99682: Info: Ereignis: Neuer Eintrag aktiviert (LOG_INFO_EVENT_ENTRY_ACTIVATED)
99682: Info: Starten der Ausf�hrung von Git 'addForCommit' (GIT_START)
99682: Info: Ausf�hrung von Git 'addForCommit' erfolgreich abgeschlossen (GIT_FINISH)
99682: Info: Starten der Ausf�hrung von Git 'upload' (GIT_START)
99682: FINER: Time for find matches: 1641 
99682: Info: Ereignis: Projektwechsel - "MODIFIED" (LOG_INFO_EVENT_PROJECT_CHANGE)
99682: Info: Ereignis: Neuer Eintrag aktiviert (LOG_INFO_EVENT_ENTRY_ACTIVATED)
99682: Info: Ereignis: Neuer Eintrag aktiviert (LOG_INFO_EVENT_ENTRY_ACTIVATED)
99682: FINER: Time for find matches: 1693 
99682: Info: Ausf�hrung von Git 'upload' erfolgreich abgeschlossen (GIT_FINISH)
99682: Info: Starten der Ausf�hrung von Git 'addForCommit' (GIT_START)
99682: Info: Ausf�hrung von Git 'addForCommit' erfolgreich abgeschlossen (GIT_FINISH)
99682: Fehler: Fehler beim Speichern der Projektdatei! (CT_ERROR_SAVING_PROJ)
99682: Fehler: java.lang.RuntimeException: Version changed 
99682: Fehler:  at org.omegat.core.team2.impl.GITRemoteRepository2.commit(GITRemoteRepository2.java:209) 
99682: Fehler:  at org.omegat.core.team2.RemoteRepositoryProvider.commitFileAfterVersion(RemoteRepositoryProvider.java:183) 
99682: Fehler:  at org.omegat.core.team2.RebaseAndCommit.commitPrepared(RebaseAndCommit.java:223) 
99682: Fehler:  at org.omegat.core.data.RealProject.lambda$null$0(RealProject.java:822) 
99682: Fehler:  at org.omegat.core.Core.executeExclusively(Core.java:373) 
99682: Fehler:  at org.omegat.core.data.RealProject.lambda$teamSync$1(RealProject.java:814) 
99682: Fehler:  at java.lang.Thread.run(Unknown Source) 
99682: FINE: Start project save from SaveThread 
99682: Info: Starten der Projektspeicherung (LOG_DATAENGINE_SAVE_START)
99682: FINE: Disable autosave 
99682: FINE: Enable autosave 
99682: Info: Beenden der Projektspeicherung (LOG_DATAENGINE_SAVE_END)
99682: FINE: Prepare team sync 
99682: Info: Ereignis: Projektwechsel - "SAVE" (LOG_INFO_EVENT_PROJECT_CHANGE)
99682: FINE: Retrieve BASE(799ee8bf4584e903e8fb010ccec564194446ce46) version of 'omegat/project_save.tmx' 
99682: FINE: GIT switchToVersion 799ee8bf4584e903e8fb010ccec564194446ce46  

The last two revisions, 614cfce7... and 799ee8bf... both don't contain the lost segment, so it wasn't somehow restored or lost, but never saved instead. None of the backups of project_save.tmx include the lost segment either.

Also note that this happened while I worked exclusively on the project/repository using the latest nightly build from last week. Autosave kicked in before I left the segment; I'm not sure whether I went to the next segment before or after it was done.

[Didier Briel]

Can someone else confirm the issue is still there?

Didier

[Aaron Madlon-Kay]

Yes, I have been getting it recently. It may be a coincidence, but I only started getting it since Alex's recent changes.

[Mario Liebisch]

From my experience it's some really odd timing related thing. A co-worker recently lost like 18 out of 20 segments, other times we can do 30-40 without a hitch.

[Didier Briel]

project_save.tmx saving code was missing in teamSync() (executed on segment deactivation) in the last fix.
Can you try again revision 9418?

Didier