#1277 Concurrency issue in COMPILE-triggered scripts in team projects

[msoutopico]

Background

Some scripts may run triggered by the COMPILE event, e.g. to change something in target files, etc. That must happen of course before they are committed to the repository.

Preconditions

1. A team project containing a large number of files and/or a large amount of text / segments
2. A script that runs on COMPILE saved in scripts/project_changed, e.g. something like this.

Steps to reproduce

Commit target files

Expected results

The committed target files include the modifications made by the script.

Actual results

There is a discrepancy between the file in <project>/target (correctly updated by the script triggered by the COMPILE event ) and the equivalent file in <project>/.repositories/<project.git>/target (which is not updated). It seems that the scripts work on the target files already after the files have been copied over to .repositories for commit.

In other (or basically the same) words, the following seems to happen:

• target files are created in <project>/target
• target files are copied to <project>/.repositories/<project_repo_copy>/target and then committed
• the script works on files in <project>/target

It's reproducible in OmegaT 6.0.1 on Windows (not on Linux though), as well as on previous builds too (e.g. 5.7.2).

The issue does not occur consistently. It seems to happen only when the amount of files / text / segments is high enough to delay the script's execution enough so that the modified version of the target files is not ready in time when the commit happens.

The log (see fragment below) might show a concurrency issue: the groovy script was running while the git repo was updated. In other words, OmegaT does not seem to wait for the groovy scripts triggered by the COMPILE event to complete. The "Check in" lines are written by the script.

92144: Info: Check in: file C:\Users\user\Desktop\<project>\target\folder\R550-CliffPalace_tr-TR.xml
92144: Info: Check in: file C:\Users\user\Desktop\<project>\target\folder\R552-FestiRock_tr-TR.xml
92144: FINE: GIT switchToVersion origin/main  
92144: Info: Check in: file C:\Users\user\Desktop\<project>\target\folder\R553-HealthForum_tr-TR.xml
92144: Info: Check in: file C:\Users\user\Desktop\<project>\target\folder\R556-Microlending_tr-TR.xml

It seems the groovy scripts and the git library run asynchronously in two different threads and there's no guarantee that one will finish before the other one.

Suggested solution

Would it be possible to block the thread that copies the target files over to <project>/.repositories/<project_repo_copy>/target for commit until the process that runs the scripts has finished? (with something like an event listener, a semaphore, or something like that). In a nutshell, the commit should wait until scripts finish running.

[Kos Ivantsov]

The same issue as Manuel describes is observed with post-processing commands. Committing and pushing generated target files doesn't wait for the post-processing command to finish.

[Jean-Christophe Helary]

@t_cordonnier, do you have blockers on the issue ?
https://github.com/omegat-org/omegat/pull/1254