#981 5.2.0 painfully slow

[kirk mitchell]

5.2.0 painfully slow. Tabbing to the next segment causes either a long delay (1-4 sec) or hangs forever. Reverted back to 4.3.2 where problem not present.

SPECS: Ryzen2600/16g mem. Same on my 2400G laptop

NOTES: I have relatively many, and a couple large TMXs being referenced due to nature of work

STEPS TO REPRODUCE: Install and use 5.2.0

STEPS TO FIX: Revert to 4.3.2

[Kos Ivantsov]

I can't reproduce it with 5.2.x, no slowdown is exprienced whatsoever.
Before filing this bug it would have been wise to consult the users group and maybe get an advice on how to fix your issue.
Please specify your Java version (both version number and bitness), your OS, and the amount of memory allocated to OmegaT.

[kirk mitchell]

Thanks for replying. Here is more information.

Memory allocated to Omegat: -Xmx4096M (about 4gig)
Actual memory used by Omegat: according to task manager, about 2gig
OS: Windows 10 Pro, version 1909
Java Version: 1.8.0_241 (latest, I believe) 64-bit
Java Version for 5.2.x: Task manager says "OpenJDK Platform binary" (I wonder if this means the above Java is irrelevant ?)
Language pair: Japanese to English

Tried:
1) removed all files in TM/Auto = no improvement
2) replaced project_save.tmx with smaller (5gig) file = no improvement

Oberservations:
1) Shorter sentences (under 75 characters) = freeze time very short to none
2) Longer sentences (150 characters+) = long freeze, or occasional lock-up requiring force shutdown
3) Fetching machine translation setting ON/OFF = no effect (not faster or slower)
4) Same behavior on desktop (Ryzen2600) and laptop (Ryzen 2400g) suggests it is not computer specific
4) Reverting back to 4.3.2 = fixes everything. I can tab as fast as I want with no noticable slowdown

Guesses based on changes file:
1) "Switch to a non-Oracle runtime: Bundled JRE is now AdoptOpenJDK" . Open JDK is the cause?
2) "Display fuzzy matches for all translations in the matching language". Something with this feature is the cause?

Since I see this on multiple computers, I am surprised to be the only person experiencing this issue. I don't know what else I can do, but any advice would be appreciated.

Thanks

[Kos Ivantsov]

Please try with Java11 and another config folder with no previous settings (i.e point it to an empty folder like this java -jar OmegaT.jar --config-dir=/path/to/config-files/). Make sure no plugins are installed, and report back.
As to your guess #2, do your TMX files contain TU's in languages other than Japanese and English? Besides that setting can be disabled.

[kirk mitchell]

Thank you, I will try that as soon as time permits and post back.

[kirk mitchell]

After much trial and error, I have finally found the cause and can reliably reproduce the problem.
It is the GLOSSARY folder.

On Ver.4.3.2, my glossary folder contains around 55 files, totaling around 40MB (largest file is 12MB, avg. is around 500KB). This version has no problem with these. Tabbing backwards and forwards is completely smooth.

On Ver.5.3.x, it works fine with up to 2 txt files in the glossary folder.
Once there are 3 txt files, it begins to slow.
With 5 (under-1MB) files plus 1 (12MB) file, it can no longer operate properly.
Once I replace all my glossary files it will freeze within a few minutes of work, becomes unresponsive, and must be force-closed with task manager.

The problem is identical on two different computers. I hope this is helpful.

[Hiroshi Miura]

From the reports, I've just checked commits between v4.3.2 to v5.2.0 on source directory of glossary (src/org/omegat/gui/glossary/) then just find FOUR commits on there.

• Tooltip attribute set as origin, not as clone of the full entry
• Fix Spotbugs error
• Add tooltips to GlossaryTextArea
• Abstract glossary entry rendering API

I hope it helps investigating the problem.

[kirk mitchell]

Hiroshi Miura, thank you for the input.
To update my experience. I have tried every version up to v5.5.0.
Anything past v4.3.2 has the same problem.
Also, a fresh install on my latest laptop (Ryzen 4500u) shows the same behavior.
Even with identical Java versions and configurations, the old version (v4.3.2 or older) works and any newer version does not.
Something must have changed between those versions.
Actually, I have given up on it being fixed, but perhaps some future version will work again.

[st. pakebusch]

i can reproduce the bug in 6.2. development version and am currently working on a fix, PR to follow soon.

root cause analysis:
https://github.com/omegat-org/omegat/commit/45b9c44ca16f7b85c4f9eabdf0973f65ddb6839a ("Abstract glossary entry rendering API")

4.3.2: all matches concatenated into one string, single setText call — linear. Since 5.x: every entry inserted separately into live StyledDocument on EDT; each insert costs proportional to document length — quadratic in match count. Measured on master: 400 matches = 1.1 s, 800 matches = 4.5 s (factor 3.9). Large glossaries yield hundreds of matches per long segment → seconds-long EDT freeze on every segment switch, matching report exactly (threshold at 3+ glossary files, long segments worst, "hangs forever"). Search code itself byte-identical between 4.3.2 and 5.2.0 and runs on background thread — not the cause. Aggravated by per-term attribute sets (edc7dffca) and linkifier full-text regex per insert.

Fix direction: render into offline document, set into pane once — API kept, cost linear again.

[st. pakebusch]

https://github.com/omegat-org/omegat/pull/2318 should fix it.