Menu
▾
▴
#1131 Very slow segment switching with a lot of duplicates
I have a not so big project with 8K segments and some segments occur pretty often in text. Going from those segments to a next or a previous one without any editing is really slow . For example, there is a segment with 83 duplicates and switching from them takes up to 3.8 seconds.
I dug down into the source code and found the problem. It is a loop right below commentary find all identical sources and redraw them in EditorController.java. I fixed the problem for myself by wrapping the loop into condition if (translationChanged || noteChanged) but I guess there might be more space for improvement for cases under this condition.
A link to a loop mentioned above https://github.com/omegat-org/omegat/blob/dd623dedecf94ef092b5d63f2857b3cc99a18a62/src/org/omegat/gui/editor/EditorController.java#L1241
@miurahr9, would you mind taking a look at that please ?
@abashkinl could you provide test data that can reproduce the situation without leaking any trade secrets?
We can use java profiling tool to analyze performance with empilical approach.
@miurahr9, here is a sample project with only one source file containing 200 duplicate segments. It takes around 19 seconds to switch segments on my laptop with i7-9850H.
I don't reproduce the lag on my Apple machine (2.8 GHz Quad-Core Intel Core i7, 16gb). There is a small lag when I keep Cmd+N pressed, though. But nothing too painful. I'd be curious to see how fast it becomes with your fix.
For the reference I use Debian Testing x64 on i7-9850H, 32 GB. OmegaT built from git master and runs on OpenJDK 11.0.17.
I also don't reproduce the lag on Mint Linux x64 on AMD Ryzen 7 PRO 32GB. Taking profileing on IDE then I understand EditorController#commitAndDeactivate is majority of CPU sampling in user space, but generic majority is Java/Swing thread.
I try to add alternative translation, with alternative source files.
Are their any other special conditions?
A process
// find all identical sources and redraw themmay be able to run under another thread and redraw asynchronously, other than target is next/prev segment.Last edit: Hiroshi Miura 2023-02-20
Are you sure? The method starts with UIThreadsUtil.mustBeSwingThread(); so I would have a doubt.
But we should check whenever we are really obliged to redraw all. Maybe we could consider only those which are visible?
Last edit: Thomas CORDONNIER 2023-02-21
@t_cordonnier you are right. It should be in the SwingThread and cannot be as another thread.
In many case I experienced, lag on OmegaT operations may be caused by Swing process to update UI parts. For example, dictionary pane become slow down when huge dictionary is searched, that is warned in dev mail message at
https://sourceforge.net/p/omegat/mailman/message/37777933/
In that case most processes are done in another thread, so most duration is in the last process in Java/Swing that putting large contents into pane.
We can observe similar situation when you open a huge source text that requires marks and render on editor pane.
This page may help understand JTextPane, JEditorPane slow performance with huge text.
https://web.archive.org/web/20210512232211/http://java-sl.com/JEditorPanePerformance.html
FYI: here is an idea to "folding" contents on text editor.
https://www.eclipse.org/articles/Article-Folding-in-Eclipse-Text-Editors/folding.html
Hi Lev
I did not succeed to reproduce your problem but I remember that I had solved something very similar in DGT years ago
Could you please test the following versions:
http://185.13.37.79/sites/all/modules/pubdlcnt/pubdlcnt.php?fid=271
http://185.13.37.79/sites/all/modules/pubdlcnt/pubdlcnt.php?fid=348
and tell me if you have the same problem with one of them, none or both?
Sorry for using DGT-OmegaT instead of the original, but this correction is not trivial so before proposing it in standard OmegaT I would prefer to know whenever it still solves something reproductible or not. The document I used for my tests in 2015 worked correctly with standard OmegaT 4, meaning that in the meantime, the core team had implemented another solution, reason why I did not propose mine. But if it also solves your problem, maybe it is a good reason why to propose it, after all.
So, if you can do the test and if the problem disappears with one of the proposed downloads, then I wll describe here what I did and we can discuss whenever to port it to OmegaT 5.8 or not. If it solves nothing, no reason why to continue in this direction, we can search something else instead.
Thomas
@abashkinl could you give a feedback of @t_cordonnier suggestion?
Hi Thomas,
Both of your versions of DGT do not have this bug. The cause of the bug seems obvious though. OmegaT redraws identical segments regardless of changes made to them. I still use a patched version with the fix I proposed here.
@t_cordonnier could you propose your solution?
FYI:
This is a dev-ML discussion in Jan. 2016 about peformance against very large source HTML file which produce tons of segments .
f75439d4 is one Aaron implement lazy loading.
OmegaT from current git still suffers from this issue. Reproduced on Debian x64 with OpenJDK17.0.10-ea from Debian repository and on Arm macOs 14.2.1 (23C71) Sonoma. Attaching a patch from the ticket body.
I try to run it with java profiler.
A
commitAndDeactivemethod consumes 400msec in my environment with test project.A code
builder.createSegmentElement(false, Core.getProject().getTranslationInfo(builder.ste), !defaultTranslation);consumes 300msec, 75% of the method.During a code path,
SegmentBulider#addModificationInfoPartconsumes 200msec. AlsoSegmentBuilder#addInactiveSegpartconsumes 80msec.There are several GUI updates which consume machine time;
doc.insertStringdoc.setAlignmentThese are an evidence that the performance issue is in GUI updates, and I confirm the proposed modification is effective to improve performance.