Menu
▾
▴
Re: [ jEdit-devel ] Text drawing speedup
|
From: Matthieu C. <cho...@gm...> - 2020-05-29 19:08:04
|
About softwrap that’s right, the entire text is computed in order to update the vertical scrollbar size, maybe we can find a way to do it in background but I am not sure it will be easy > Le 29 mai 2020 à 21:05, Dale Anson <da...@da...> a écrit : > > > "Based on the trigger (soft wrap enabled) I still think there may be something that is processing the entire file that should be limiting itself to a screen or two at a time. " > > Many plugins assume they have access to the full text of the buffer all at once. You might try disabling all your plugins, then adding them back in one at a time to see which one is the culprit. > >> On Fri, May 29, 2020 at 5:32 PM Michael H <cm...@gm...> wrote: >> Hi Matthieu, >> >> I got the 5.6 pre version to run on a mac, and for me it still hangs forever with the Bengali text. >> >> However, when I changed the mode to wrap = none, it's lightning fast. I don't see how to disable a mode temporarily or force a file to be treated as a big file. But there may still be a gap between too big, and jedit knows it's too big, and no good manual way to deal with it. >> >> The following suggestions are safe to ignore. I'm just trying to think How I'd deal with this. >> >> Possibly a checkbox on File/Load dialog "Large File Mode" (same text as in the Global options) that overrides the large file detection. >> >> In the load Dialog there is an "encoding" option, but no "mode" option. That would make it possible to work around/disable syntax highlighting for files that are too big for it to be painted timely. >> >> Possibly have a idle state timer on file loads that falls back to large file mode if the file hasn't loaded completely in 20-30 seconds (or throws a dialog asking to wait or cancel or use large file mode). >> >> Based on the trigger (soft wrap enabled) I still think there may be something that is processing the entire file that should be limiting itself to a screen or two at a time. >> >> >>> On Fri, May 29, 2020 at 11:48 AM Matthieu Casanova <cho...@gm...> wrote: >>> So I finally tried to disable chunk merging and it seems the chunk cache works much better, not sure if there is a drawback on this, but the speed is lighning now even compared to the version of yesterday. >>> It's on svn and my compiled version is here >>> >>> http://home.kpouer.com/jEdit/jedit.jar >>> >>> -- >>> Matthieu Casanova >>> cho...@gm... >>> >>> >>> >>> Ven 29 mai 2020, à 08:46, Matthieu Casanova a écrit : >>>> Thanks, I am glad you are happy with the fix. >>>> I added another fix yesterday that helps drawing text containing special characters too. >>>> I have one or two other idea but after you are right it would be nice to release jEdit >>>> with those fixes. >>>> >>>> -- >>>> Matthieu Casanova >>>> cho...@gm... >>>> >>>> >>>> >>>> Mar 26 mai 2020, à 15:56, James a écrit : >>>>> Trying again to the list... >>>>> >>>>> Just to add my 2c worth of opinions: if you don't find any issues with this fix, this change is AMAZING and warrants a 5.6 release asap! >>>>> >>>>> What a DIFFERENCE this makes! >>>>> >>>>> Thanks for your hard work on this. >>>>> >>>>> On Mon, May 25, 2020 at 7:29 PM Matthieu Casanova <cho...@gm...> wrote: >>>>> >>>>> In fact there are two steps : >>>>> >>>>> the tokenizer which is parsing the document and create chunks. It is done line by line, but to tokenize a line, you might have to know the context of the document. >>>>> For example if it is a programming language such as java you have to know the context of the previous line in order to know for example if you are in a comment or not). >>>>> Some other edit modes like logs are insensitive to the context and you can display any line without parsing the beginning of document. >>>>> >>>>> After for any kind of document the glyphs are computed only for visible lines, but the problem was that we were giving the entire text and the interesting offsets to the Font api (it was nice we did not any array copy). Until java 8 it worked perfectly, but since Java 11 they analyze the entire document even if you want to draw a few chars. >>>>> What I did is creating a copy of the chars that have to be displayed to call that Font api. It should not be a problem as it is the text of a token which must be consistent. It will not be a letter in the middle of a word. >>>>> And that copy is also not a performance problem as we already copy that portion of text to be used as key for a glyph cache (to avoid recomputing the same glyphs again and again) >>>>> >>>>> -- >>>>> Matthieu Casanova >>>>> cho...@gm... >>>>> >>>>> >>>>> >>>>> Lun 25 mai 2020, à 18:33, Michael H a écrit : >>>>>> And it also would be useful (read speed increase to nearly eliminate size issue) to limit the 'layout' to immediately viewable text. That is, compose each paragraph, then test if the screen is full, and stop. Rather than compose the whole document every time the display changes. >>>>>> >>>>>> On Mon, May 25, 2020 at 11:29 AM Michael H <cm...@gm...> wrote: >>>>>> It would be nice if the "Full Layout" 'feature' could be selected. It's great to have at times, but if it could be disabled (like word wrap on the status bar) it would make jedit infinitely more useful. Most edits in plain text editors of 10mb files aren't composing the text, but are looking at code, and most of the code is restricted to unaccented, non-ligature, noncomplex text. The relatively few cases where someone wants to copyedit, having a toggle on the status bar .... >>>>>> >>>>>> >>>>>> On Mon, May 25, 2020 at 11:20 AM Matthieu Casanova <cho...@gm...> wrote: >>>>>> >>>>>> In fact it's in the doc, >>>>>> >>>>>> Returns a new GlyphVector object, performing full layout of the text if possible. Full layout is required for complex text, such as Arabic or Hindi. Support for different scripts depends on the font and implementation. >>>>>> >>>>>> It's not a bug but a feature. >>>>>> >>>>>> -- >>>>>> Matthieu Casanova >>>>>> cho...@gm... >>>>>> >>>>>> >>>>>> >>>>>> Lun 25 mai 2020, à 09:23, Matthieu Casanova a écrit : >>>>>>> Hi, >>>>>>> I tested with Java 11 to java 15 and all of them have the same problem, I am 99% sure it is in C++ native code, I am trying to understand what’s wrong. >>>>>>> Maybe they are doing glyph of all part of the char array in order to get a context or something like that ? But it would be strange. >>>>>>> I could try some Arabic and see if doing glyph of the middle of a word is different if I give the entire char array of the word or only the letters I want. >>>>>>> >>>>>>> Matthieu >>>>>>> >>>>>>>>> Le 24 mai 2020 à 20:11, Jesse Pavel <jp...@al...> a écrit : >>>>>>>> >>>>>>>> Hi Matthieu, >>>>>>>> >>>>>>>> Both Hotspot builds and OpenJ9 builds use the Harfbuzz shaping engine, I believe, and the native shape() method of SunLayoutEngine seems to pass the heavy lifting to Harfbuzz. >>>>>>>> >>>>>>>> Did you happen to test the code with Java 13 or 14? The bug reports for slow glyph performance and poor LineBreakMeasurer performance have Java 13 listed as a fix version, though the details of the fix don't appear on the Java Bug Tracker. >>>>>>>> >>>>>>>> Speaking of which, non-JDK contributors can use >>>>>>>> >>>>>>>> http://bugreport.java.com/ >>>>>>>> >>>>>>>> to report bugs, though they don't appear in the public bug tracker until after an internal triage process (or so I gather from the mailing list). >>>>>>>> >>>>>>>> Jesse >>>>>>>> >>>>>>>> On Sat, May 23, 2020 at 6:29 PM Matthieu Casanova <cho...@gm...> wrote: >>>>>>>> >>>>>>>> Hey, >>>>>>>> in fact I did not find the bug on the OpenJDK bugtracker, maybe it is a new bug : >>>>>>>> I quickly saw that the markTokens() method was the problem, in the beginning I thought we were invalidating the lines too often or something like that and I tried to optimize it. >>>>>>>> But then I found it was strange that a big file was so slow against a smaller file. >>>>>>>> So I took a 10MB log file and duplicated the text until I had 100MB file. >>>>>>>> And when going to any line of the file, the 100 MB file was much slower than the 10MB file to display exactly the same thing. >>>>>>>> Debugging markTokens showed me that marking one line could take 10ms in one file and 100ms for the bigger file even if it is the same text. >>>>>>>> Then I figured how to use visualvm to monitor CPU and saw that the slow method when scrolling what the shape() method of SunLayoutEngine. >>>>>>>> The method is c++ and what seems strange is that Hotspot and OpenJ9 have the same problem but I don't think it is the same code isn't it ? >>>>>>>> >>>>>>>> To be sure I used JMH : >>>>>>>> >>>>>>>> import org.openjdk.jmh.annotations.*; >>>>>>>> import sun.font.GlyphLayout; >>>>>>>> import sun.font.StandardGlyphVector; >>>>>>>> >>>>>>>> import javax.swing.*; >>>>>>>> import java.awt.*; >>>>>>>> import java.awt.font.FontRenderContext; >>>>>>>> import java.io.IOException; >>>>>>>> >>>>>>>> import static java.util.concurrent.TimeUnit.SECONDS; >>>>>>>> >>>>>>>> @State(Scope.Benchmark) >>>>>>>> @Warmup(iterations = 2, time = 5, timeUnit = SECONDS) >>>>>>>> @Fork(value = 0, warmups = 2) >>>>>>>> public class GlyphBench >>>>>>>> { >>>>>>>> public static final char[] smallText = new char[100000]; >>>>>>>> public static final char[] doubleText = new char[1000000]; >>>>>>>> public static final FontRenderContext FRC = new FontRenderContext(null, true, true); >>>>>>>> public Font font; >>>>>>>> private int flags; >>>>>>>> private GlyphLayout gl; >>>>>>>> >>>>>>>> @Setup(Level.Trial) >>>>>>>> public void setUp() throws IOException >>>>>>>> { >>>>>>>> font = new JLabel().getFont(); >>>>>>>> flags = Font.LAYOUT_LEFT_TO_RIGHT >>>>>>>> | Font.LAYOUT_NO_START_CONTEXT >>>>>>>> | Font.LAYOUT_NO_LIMIT_CONTEXT; >>>>>>>> gl = GlyphLayout.get(null); >>>>>>>> } >>>>>>>> >>>>>>>> @Benchmark >>>>>>>> @Measurement(iterations = 3, time = 10, timeUnit = SECONDS) >>>>>>>> public void smallText() >>>>>>>> { >>>>>>>> StandardGlyphVector gv = gl.layout(font, FRC, smallText, >>>>>>>> 0, 3, flags, null); >>>>>>>> } >>>>>>>> >>>>>>>> @Benchmark >>>>>>>> @Measurement(iterations = 3, time = 10, timeUnit = SECONDS) >>>>>>>> public void doubleText() >>>>>>>> { >>>>>>>> StandardGlyphVector gv = gl.layout(font, FRC, doubleText, >>>>>>>> 0, 3, flags, null); >>>>>>>> } >>>>>>>> } >>>>>>>> >>>>>>>> The small text is 100000 and big text is 1000000, but I want glyph of 3 chars only. >>>>>>>> And the result are >>>>>>>> >>>>>>>> GlyphBench.doubleText thrpt 3 1225,655 ± 47,771 ops/s >>>>>>>> GlyphBench.smallText thrpt 3 27119,560 ± 1101,963 ops/s >>>>>>>> >>>>>>>> With a text that is 10 time bigger the big text is almost 20 time slower to compute. >>>>>>>> About posting bugs to OpenJDK, only contributors can do that ? >>>>>>>> >>>>>>>> -- >>>>>>>> Matthieu Casanova >>>>>>>> cho...@gm... >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Sam 23 mai 2020, à 21:05, Jesse Pavel a écrit : >>>>>>>>> Great catch, Matthieu! >>>>>>>>> >>>>>>>>> Can you please post a link to where you saw the JDK bug? >>>>>>>>> >>>>>>>>> I've been following bugs like >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214573 >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8202131 >>>>>>>>> >>>>>>>>> but I was wondering how you figured out it was the length of the char array. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jesse >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, May 23, 2020 at 3:38 AM Matthieu Casanova <cho...@gm...> wrote: >>>>>>>>> >>>>>>>>> Hey, >>>>>>>>> since 2 months I was searching the reason why jEdit was so slow with medium and big files. In my memories it was fast years ago. >>>>>>>>> First I looked at every changes in jEdit's code since years but found yesterday that the bug is in the JDK : >>>>>>>>> an internal method used to create glyphs : >>>>>>>>> >>>>>>>>> sun.font.SunLayoutEngine.shape() >>>>>>>>> >>>>>>>>> When drawing glyphs we were giving a big char array of the size of the buffer with start and stop offset of the interesting part of the text. >>>>>>>>> The problem is that the shape() method performances is proportionnal to the length of the char array, not to the offsets. >>>>>>>>> >>>>>>>>> So I now copy only the few necessary chars to a new array and it is fast again. >>>>>>>>> From what I saw this problem came with Java 11 and do not appear with Java 8. I did not check Java 9 and 10. >>>>>>>>> >>>>>>>>> Please pay attention to the textarea display in any case, if you see display problems, please let me know. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Matthieu Casanova >>>>>>>>> cho...@gm... >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> ----------------------------------------------- >>>>>>>>> jEdit Developers' List >>>>>>>>> jEd...@li... >>>>>>>>> https://lists.sourceforge.net/lists/listinfo/jedit-devel >>>>>>>> >>>>>> >>>>>> -- >>>>>> ----------------------------------------------- >>>>>> jEdit Developers' List >>>>>> jEd...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/jedit-devel >>>>> >>>>> -- >>>>> ----------------------------------------------- >>>>> jEdit Developers' List >>>>> jEd...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/jedit-devel >>>>> >>>>> -- >>>>> ----------------------------------------------- >>>>> jEdit Developers' List >>>>> jEd...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/jedit-devel >>>>> >>>> >>> >>> -- >>> ----------------------------------------------- >>> jEdit Developers' List >>> jEd...@li... >>> https://lists.sourceforge.net/lists/listinfo/jedit-devel >> -- >> ----------------------------------------------- >> jEdit Developers' List >> jEd...@li... >> https://lists.sourceforge.net/lists/listinfo/jedit-devel |
