Menu

#3 poor performance with large text files and soft wrap

closed-fixed
None
5
2012-03-13
2004-03-17
Tyler Ward
No

Jedit does a tolerable job of opening medium size text
files, but is no match for UltraEdit with large files.
if there was a way to easily open very large files,
that would be very nice.

This is the algorithm I reccomend, though it's just a
though.

1) If the file size is over a set limit, go into large
file mode.

2) In large file mode, on opening a file, scan through
it for newlines and record the position of each newline.

3) Count the lines (number of newlines found) and load
up an area around the current text display window, this
can be done by seeking to the appropriate spot in the
file. For instance, if someone scrolls 2/3 of the way
down a million line file, then you need to load in
lines around line 670,000, and you conveniently know
the offset into the file of the line, because you made
a table of it. So you pick your spot, then load in a
few thousand lines above and below, using that as the
display. Whenever the user gets too close to the edge,
refresh it.

This is a big task, but something that jedit should
consider if it is going to become the ultimate text
editor.

-Tyler

Discussion

1 2 > >> (Page 1 of 2)
  • rletzler

    rletzler - 2009-11-02

    jEdit also gives no feedback about progress beyond telling us that 1 i/o operation is in progress, so it is difficult to tell whether it is hard at work opening a huge file or has frozen up. Some kind of prominent percent done text or thermometer display would be quite useful.

     
  • Matthieu Casanova

    moved to feature requests

     
  • Matthieu Casanova

    • labels: 102668 -->
    • milestone: 101608 -->
     
  • Alan Ezust

    Alan Ezust - 2011-11-30
    • assigned_to: nobody --> kpouer
    • status: open --> closed-fixed
     
  • Alan Ezust

    Alan Ezust - 2011-11-30

    jEdit 4.5pre1 now has a large-file mode that kicks in and allows you to pick a faster syntax highlighting mode.
    This still doesn't prevent sidekick from doing its thing but that's another issue people are looking at now.

     
  • tvojeho

    tvojeho - 2011-11-30

    Hi, I am not sure how much delay the Sidekick is causing because I am not using this plugin, but for me the biggest problem is soft wordwrap. I am running jEdit 5.0 pre version on Win7 on a laptop 2.2 GHz AMD cpu with 4 GB RAM, Sun java 1.6.0_29 and opening a 4 MB text file with soft wrap is a chore.

    I tried how much difference the syntax highlighting mode makes, and it seems not much.

    Time for text to load:
    Word wrap soft, full syntax highlight...1 min 55 secs
    Word wrap soft, context insensitive syntax highlight...1 min 54 secs
    Word wrap none, full syntax highlight...0 min 5 secs
    Word wrap none, context insensitive syntax highlight...0 min 4 secs

    Regards, tvojeho

     
  • Alan Ezust

    Alan Ezust - 2011-11-30
    • summary: poor performance with large text files. --> poor performance with large text files and soft wrap
    • status: closed-fixed --> open
     
  • Alan Ezust

    Alan Ezust - 2011-11-30
    • assigned_to: kpouer --> nobody
     
  • Alan Ezust

    Alan Ezust - 2011-11-30

    Soft wrap is expensive, there is no doubt about it.

     
  • Alan Ezust

    Alan Ezust - 2011-11-30

    Perhaps the "large file mode" should also have an option to disable soft-wrap and not just highlighting.

     
  • Alan Ezust

    Alan Ezust - 2011-11-30
    • assigned_to: nobody --> kpouer
     
  • tvojeho

    tvojeho - 2011-11-30

    Yes, that could help in some cases, unfortunately for me most of the time I need to use the soft wrap. At least I have the time to make coffee while opening the larger files ;)

    tvojeho

     
  • tvojeho

    tvojeho - 2011-11-30

    Speaking of the large file mode, is there a way to switch off this feature? Seeing that the simple text mode highlighting does not slow the file loading, it is practically useless for text files to show this warning.

    tvojeho

     
  • Matthieu Casanova

    I don't understand. If the edit mode is text, the dialog do not appear.

     
  • Matthieu Casanova

    In fact you're right, there is a bug

     
  • tvojeho

    tvojeho - 2011-11-30

    Ah, it seems that this is my doing then; when I first started using jedit, I made a copy of 'text' mode, customized the syntax highlighting and renamed it as 'txt' mode which I use as default for text files.

     
  • Matthieu Casanova

    but what was the edit mode used by your file when you opened it ?

     
  • tvojeho

    tvojeho - 2011-11-30

    the edit mode was 'txt' - as I stated in catalog in mode folder for handling *.txt files.

     
  • Matthieu Casanova

    Does your text.xml file contains this ?

    <PROPS>
    <PROPERTY NAME="contextInsensitive" VALUE="true" />
    </PROPS>

     
  • tvojeho

    tvojeho - 2011-11-30

    it is commented out - does this property drive the large buffer dialog?

    <PROPS>
    <PROPERTY NAME="wordBreakChars" VALUE=",+-=&lt;&gt;/?^&amp;*&nbsp;" />
    <!-- <PROPERTY NAME="contextInsensitive" VALUE="true"/> -->
    </PROPS>

     
  • Matthieu Casanova

    Yes, this property means that the edit mode is not context insensitive (jEdit do not have to calculate syntax highlight of line n-1 before calculating syntax highlight of line n.
    This is very important because with a large buffer when you go to the latest line jEdit have to highlight the entire buffer if the context is sensitive.

    What jEdit does when a buffer is very large is to suggest to switch to context insensitive if the edit mode do not already have this option.

     
  • tvojeho

    tvojeho - 2011-12-01

    Thanks Matthieu, it works fine. I just didn't find it mentioned in help or mailing lists.

    tvojeho

     
  • Matthieu Casanova

    • status: open --> closed-fixed
     
  • Kazutoshi Satoda

    • status: closed-fixed --> open
     
1 2 > >> (Page 1 of 2)

Log in to post a comment.