Menu

#2822 Editing files with 100000+ character lines is impossible

normal bug
open
5
2017-12-03
2007-02-21
Marc DENTY
No

Hi,

When you edit a MySQL dump (~2600 lines / 13.8 Mo), jEdit freeze with 100% CPU.
I think this is due to very long lines (1 insert statement per table).

This is very annoying and I must use an other text editor.

--Marc

Discussion

  • Marc DENTY

    Marc DENTY - 2007-02-21
    • priority: 5 --> 7
     
  • Marc DENTY

    Marc DENTY - 2007-02-21

    Logged In: YES
    user_id=84736
    Originator: YES

    This test was done under jEdit 4.3pre9

     
  • Matthieu Casanova

    Logged In: YES
    user_id=285591
    Originator: NO

    Do you have an OutOfMemoryError in the activity log ?

     
  • Marc DENTY

    Marc DENTY - 2007-05-14

    Logged In: YES
    user_id=84736
    Originator: YES

    It's difficult to say since Jedit is frozen after the bug triggers, but JEdit eats 100% of one CPU.
    Here is the stack trace given by jconsole :
    Name: AWT-EventQueue-0
    State: RUNNABLE
    Total blocked: 133 686 Total waited: 766 480

    Stack trace:
    org.gjt.sp.jedit.TextUtilities.getTokenAtOffset(TextUtilities.java:73)
    org.gjt.sp.jedit.TextUtilities.findMatchingBracket(TextUtilities.java:207)
    org.gjt.sp.jedit.textarea.StructureMatcher$BracketMatcher.getMatch(StructureMatcher.java:70)
    org.gjt.sp.jedit.textarea.TextArea.updateStructureHighlight(TextArea.java:5510)
    org.gjt.sp.jedit.textarea.TextArea.access$300(TextArea.java:65)
    org.gjt.sp.jedit.textarea.TextArea$1.actionPerformed(TextArea.java:6042)
    javax.swing.Timer.fireActionPerformed(Timer.java:271)
    javax.swing.Timer$DoPostEvent.run(Timer.java:201)
    java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
    java.awt.EventQueue.dispatchEvent(EventQueue.java:597)
    java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:273)
    java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:183)
    java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:173)
    java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:168)
    java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:160)
    java.awt.EventDispatchThread.run(EventDispatchThread.java:121)

    Hope this help,

    --Marc

     
  • Marcelo Vanzin

    Marcelo Vanzin - 2007-06-15

    Logged In: YES
    user_id=75113
    Originator: NO

    This doesn't really need to be that severe. It's a limitation of the current code.

    I started a new branch to try to minimize copying of strings, and editing files with long lines is MUCH MUCH faster with the current code in that branch. The code doesn't work 100% correctly yet, though (so don't try it at home!), and I haven't had much time to play with it.

     
  • Marcelo Vanzin

    Marcelo Vanzin - 2007-06-15
    • milestone: 101607 --> normal bug
    • priority: 7 --> 5
     
  • Matthieu Casanova

    Logged In: YES
    user_id=285591
    Originator: NO

    Hi I opened bigger files, could you try to open such file with the text edit mode ? (maybe rename the file so it doesn't match any glob in the catalog)

     
  • Marc DENTY

    Marc DENTY - 2007-07-31

    Logged In: YES
    user_id=84736
    Originator: YES

    Hi,

    This does not change anything since it seems to be caused by very long lines (1000000 chars).

    --Marc

     
  • Alan Ezust

    Alan Ezust - 2007-07-31
    • summary: Editing MySQL dumps is impossible. --> Editing files with 1000000+ character lines is impossible
     
  • Alan Ezust

    Alan Ezust - 2007-07-31

    Logged In: YES
    user_id=935841
    Originator: NO

    Are you using "soft", "hard" or no line wrap? I bet that for soft wrap, it would perform quite badly, but perhaps if you are not wrapping lines, it won't need to do as much work... not that you'd want to use the horizontal scrollbar to see a 1000000 character line, but this could narrow down the problem. I'm changing the subject line to be more descriptive.

     
  • Alan Ezust

    Alan Ezust - 2007-07-31
    • summary: Editing files with 1000000+ character lines is impossible --> Editing files with 100000+ character lines is impossible
     
  • Marc DENTY

    Marc DENTY - 2007-08-01

    Logged In: YES
    user_id=84736
    Originator: YES

    Hi,
    By default line wrapping is at "none" state. (freezing JEdit when moving the cursor)
    I tried "hard" state, JEdit froze immediately.
    I tried "soft" state, JEdit reformat the sql queries but when I move the cursor for few lines it froze too.

    Usually I use "none" since on a such file I only do search/regexp search and I don't need to browse it all.

    --Marc

     
  • Alan Ezust

    Alan Ezust - 2011-12-04
    • assigned_to: nobody --> kpouer
     
  • Matthieu Casanova

    I have a solution that is not very good :
    the problem is that syntax highlight is slow on so many text, the solution is that when we find a very long line we can use the text edit mode for that line.
    This makes jEdit fast, the problem is that it will break the syntax highlight for that line, and probably for the next lines

     
  • Jarek Czekalski

    Jarek Czekalski - 2011-12-05

    Or the similar way:
    On/after loading of a file with at least one long line give a warning and suggest to switch to text mode. One of the buttons would do it automatically.

    I don't think anyone really needs syntax highlighting in these machine generated files.

     
  • Matthieu Casanova

    In fact what I did is to switch to context insensitive highlight so at least the previous lines are correctly highlighted, and the next lines may be highlighted as they should in several languages

     
  • Roman Tsourick

    Roman Tsourick - 2017-12-03

    All the options like switching to text mode or relaxed highlighting are now implemented (see "large files" section of global options) and the dialog pops up on file open.

    Besides, I have no problem editing 20Mb+ MySQL dump in 2017 with all those corei7, ddr4 etc.

    A good reason for closing...

     

Log in to post a comment.