Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

sorting, tab/untab, plugin shortcut keys

f0dder
2005-10-13
2012-11-13
  • f0dder
    f0dder
    2005-10-13

    Hey!

    I just tried out the SortSelection plugin from another thread, and it's ungodly slow when working on larger textfiles (1meg is pushing it). I think you ought to implement a sort+uniq directly in NP++, so there isn't the overhead of either a lot of windows messages, or chunk->lines->chunk conversion.

    Tabify/Untabify (or spaces->tab and tab->spaces) would also be nice to have directly integrated, but isn't just as important - I usually use this on shorter pieces of text only.

    Finally, *please* make it possible to use the shortcut mapper to setup shortcuts for plugins! It ought to be a relatively simple thing to implement, and would be very nice; it's simply too tedious to go to the plugins menu all the time for often used functionality.

     
    • Try the sort tool in NPPTextFX. I've sorted 200KB files and am quite happy with the speed.

      >so there isn't the overhead of either a lot of windows messages, or chunk->lines->chunk conversion.

      I don't use either of those methods. I grab the entire selected text, parse the lines, qsort, and post back to Scintilla. A plugin is equally able to access Scintilla efficiently or unefficiently as N++. Using a plugin generates no extra overhead. For example, on a rectangular selection area the upper/lower case conversion tools in N++ are orders of magnitude slower than my NPPTextFX case conversion tools, only because the techniques for retrieving and posting back text are different. You can observe this on less than a screenfull of text. I'm a bit of a speed freak.

      Unique? What is that? Are you trying to only output unique lines after sorting?

      >or spaces->tab and tab->spaces

      IMHO, most of what's in NPPTextFX belongs in Notepad, not as a plugin. It's at least in a plugin for those willing to run through the menus.

      >*please* make it possible to use the shortcut mapper to setup shortcuts for plugins

      Yes, please. I just wrote code today where I potentially need to hijack all Copy-Cut-Delete menus and shortcuts. I would prefer to have my plugin hijack them but others have suggested the Shortcut Mapper. If the Shortcut Mapper is used, there needs to be a way to adapt to the presence or absence of the plugin, and there needs to be a way have the plugin request many shortcuts and allow the user to grant all of them at once. If a plugin needs one shortcut, this is pretty easy to enable by hand. If a plugin needs 25 shortcuts, it would be nice if the user didn't need to enable all of them one by one. The plugin grant system would allow NPP to grant the same shortcuts to the same options as the plugin evolves, and to always grant the shortcut to the user's desired plugin no matter how many plugins request it.

      > it's simply too tedious to go to the plugins menu all the time for often used functionality.

      Until we get shortcuts for plugins, I have separated out the most commonly used tools into a separate Quick menu. My DLL is set up so that you can hack it with a hex editor and change which menu items show up in.

       
    • f0dder
      f0dder
      2005-10-20

      > I don't use either of those methods. I grab the entire selected text, parse the lines, qsort, and post back to Scintilla.

      That's "chunk->linkse->chunk" :)
      Perhaps I tried sort with another plugin than yours, or with an older version - but it was pretty slow on a 1meg text file. I guess I'll have to try it again. (Yes, I expect sorting a 1meg text file to take less than half a second on a amd64 3500+).

      > A plugin is equally able to access Scintilla efficiently or unefficiently as N++

      Oh, so there's no access to the internal datastructures, and even N++ has to use windows messages?

      > Unique? What is that? Are you trying to only output unique lines after sorting? 

      Standard UNIX sequence of "cat file | sort | uniq" - and yes, that's the result.

      Dunno exactly how the shortcut mapper for plugins ought to work, but just about anything would be orders of magnitude better than the nothing present right now... Something simple like binding keys to "plugin::function" ought to work, but of course I haven't looked at the internal workings of N++ yet.

      I wouldn't mind coding some of this myself to take the burden off DonHo, but I don't want to code something I'll have to diff/patch into each release...

       
    • f0dder
      f0dder
      2005-10-20

      > I don't use either of those methods. I grab the entire selected text, parse the lines, qsort, and post back to Scintilla.

      That's "chunk->lines->chunk" :)
      Perhaps I tried sort with another plugin than yours, or with an older version - but it was pretty slow on a 1meg text file. I guess I'll have to try it again. (Yes, I expect sorting a 1meg text file to take less than half a second on a amd64 3500+).

      > A plugin is equally able to access Scintilla efficiently or unefficiently as N++

      Oh, so there's no access to the internal datastructures, and even N++ has to use windows messages?

      > Unique? What is that? Are you trying to only output unique lines after sorting? 

      Standard UNIX sequence of "cat file | sort | uniq" - and yes, that's the result.

      Dunno exactly how the shortcut mapper for plugins ought to work, but just about anything would be orders of magnitude better than the nothing present right now... Something simple like binding keys to "plugin::function" ought to work, but of course I haven't looked at the internal workings of N++ yet.

      I wouldn't mind coding some of this myself to take the burden off DonHo, but I don't want to code something I'll have to diff/patch into each release...

       
    • >That's "chunk->lines->chunk" :)

      Many transforms could be done with lots of windows messages but sort is only practical with chunk-lines-chunk. I've implemented unique and add-remove linenumbers in NPPTextFX 0.2 so you'll be able to both output unique lines and put the lines back into the original order once the uniques are removed. The only further extension is to allow 3 or more sort keys on specific columns which can't be done without a dialog box.

      >Oh, so there's no access to the internal datastructures, and even N++ has to use windows messages?

      The host can get the address of the Scintilla functions so they can be called directly to reduce overhead but N++ does employ this technique. No other internal structures are available. I think using messages patterns after the Windows Common Controls standard. Fortunately the messages are well structured so they are very efficient. There's no access to the N++ internal structures because there hasn't been any need to provide a standard interface.

      >Dunno exactly how the shortcut mapper for plugins ought to work

      I'm already running the plugin system to the edges of what Don designed it for. For example: Don reserved message space 200 menu entries and I'm using almost 150 with many more planned. The plugin system does not allow plugins to build their own dynamic menus, necessary for decent HTML Tidy support and menus generated from user supplied text files. Just last night I subclassed (hijacked) the N++ message proc and I'm experimenting with altering menu commands but so far this doesn't seem practical in the long term. Overuse of subclassing and random or patterned wID assignment will lead Don into situations where some desired changes to N++ breaks plugins.

      So far I can't even get all I want. It seems that Scintilla is handling Ctrl+C, Ctrl+V, and Ctrl-X directly, a bad design choice since there will always be a host application to Scintilla and it is always preferrable to allow the host application to handle these keys and send in the SCI_COPY, SCI_CUT, SCI_PASTE as necessary. I don't know whether to hijack the Scintilla message proc or submit it as a bug fix.

      I would always provide the ability to turn subclassing off which would allow a disaffected plugin to operate but all the function keys and dynamic menus would be lost, hardly an optimal solution.

      >Something simple like binding keys to "plugin::function" ought to work, but of course I haven't looked at the internal workings of N++ yet

      If the Shortcut mapper chooses the functions, it will also need to adapt when the plugin is updated. If the plugin chooses, or at least assists the Shortcut mapper, the plugin can adapt.

      >I wouldn't mind coding some of this myself to take the burden off DonHo, but I don't want to code something I'll have to diff/patch into each release...

      Generally in return for assistance, authors are more than willing to help incorporate the changes into the release.

       
    • Neil Hodgson
      Neil Hodgson
      2005-10-24

      Text is stored in Scintilla in a reasonably simple structure as a gapped array of (character byte, style byte) pairs and the gap moves around as changes occur. If you really wanted read-only access to this it wouldn't be too unpleasant and you could add a method to return the CellBuffer's address. Doubt it would help much though. Writing into this would be far messier bypassing the undo/redo and notification mechanisms. You'd have to duplicate the functionality or drop functionality (not allowing undo to before an intervention) or do a lot of work in Scintilla to repair dependent state. Performing a sort in-place would create quite a few uninteresting undo nodes so should be implemented by extract, sort, replace.
      The standard key bindings can all be dropped with SCI_CLEARALLCMDKEYS and the context menu removed with SCI_USEPOPUP(0). However there is no way to override mouse manipulations like drag and drop without subclassing.
      Neil Hodgson (I don't normally read this forum)

       
    • I just recompiled one of the sort plugins in C++ and my 250K source code took about a minute to sort. In my NPPTextFX C plugin, the same 250K source sorts in the blink of an eye, all on a P3-500. Direct access to the buffer isn't necessary because "extract, sort, replace" is plenty fast when coded right. C++ only gets more coding layers in the way making it harder to claw through to figure out why performance is low.

      One reason to not offer direct access ever is because Scintilla may evolve to where it can files > memory. This is an important feature and it is important to understand how such a feature harms the editor. Editors can be generally classed into two groups, those that handle huge files, and those that don't. Small text editors are great for programmers whos' files aren't big but they need tons of features, features most easily implemented if the file can be handled multiple times in memory. For my sort tool, Scintilla has a copy, sends me a copy, I make a sorted copy, free the unsorted copy, send the sorted copy back to Scintilla, and Scintilla may opt to make a backup copy in the undo buffer. That won't be happening with 4TB files any time soon. The large file editors are unable to provide ton's of features because each feature added to an editor that interacts with the file and various temp files on disk requires a lot of work and some tradeoffs to make it run efficiently. Development of new features is slow and difficult which keeps the bugs high and the value low for anyone not wanting to work on large files. Macro languages can't be very functional and there's little option for plugins. The editor's host program (e.g. N++) must turn off most of it's features or pretend they can't be implemented. The cost of large file handling is minimal functionality and I see that in every editor I review. People complain about small editors not editing large files and people ask repeatedly for features to be added to the large text editors and get put off.

      The way to solve that is to offer both modes. One set of messages is entirely devoted to memory mapped files. Another much smaller set of messages is devoted to disk mapped files. Few advanced features would be offered on disk mapped files and most functionality would be implemented only by Scintilla internals.

      I've held off on capturing Cut-Copy-Paste. My versions are sufficiently more advanced that you wouldn't want them unless you know what they do and are aware of the settings. Capturing Ctrl+C and having the clipboard text appended, NULs converted to spaces, and end of lines converted to CRLF would be highly unexpected for a casual NPPTextFX user.

       
    • Don HO
      Don HO
      2005-10-28

      > Finally, *please* make it possible to use the shortcut mapper to setup shortcuts for plugins!

      Plugin shortcut will be supported in the version 3.3. Please see :
      http://sourceforge.net/forum/forum.php?thread_id=1375902&forum_id=482781

      Don