A few days ago I started a thread called "Navigating using the find all in current document" https://sourceforge.net/forum/forum.php?thread_id=3295804&forum_id=331753 . Trying to overcome the problem I removed completely 5.3.1 and installed 5.4.3 and the plugging "Function List 2.0." By the way the plugging is great, thanks.
Here is what I found: When I open a 4 meg file, n++ 5.4.3 remains using 100% of cpu for ~5 extra seconds after the file is open; for a 12 Meg file it keeps running for extra ~10 seconds; and for a file ~100 meg it keeps running for 63 extra seconds after the file is open. I found that very weird, so I did the same exercise with scintilla; for the same files, once the file is open the cpu usage takes ~1/3 to 1/2 a second to go to zero. I removed completely 5.4.3 and went back to 5.3.1 and the same plugging. I did open the same files, and the behavior is the same as scintilla, basically once the file is open the cpu usage goes to zero until I do something with the file. On 5.4.3 even resizing the window recreates the extra cpu time. That could explain why the "navigating using the find all in current document" crashes the software, n++ is still doing something and sending one additional instruction crashes it. I have two Intel Xeon 3.4 MHz, 3.5 Gig memory and running XP professional. So, it is not a lack of resources. I noticed that if I call the function plugging while n++ is still going at it, my second cpu kicks in and uses it, while the other one is still going at 100% usage. The function/plugging finish, but n++ keeps going. I do not think is the plugging because when I discover the problem I did not have the plugging installed.
Hope this help.
Thanks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have a similar problem with quite large files with a size of 1 to 10 MB.
When I do a search for a word, the word is found immediately, indicated by the vertical scrollbar having moved to the correct position. But then Notepad++ takes some seconds, not obviously for what, before it shows the corresponding part of the file.
I guess this has to do something with the Syntax Highlighting. Those files are some simple log files, but I have SQL Syntax Highlighting applied to them. The only things that (have to) get highlighted are numbers, though.
Perhaps the same is the case for you. Funny, I didn't test those files and the find and show behavior without the Syntax Highlighting applied. Could you?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have tried with both Syntax Highlighting and just pure text. In my case, the files are mainly numbers and I just look for a few lines that start with *. Also I noticed that when I use the Function Plugging, if the file is large the cursor location is off for a couple of lines. It can be very confusing. The cursor is located typically before/above the line that I'm looking for. With small files that does not happen. As I would expect the language highlighting slow down the file opening, but once the file opens the cpu usage should goes to zero.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Don thanks for taking the time to answer.
I discovered this morning that the Function Plugging double (2.4 times) the memory used by n++ (5.3.1 or 5.4.3). I did a test with a 38 Meg file and n++ 5.3.1. To open that file n++ takes 93 Meg, but when I call the Function plugging it jumps to 220 Meg. In addition when I close the Function window, it does not release the memory, it still hugs 220 Meg.
I just removed all the plugging from 5.3.1, the ones on the installation files as well as the ones on the app files. With the 38 Meg file, n++ still keeps going ~25 seconds after it opens the file (I can see the content of the first window). After the cpu usage goes to zero (after 24 seconds) I resize the window and n++ goes again on a ~25 seconds crunching. So, if you want to reproduce the problem, at least with 5.3.1, just open a “large file” and monitor the cpu usage, when the cpu goes to zero, the resize the window and it goes again for about the same time.
I have to re-install 5.4.3 since I removed it. I will post the results as soon as I can get to it.
Thanks,
carlos
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Don, I removed completely 5.3.1 and installed 5.4.3 with the pluggins that come with the installation package. The only thing that I did was add my userDefinedLang.xml and modified a little the contextMenu file. Somehow it is working great, even the Find in Files and the navigation using the found line is working. Once in a while it get slightly slow, but the times went from 60 seconds to 3 or 4 seconds with a 100 Meg file. I think what was causing the problem with large files was the Function Plugging.
Thanks for a great software.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Well, the "couple of seconds" is due to the Syntax Highlighting, I guess. I was able to do some tests today.
A (quite) large SQL file is loaded and displayed immediately. When changing Syntax Highlighting to text, searching and showing the found match experiences no lag.
Turning Syntax Highlighting Language to SQL again, may take some seconds again.
It sure takes some seconds when:
- scrolling from a just opened SQL file to the bottom of it by using Ctrl+End
- opening a SQL file with the extension .txt (as normal text file by default) and then changing Syntax Highlighting to SQL
One the total file has been processed for Syntax Highlighting, the lags no longer occur.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The lag that you're experiencing also has something to do with Scintilla, which wasn't originally intended to work with extremely large text files.
Thanks for posting the results from your investigations with syntax highlighting. Maybe we could put a condition on newly opened files to use Normal Text when the file is above a given size?
Something for IdeaTorrent, maybe.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think a better solution for big files is to add a "minimal mode" to Npp. Where Npp is in this mode, it should:
1. alert it is in "minimal mode" with a slide alert (as Firefox do when you can save passwords)
2. disable all plugins
3. disable all autocomplete features
4. do not apply syntax styles
5. disable all highlight and similar features
6. do not wrap the output
7. do not show any simbols (white chars, indent guide, ...)
8. disable indent features (folding etc.)
9. do not display vertical edges
10. do not save session at exit
11. disable clickable link feature
12. disable file status auto-detection
This mode can be started automatically when a large file is opened, as suggested, but it could also added as a command line argument.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the extra testing. I agree the syntax higlighting should generate some extra work. What I have noticed is that the first search/go to is the one that takes the longest, after that it is almost instantaneous. My problem was that the first search was taking ~60 seconds on a 100 Meg file. With the clean installation it takes a few seconds with the syntax higlighting on. I think that time is very reasonable.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hey I click the post to fast. This is what I was trying to said.
Thanks for the extra testing. I agree that syntax highlighting should generate some extra work. What I have noticed is that the first “search/go to” is the one that takes the longest, after that it is almost instantaneous, independent of the file size. My problem was that the first search was taking ~60 seconds on a 100 Meg file. With the clean installation of 5.4.3, n++ takes a few seconds to settle down when opening the 100 Meg file with the syntax highlighting on. For me that time is very reasonable. I think the software is working properly. Since I am not a programmer I do not know if those few extra seconds at the start can be worked out. For now I’m very happy. Don, thanks again for a great software.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Don,
First of all I was using FunctionList_2_0_Uni_dll.zip.
It is hard to say for sure, but this is what I have observed:
- n++ 5.3.1: it appears that some delay is associated with n++, because I did a test removing all the plugging directories and n++ still showed some delays after the file was open.
- n++5.4.3: once the FunctionList is installed, opening a 100 Meg file takes more than 60 seconds after the window shows the first page. In addition, once that the FunctionList is called the memory usage is about 2.4 timed of the original memory required to open the file, and the memory is not released when the FunctionList is closed. After the test, I removed the components of the FunctionList but now n++ is behaving badly again. That is 60 seconds to open the large file. Something is written somewhere that I’m not removing. I use the “uninstall” and remove all the n++ folders and did a clean installation and now n++ is working fine again. The FunctionList write something that messes up the behavior of n++.
Hope this help.
carlos
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I did more testing with n+ 5.4.3 and the plugging’s in the original files. It took me a while to discover what is causing the problem. Sometimes it took very long time to open/settle down and some other times was “very” fast (~2 seconds) on a 100 Meg file. I finally discover that the delay is caused by the “word wrap”. When word wrap is "on" when the file is open (for that matter at any time), the 100 Meg file takes more than 60 seconds to settle down as well as the windows resize. When word wrap is "off" things become very fast. Unfortunately I did not have time to test the behavior of FunctionList. Maybe the problem is the word wrap thing and not the FunctionList plugging. I will test it very soon. I also noticed that with the word wrap off the Find in files search is very fast, but the first find takes about 5 seconds (100 Meg file), after that, it is almost instantaneous. I do not know what is different with the first search and the subsequent searches, but if the first search is as fast as the other searches it will be great.
Hope this help,
carlos
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Finally I have time to finish my testing. Here is what I found:
1- Word warp can take a lot of cpu time on large files. When word warp is “off “opening a 100 Meg file takes about 2 seconds if it is plain text, and about 3 seconds with syntax highlighting. I think it is excellent. When word wrap is on, the same file can take more than 60 seconds to open, a resize window takes about the same as well as other operations.
2- The FunctionList do not increase significantly the opening time or any other operation, the big problem is that it doubles (2.4) the amount of memory used by n++. Working with large files, that becomes an issue. Today I tried the FunctionLIst with a 38 Meg file. N++ uses 94 Meg to open the file (understandable if you want undos, etc), but when I click on FunctionList the memory goes to 218 Meg. That is, the amount of memory to open a file using FunctionList is about 6 times the size of the file.
3- When using Find all in the file, with only text or text highlighting, the first time that something is click on the found items, it takes 5 to 6 seconds to find the line (38 Meg file or larger), but after that it is almost instantaneous. It would be great if the first time can be speed up to match the subsequent searches.
4- I found today that if I try to copy a large section of a file and try to paste it to an open file (on the same session or with different sessions) n++ crashes both sessions. I was working with the 38 meg file, and try to copy maybe 10 meg of that file to the other one (empty), I could not do it. Maybe I will open a different thread to investigate that in more depth.
I hope this help to make N++ an even better text editor. I recognize that only a handful of people work with large files, but I know of only two software, jEdit and N++/Scintilla that do “hypersearch”, or find all in file and/or several files. jEdit is very powerful with small files, but when the files become large things goes south very quickly. So, my only hope is N++.
carlos
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
There is an idea about large file support on IdeaTorrent. One of the solution is to support editing of windowed files, ie just load a setion. You may want to support that. I never saw source files 10Mb or more in size, apart from unreasonable disassembly listings, but log files routinely do, and one hardly ever uses the whole o a log file.
CChris
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
From what I understand, there's a slim to none chance of such a feature ever seeing the light of day. That would require serious changes to the scintilla component.
IMO, unless Neil (or somebody else) does it in the Scintilla code base, it'll probably never get done.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
"Windowed" editing could be implemented without Scintilla changes, but it would take a decent amount of code, and would break most syntax highlighting.
I don't understand why people don't just use another editor when dealing with huge files - Notepad++'s goal is being a source code editor :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the suggestion. By the way I work on Finite Element Analsys. We recreate parts, components, etc by dividing them in very small "bricks"/pieces. I do not work with huge parts, put a typical model that I build can have a million nodes, each node has to be defined by the x,y, and z cordinates, plus each node belongs to an element, so we need to know what node goes with each element, etc, so, you can imagine that the amount of data grows very quickly. More than 90% are numbers, with some line in between. Those text files is what we called the input deck (definition and instructions for the FE software). My output results are on the Gig side. Fortunately they are binary and typically the FE software handle them.
I will try the IdeaTorrent. Thanks for all the support and patience.
carlos
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think proper use of Windows file mapping objects will make the changes trivial except for the UI. User will have to be reminde that some head and tail stuff should be invalidated - or there should be an option to load only full lines.
But I haven't looked seriously into this though.
CChris
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
A few days ago I started a thread called "Navigating using the find all in current document" https://sourceforge.net/forum/forum.php?thread_id=3295804&forum_id=331753 . Trying to overcome the problem I removed completely 5.3.1 and installed 5.4.3 and the plugging "Function List 2.0." By the way the plugging is great, thanks.
Here is what I found: When I open a 4 meg file, n++ 5.4.3 remains using 100% of cpu for ~5 extra seconds after the file is open; for a 12 Meg file it keeps running for extra ~10 seconds; and for a file ~100 meg it keeps running for 63 extra seconds after the file is open. I found that very weird, so I did the same exercise with scintilla; for the same files, once the file is open the cpu usage takes ~1/3 to 1/2 a second to go to zero. I removed completely 5.4.3 and went back to 5.3.1 and the same plugging. I did open the same files, and the behavior is the same as scintilla, basically once the file is open the cpu usage goes to zero until I do something with the file. On 5.4.3 even resizing the window recreates the extra cpu time. That could explain why the "navigating using the find all in current document" crashes the software, n++ is still doing something and sending one additional instruction crashes it. I have two Intel Xeon 3.4 MHz, 3.5 Gig memory and running XP professional. So, it is not a lack of resources. I noticed that if I call the function plugging while n++ is still going at it, my second cpu kicks in and uses it, while the other one is still going at 100% usage. The function/plugging finish, but n++ keeps going. I do not think is the plugging because when I discover the problem I did not have the plugging installed.
Hope this help.
Thanks.
I have a similar problem with quite large files with a size of 1 to 10 MB.
When I do a search for a word, the word is found immediately, indicated by the vertical scrollbar having moved to the correct position. But then Notepad++ takes some seconds, not obviously for what, before it shows the corresponding part of the file.
I guess this has to do something with the Syntax Highlighting. Those files are some simple log files, but I have SQL Syntax Highlighting applied to them. The only things that (have to) get highlighted are numbers, though.
Perhaps the same is the case for you. Funny, I didn't test those files and the find and show behavior without the Syntax Highlighting applied. Could you?
I have tried with both Syntax Highlighting and just pure text. In my case, the files are mainly numbers and I just look for a few lines that start with *. Also I noticed that when I use the Function Plugging, if the file is large the cursor location is off for a couple of lines. It can be very confusing. The cursor is located typically before/above the line that I'm looking for. With small files that does not happen. As I would expect the language highlighting slow down the file opening, but once the file opens the cpu usage should goes to zero.
> I do not think is the plugging because when I discover the problem I did not have the plugging installed.
To make sure it's not plugins issue, just remove ALL the plugins, then try to reproduce the problem. Please let me know the result of your test.
Don
Don thanks for taking the time to answer.
I discovered this morning that the Function Plugging double (2.4 times) the memory used by n++ (5.3.1 or 5.4.3). I did a test with a 38 Meg file and n++ 5.3.1. To open that file n++ takes 93 Meg, but when I call the Function plugging it jumps to 220 Meg. In addition when I close the Function window, it does not release the memory, it still hugs 220 Meg.
I just removed all the plugging from 5.3.1, the ones on the installation files as well as the ones on the app files. With the 38 Meg file, n++ still keeps going ~25 seconds after it opens the file (I can see the content of the first window). After the cpu usage goes to zero (after 24 seconds) I resize the window and n++ goes again on a ~25 seconds crunching. So, if you want to reproduce the problem, at least with 5.3.1, just open a “large file” and monitor the cpu usage, when the cpu goes to zero, the resize the window and it goes again for about the same time.
I have to re-install 5.4.3 since I removed it. I will post the results as soon as I can get to it.
Thanks,
carlos
Don, I removed completely 5.3.1 and installed 5.4.3 with the pluggins that come with the installation package. The only thing that I did was add my userDefinedLang.xml and modified a little the contextMenu file. Somehow it is working great, even the Find in Files and the navigation using the found line is working. Once in a while it get slightly slow, but the times went from 60 seconds to 3 or 4 seconds with a 100 Meg file. I think what was causing the problem with large files was the Function Plugging.
Thanks for a great software.
Well, the "couple of seconds" is due to the Syntax Highlighting, I guess. I was able to do some tests today.
A (quite) large SQL file is loaded and displayed immediately. When changing Syntax Highlighting to text, searching and showing the found match experiences no lag.
Turning Syntax Highlighting Language to SQL again, may take some seconds again.
It sure takes some seconds when:
- scrolling from a just opened SQL file to the bottom of it by using Ctrl+End
- opening a SQL file with the extension .txt (as normal text file by default) and then changing Syntax Highlighting to SQL
One the total file has been processed for Syntax Highlighting, the lags no longer occur.
The lag that you're experiencing also has something to do with Scintilla, which wasn't originally intended to work with extremely large text files.
Thanks for posting the results from your investigations with syntax highlighting. Maybe we could put a condition on newly opened files to use Normal Text when the file is above a given size?
Something for IdeaTorrent, maybe.
I think a better solution for big files is to add a "minimal mode" to Npp. Where Npp is in this mode, it should:
1. alert it is in "minimal mode" with a slide alert (as Firefox do when you can save passwords)
2. disable all plugins
3. disable all autocomplete features
4. do not apply syntax styles
5. disable all highlight and similar features
6. do not wrap the output
7. do not show any simbols (white chars, indent guide, ...)
8. disable indent features (folding etc.)
9. do not display vertical edges
10. do not save session at exit
11. disable clickable link feature
12. disable file status auto-detection
This mode can be started automatically when a large file is opened, as suggested, but it could also added as a command line argument.
I like this, though I'm not an NPP developer. Want to propose it as a new solution on IdeaTorrent here?
https://sourceforge.net/apps/ideatorrent/notepad-plus/ideatorrent/idea/24/
lucas_malor, eksortso,
I've added lucas_malor's idea to idea torrent and voted it up:
https://sourceforge.net/apps/ideatorrent/notepad-plus/ideatorrent/idea/24/
Do the same if you'd like.
Thanks for the extra testing. I agree the syntax higlighting should generate some extra work. What I have noticed is that the first search/go to is the one that takes the longest, after that it is almost instantaneous. My problem was that the first search was taking ~60 seconds on a 100 Meg file. With the clean installation it takes a few seconds with the syntax higlighting on. I think that time is very reasonable.
Hey I click the post to fast. This is what I was trying to said.
Thanks for the extra testing. I agree that syntax highlighting should generate some extra work. What I have noticed is that the first “search/go to” is the one that takes the longest, after that it is almost instantaneous, independent of the file size. My problem was that the first search was taking ~60 seconds on a 100 Meg file. With the clean installation of 5.4.3, n++ takes a few seconds to settle down when opening the 100 Meg file with the syntax highlighting on. For me that time is very reasonable. I think the software is working properly. Since I am not a programmer I do not know if those few extra seconds at the start can be worked out. For now I’m very happy. Don, thanks again for a great software.
You're welcome.
Could you confirm me the culprit is FunctionList plugin? Which version is it?
Don
Don,
First of all I was using FunctionList_2_0_Uni_dll.zip.
It is hard to say for sure, but this is what I have observed:
- n++ 5.3.1: it appears that some delay is associated with n++, because I did a test removing all the plugging directories and n++ still showed some delays after the file was open.
- n++5.4.3: once the FunctionList is installed, opening a 100 Meg file takes more than 60 seconds after the window shows the first page. In addition, once that the FunctionList is called the memory usage is about 2.4 timed of the original memory required to open the file, and the memory is not released when the FunctionList is closed. After the test, I removed the components of the FunctionList but now n++ is behaving badly again. That is 60 seconds to open the large file. Something is written somewhere that I’m not removing. I use the “uninstall” and remove all the n++ folders and did a clean installation and now n++ is working fine again. The FunctionList write something that messes up the behavior of n++.
Hope this help.
carlos
I did more testing with n+ 5.4.3 and the plugging’s in the original files. It took me a while to discover what is causing the problem. Sometimes it took very long time to open/settle down and some other times was “very” fast (~2 seconds) on a 100 Meg file. I finally discover that the delay is caused by the “word wrap”. When word wrap is "on" when the file is open (for that matter at any time), the 100 Meg file takes more than 60 seconds to settle down as well as the windows resize. When word wrap is "off" things become very fast. Unfortunately I did not have time to test the behavior of FunctionList. Maybe the problem is the word wrap thing and not the FunctionList plugging. I will test it very soon. I also noticed that with the word wrap off the Find in files search is very fast, but the first find takes about 5 seconds (100 Meg file), after that, it is almost instantaneous. I do not know what is different with the first search and the subsequent searches, but if the first search is as fast as the other searches it will be great.
Hope this help,
carlos
Finally I have time to finish my testing. Here is what I found:
1- Word warp can take a lot of cpu time on large files. When word warp is “off “opening a 100 Meg file takes about 2 seconds if it is plain text, and about 3 seconds with syntax highlighting. I think it is excellent. When word wrap is on, the same file can take more than 60 seconds to open, a resize window takes about the same as well as other operations.
2- The FunctionList do not increase significantly the opening time or any other operation, the big problem is that it doubles (2.4) the amount of memory used by n++. Working with large files, that becomes an issue. Today I tried the FunctionLIst with a 38 Meg file. N++ uses 94 Meg to open the file (understandable if you want undos, etc), but when I click on FunctionList the memory goes to 218 Meg. That is, the amount of memory to open a file using FunctionList is about 6 times the size of the file.
3- When using Find all in the file, with only text or text highlighting, the first time that something is click on the found items, it takes 5 to 6 seconds to find the line (38 Meg file or larger), but after that it is almost instantaneous. It would be great if the first time can be speed up to match the subsequent searches.
4- I found today that if I try to copy a large section of a file and try to paste it to an open file (on the same session or with different sessions) n++ crashes both sessions. I was working with the 38 meg file, and try to copy maybe 10 meg of that file to the other one (empty), I could not do it. Maybe I will open a different thread to investigate that in more depth.
I hope this help to make N++ an even better text editor. I recognize that only a handful of people work with large files, but I know of only two software, jEdit and N++/Scintilla that do “hypersearch”, or find all in file and/or several files. jEdit is very powerful with small files, but when the files become large things goes south very quickly. So, my only hope is N++.
carlos
There is an idea about large file support on IdeaTorrent. One of the solution is to support editing of windowed files, ie just load a setion. You may want to support that. I never saw source files 10Mb or more in size, apart from unreasonable disassembly listings, but log files routinely do, and one hardly ever uses the whole o a log file.
CChris
From what I understand, there's a slim to none chance of such a feature ever seeing the light of day. That would require serious changes to the scintilla component.
IMO, unless Neil (or somebody else) does it in the Scintilla code base, it'll probably never get done.
"Windowed" editing could be implemented without Scintilla changes, but it would take a decent amount of code, and would break most syntax highlighting.
I don't understand why people don't just use another editor when dealing with huge files - Notepad++'s goal is being a source code editor :)
Thanks for the suggestion. By the way I work on Finite Element Analsys. We recreate parts, components, etc by dividing them in very small "bricks"/pieces. I do not work with huge parts, put a typical model that I build can have a million nodes, each node has to be defined by the x,y, and z cordinates, plus each node belongs to an element, so we need to know what node goes with each element, etc, so, you can imagine that the amount of data grows very quickly. More than 90% are numbers, with some line in between. Those text files is what we called the input deck (definition and instructions for the FE software). My output results are on the Gig side. Fortunately they are binary and typically the FE software handle them.
I will try the IdeaTorrent. Thanks for all the support and patience.
carlos
I think proper use of Windows file mapping objects will make the changes trivial except for the UI. User will have to be reminde that some head and tail stuff should be invalidated - or there should be an option to load only full lines.
But I haven't looked seriously into this though.
CChris