Drastic search speed change over versions
A Windows File Searching Utility (grep)
Brought to you by:
jackslade,
theodoreward
Menu
▾
▴
#69 Drastic search speed change over versions
User Ian reported that upgrading to this version from a previous one (unknown) reduced the overall search speed quite a bit. I suggested turning off file encoding detection to see if that helps.
Email:
Just updated to the latest version and the search speed has taken a real nosedive.
I have 124,273 .txt files in various folders all under 1 folder (10 years of documents).
With the previous version I had installed (not sure which version) a search of the files took approximately 30 minutes.
So far I have had a search running for 5 hours and it’s still only up to January 2015
Discussion
Anonymous
Made a tiny improvement in 4.4.1 by reading encoding sample from same stream as grep so not to open a file twice.
I have 4.4.1 and just upgraded from v3 to 4.4 and then 4.4.1 Version 3 SMASHES the new version. I'm scanning a lot of files too so I really notice the degraded performance.
That said, the thing that troubles me is that there's not much difference feature wise between 3 and 4, so I can't see why it needs to be slower.
Turning off file encoding detection should revert to the previous speed (Tools -> Options -> File Encoding tab -> Unselect Detect File Encoding.
If you don't need to search against unicode character sets in your files, for example non-ASCII characters or Japanese characters, then turning it off won't really affect the results. However, if searching against something like that is needed, you need to leave it on so the correct file encoding is used.
We are looking into how to improve the detection speed, better configure it for speed over accuracy, and/or more detailed settings that can turn on/off and order the encoding detection algorithms.
Fixed/addressed in version 4.4.2. We added a performance setting and caching to the file encoding detection via Tools->Options->File Encoding. You can now choose Speed, Default, Accuracy as options that will increase the sample size and number of detectors used in increasing order.