#215 HexEditor 0.9.5 resource exhaustion

open
Jens Lorenz
HexEditor (36)
5
2013-02-01
2013-02-01
Ariel Poliak
No

The HexEditor plugin (0.9.5 Unicode) will consume all available system resources under the following conditions:
1. The OS is WIndows 7 (may also occur in Vista/8)
2. The DLL is placed on the global plugins directory (e.g. "c:\Program Files (x86)\Notepad++\Plugins" )
3. Per-user plugins are disabled (default)
4. Notepad++ has not been used by the current user (e.g. "%APPDATA%\Notepad++\plugins" does not exist)

When Notepad++.exe is launched, it will start loading plugins.
Once the HexEditor DLL is loaded, it will start an endless loop of CreateFile() calls with "%APPDATA%\Notepad++\plugins" as the target.
(this is shown in the attached ProcMon log file)

However, by manually creating the "%APPDATA%\Notepad++\plugins" directory before launching Notepad++.exe, a message box stating "Unknown Exception" appears, but once acknowledged, the plugin loads successfully and Notepad++ behaves as expected.

Re-opening Notepad++ again shows no message box, and Notepad++ behaves as expected

Discussion

  • Ariel Poliak
    Ariel Poliak
    2013-02-01

    Log files are too large for upload to SourceForge,
    They have been uploaded here: https://www.dropbox.com/sh/znxvdv11m7r532m/ULIZwgQuAs

    "cleanslate.pml" - first run, post-installation
    "firstaftermd.pml" - second run, after manually creating per-user plugins directory
    "nextaftermd.pml" - third run, after closing second run.

     
  • Ariel Poliak
    Ariel Poliak
    2013-02-01

    A quick glance at the plugin's source leads me to believe that the issue may lie in the loadSettings() function within NppPlugins/HexEditor/src/Hex.cpp

     
  • Ariel Poliak
    Ariel Poliak
    2013-02-01

    lines 338-342 are responsible:

    *_tcsrchr(configPath, '\\') = NULL;
    do {
    vPaths.push_back(configPath);
    *_tcsrchr(configPath, NULL) = NULL;
    } while (::PathFileExists(configPath) == FALSE);
    ===
    The code above is supposed to add directory names to the vPaths vector in such a way that when the loop in lines 344-349 executes, the proper directory structure will be created in the correct order.

    However, the code in line 341, «*_tcsrchr(configPath, NULL) = NULL;», does not strip out a directory level. It replaces the null string delimiter with another null delimiter.
    This is what causes the infinite loop.

    If the code read «*_tcsrchr(configPath, '\\') = NULL;» instead (like in line 338), the function would work as expected.

     
  • Ariel Poliak
    Ariel Poliak
    2013-02-01

    Forgot to mention, the reason for resource exhaustion is the vPaths.push_back() call within the infinite loop.

     
  • Ariel Poliak
    Ariel Poliak
    2013-02-01

    Patch with fix added to this artifact.