[NSIS-Bugs] [ nsis-Bugs-2835731 ] Math plugin compiled with mingw32 causes an access violation

[SourceForge.net <no...@so...>]

Bugs item #2835731, was opened at 2009-08-11 19:55
Message generated for change (Tracker Item Submitted) made by f0rt
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=373085&aid=2835731&group_id=22049

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: f0rt (f0rt)
Assigned to: Nobody/Anonymous (nobody)
Summary: Math plugin compiled with mingw32 causes an access violation

Initial Comment:
Environment
Operating System: Linux / Debian Sid [i386]
NSIS version: 2.44-4 (installed via Debian package, build log: https://buildd.debian.org/fetch.cgi?pkg=nsis&arch=i386&ver=2.44-4&stamp=1237526925&file=log&as=raw)

How to reproduce the bug:
* Build Math plugin example on Debian Sid via the following command:
cd /tmp; makensis -NOCD /usr/share/doc/nsis/Examples/Math/math.nsi
* Copy the resulting math.exe file to a PC running Windows XP SP2 and execute math.exe by double clicking on its icon.
* The installer processes the Math::Script statements and prints out the results.
* Thereafter close the installer window and an access violation occurs.

Meticulously investigation revealed that the DllMain function of the Math plugin is not called so that
some values were not properly initialized.

The source files of the Math plugin actually require a C++ compiler to be built even so they 
have the ".c" file extension. The mingw32 C++ compiler is not directly invoked but rather
via "i586-mingw32msvc-gcc -xc++" as you can see in the build log for the NSIS 2.44-4 Debian package:
i586-mingw32msvc-gcc -o build/release/Math/Source/Math.o -c -Wall -g -O2 -Os -Wall -fno-strict-aliasing -xc++ "-DNSISCALL= __attribute__((__stdcall__))" -Ibuild/release/api Contrib/Math/Source/Math.c

As a consequence name mangling (name decoration) occurs so that the DllMain function
is no longer recognized as an entry function by the linker. This ultimately caused
the previously described access violation because the DllMain function provided by the
Math plugin was not called and some values were not properly initialized.

Turning off the name mangling for the DllMain function fixes the issue:
--- nsis-2.45-src/Contrib/Math/Source/Math.c    2009-02-04 15:08:31.000000000 +0100
+++ nsis-2.45/Contrib/Math/Source/Math.c        2009-08-11 09:17:01.000000000 +0200
@@ -1552,7 +1552,7 @@
  }
 }

-BOOL WINAPI DllMain(HINSTANCE hInst, ULONG ul_reason_for_call, LPVOID
lpReserved)
+extern "C" BOOL WINAPI DllMain(HINSTANCE hInst, ULONG
ul_reason_for_call, LPVOID lpReserved)
 {
    g_hInstance = hInst;
    CleanAll(ul_reason_for_call == DLL_PROCESS_ATTACH);



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=373085&aid=2835731&group_id=22049