[Mingw-users] Wrong DLL init routine being called?

[David D. <dav...@rs...>]

Hi,

I have a very strange problem that I've been working on for a few days.

When I try to load my application I'm receiving the message:

"The application failed to initialize properly (0xc0000005). Click on OK
to terminate the application."

At first I thought that this was probably due to the .rdata auto-import
problem, but I used the linker script and the problem remains.  On top
of this most of the libraries in question are explicitly exported and
imported with dllimport/export.

So I used GDB and got a really strange stack trace:

(gdb) bt
#0  0x01c53d90 in malloc ()
    at
d:/srcbuilds/xerces-c-src_2_7_0/include/xercesc/validators/datatype/Datat
ypeValidator.hpp:691
#1  0x01a710cc in lab ()
#2  0x7c9011a7 in ntdll!LdrSetAppCompatDllRedirectionCallback ()
   from C:\WINDOWS\system32\ntdll.dll
#3  0x01a70000 in ?? ()
#4  0x00000001 in ?? ()
#5  0x0022fd30 in ?? ()
#6  0x01a71000 in ?? ()
#7  0x00000001 in ?? ()
#8  0x00353480 in ?? ()
#9  0x0022fb18 in ?? ()
#10 0x7c91cbab in ntdll!LdrHotPatchRoutine ()
   from C:\WINDOWS\system32\ntdll.dll
#11 0x01a71000 in ?? ()
#12 0x01a70000 in ?? ()
#13 0x00000001 in ?? ()
#14 0x0022fd30 in ?? ()
#15 0x00000000 in ?? ()

I found it odd that the init routine landed the code up in
DatatypeValidator.hpp:691, so I used windbg to verify that that call was
indeed part of the init routine.  I also turned on showsnaps to see what
was happening...

[...snip...]
[8a4,8ac] LDR: COMDLG32.DLL loaded - Calling init routine at 763B1AB8
[8a4,8ac] LDR: OLEAUT32.DLL loaded - Calling init routine at 77121558
[8a4,8ac] LDR: wxmsw28_core_gcc_custom.dll loaded - Calling init routine
at 6E501000
[8a4,8ac] LDR: libwxextctrls.dll loaded - Calling init routine at
64601000
[8a4,8ac] LDR: libitex.dll loaded - Calling init routine at 66E01000
[8a4,8ac] LDR: libitsdk.dll loaded - Calling init routine at 02671000
[8a4,8ac] LDR: WS2HELP.dll loaded - Calling init routine at 71AA1642
[8a4,8ac] LDR: WS2_32.DLL loaded - Calling init routine at 71AB1273
[8a4,8ac] LDR: libnetch.dll loaded - Calling init routine at 6CD01000
[8a4,8ac] LDR: libxerces-c2_7_0.dll loaded - Calling init routine at
02C71000
(8a4.8ac): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=0022fa7c ebx=02c71000 ecx=7c91eb05 edx=0022f7f3 esi=0022fa04
edi=00000001
eip=02e53d90 esp=0022f9d4 ebp=0022f9f0 iopl=0         nv up ei pl zr na
pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000
efl=00010246
*** ERROR: Symbol file could not be found.  Defaulted to export symbols
for D:\Program Files\RS&A\IniTrack2\libxerces-c2_7_0.dll - 
libxerces_c2_7_0!
ZN11xercesc_2_726YearMonthDatatypeValidator9serializeERNS_16XSerializeEngineE+0x3ffe:
02e53d90 ff25d8a27897    jmp     dword ptr ds:[9778A2D8h]
ds:0023:9778a2d8=????????
0:000> u
libxerces_c2_7_0!
ZN11xercesc_2_726YearMonthDatatypeValidator9serializeERNS_16XSerializeEngineE+0x3ffe:
02e53d90 ff25d8a27897    jmp     dword ptr ds:[9778A2D8h]
02e53d96 90              nop
02e53d97 90              nop
02e53d98 0000            add     byte ptr [eax],al
02e53d9a 0000            add     byte ptr [eax],al
02e53d9c 0000            add     byte ptr [eax],al
02e53d9e 0000            add     byte ptr [eax],al
02e53da0 ff25dca27897    jmp     dword ptr ds:[9778A2DCh]
0:000> u 02c71000
libxerces_c2_7_0+0x1000:
02c71000 55              push    ebp
02c71001 89e5            mov     ebp,esp
02c71003 57              push    edi
02c71004 56              push    esi
02c71005 53              push    ebx
02c71006 83ec0c          sub     esp,0Ch
02c71009 8b7d0c          mov     edi,dword ptr [ebp+0Ch]
02c7100c 83ff01          cmp     edi,1
0:000> u
libxerces_c2_7_0+0x100f:
02c7100f 0f84ab000000    je      libxerces_c2_7_0+0x10c0 (02c710c0)
02c71015 897c2404        mov     dword ptr [esp+4],edi
02c71019 8b5d10          mov     ebx,dword ptr [ebp+10h]
02c7101c 8b4d08          mov     ecx,dword ptr [ebp+8]
02c7101f 895c2408        mov     dword ptr [esp+8],ebx
02c71023 890c24          mov     dword ptr [esp],ecx
02c71026 e8851a1e00      call    libxerces_c2_7_0!
ZN11xercesc_2_726YearMonthDatatypeValidator9serializeERNS_16XSerializeEngineE+0x2d1e (02e52ab0)
02c7102b 89c6            mov     esi,eax

Indeed, whatever it is calling for the init routine has that call to
YearMonthDatatypeValidator.serialize in it, but as far as I can tell it
shouldn't.  Could there be something going wrong during relocation, or
some other step that is causing the wrong section of code to be called
instead of the proper init routine?  I know that there are probably
calls to global constructors and destructors during init, but that call
to YearMonthDatatypeValidator9serialize looks like it is right at the
init routine, which doesn't make sense to me because it's not a ctor,
it's a method.

(as far as I can tell xerces-c has no DLL main)

I am loading a ton of DLLs and it's really hard to reproduce this with a
small example.  When I switch one of the other libraries I'm using
(wxwidgets) to debug mode, this problem stops, but I get all sorts of
unnecessary popups to the end user.  I don't know if it's some kind of
relocator interaction or what..  It does seem to stem around wxwidgets
and xerces-c, both of which have a TON of exports.

Any help would be greatly appreciated.  I'm really stuck and have no
idea where to go from here apart from following the assembly step by
step.

- Dave