[Libusb-win32-devel] libusb-win32 weird problem with 0.1.12.0 on XP SP1, but works with 0.1.10.1

[Spiro T. <an-...@sp...>]

Hello list members,

I am one of the authors of opencbm (http://sf.net/projects/opencbm). It
is a collection of tools (Linux and Windows kernel driver, shared
library, user tools) to access old Commodore equipment (floppies,
printers, etc.) on a modern PC.

All current releases support devices attached to the parallel port. We
are currently adding support for the xu1541
(http://www.harbaum.org/till/xu1541/), an USBTINY based ATMega8 device
to replace the parallel port.

If you want the current sources, you can get it through CVS with:

    cvs -z3 -d:pserver:ano...@op...:/cvsroot/opencbm co -r v0_4_0_48-xu1541_branch -P opencbm

and the xu1541 firmware (and the drivers) with
    cvs -z3 -d:pserver:ano...@op...:/cvsroot/opencbm co xu1541


The author of the xu1541 implemented support for Linux based systems
using libusb. It is now my task to port it over to Windows by using
libusb-win32.


Unfortunately, I discovered a problem. If I use libusb-win32 0.1.10.1,
everything works fine. If I use the version 0.1.10.2 (as it is called in
the download section; the INF calls itself 0.1.12.0), I have a big
problem, which results in crashes sometimes, and in access violations in
other times.


I think some notes on the architecture might be appropriate. The tool
(i.e., cbmctrl) uses a DLL (opencbm.dll), which itself uses another DLL
as plugin (in this case, opencbm-xu1541.dll). That DLL imported
libusb0.dll.

Note that opencbm-xu1541.dll is dynamically loaded by opencbm.dll by
using LoadLibrary(), GetProcAddress(), and FreeLibrary afterwards.

Additionally note that I am using the MS DDK compiler (3790.1830, 2003
SP1 DDK) and the DDK build tools for the project.

Thus, the outline is as follows:

    cbmctrl
       ^
       |    statically linked
       v 
    opencbm.dll
       ^
       |    dynamically linked (LoadLibrary, GetProcAddress())
       v 
    opencbm-xu1541.dll
       ^
       |    statically linked
       v 
    libusb0.dll



Interestingly, the crashes almost always occur on program exit. To me,
it seems the heap gets corrupted, which results in the crash.

Note that my problems are with XP SP2. Another tester has the same
problem. Somehow weird, another tester using Windows 2000 reports no
problems at all!


I started some debugging and found something. At least, I could
make go away that crash. Now, all commands which only use "reading" from
the USB work - that is, calls to _usb_io_sync() where "requesttype &
USB_ENDPOINT_IN" is true.

What did I change? I looked into the sources file for libusb-win32 and
found out that it uses MSVCRT.DLL as run-time library (instead of a
statically linked run-time library).

I changed opencbm.dll and opencbm-xu1541.dll to use the same DLL, and
these crashes are gone now.

Note that I am no expert on DLLs. Is this the expected behaviour? Should
it be the same? Why should "my" DLLs use the same run-time library as
you?



Unfortunately, the other problem (with access violations) remains for
the commands where _usb_io_sync() "requesttype & USB_ENDPOINT_IN" is
false.

I debugged it as follows in WinDBG: I compiled libusb0.dll myself (using
make_dll in the ddk_make/ directory, and utilizing the LIB, DLL and PDB
I got).

Have a look at libusb/src/windows.c
(http://libusb-win32.svn.sourceforge.net/viewvc/libusb-win32/trunk/libusb/src/windows.c?revision=148&view=markup),
function usb_control_msg() (lines 659ff):

Everything works fine as long as the if() in line 770 (if(!(requesttype
& USB_ENDPOINT_IN))) is not true. Line 783ff (_usb_io_sync()) succeeds,
and everytings works.

This is the reason why the access violation depends upon the requesttype
& USB_ENDPOINT_IN value.

If the if() in line 770 is true, lines 771-781 malloc()s a new memory
area for the request req. req is a local variable, which is copied over
the newly allocated memory. Additionally, the output data is copied
after that request.

I can see that the malloc() succeeds, and the copying works as expected
(at least as I expect it), too. Nevertheless, the call in line 783
(_usb_io_sync()) fails now!

Have a look into _usb_io_sync(), line 1170 of windows.c: It fails with
error code 0xc0000005 (access violation). I can clearly see in the
debugger that the memory referenced by the pointer out is valid, as is
the data referenced by the pointer in. I can follow the OS function
DeviceIoControl() until the SYSENTER. All pointers up to there are valid
and correctly used. Doing the next step, I get the access violation.

To be honest, I don't have a clue what is really going on, and how this
can be solved.

Unfortunately, I cannot debug this better, as

1. I don't have a second XP based machine to debug the call into
   kernel-mode, too, and

2. somehow, I have not managed to get libusb-win32 claim the device
   inside of VMWare (which would allow me to debug the kernel calls,
   too).


I am not sure if this is somehow related to restrictions of a DLL. I
remembered that malloc() and printf() are problematic in a DLL. I came
up with the following article in MSDN:

   http://support.microsoft.com/?scid=kb%3Ben-us%3B94248&x=15&y=15

I thought I might have some problem with the initialization of the CRT.
For this, I changed all DLL entry points in my code to DllEntry();
anyway, the problems did not go away.


Any hints and ideas are welcome.

Regards,
   Spiro.

-- 
Spiro R. Trikaliotis                              http://opencbm.sf.net/
http://www.trikaliotis.net/                     http://www.viceteam.org/