libusb-win32-devel

[Libusb-win32-devel] libusb-win32 weird problem with 0.1.12.0 on XP SP1, but works with 0.1.10.1

[Spiro T. <an-...@sp...>]

Hello list members,

I am one of the authors of opencbm (http://sf.net/projects/opencbm). It
is a collection of tools (Linux and Windows kernel driver, shared
library, user tools) to access old Commodore equipment (floppies,
printers, etc.) on a modern PC.

All current releases support devices attached to the parallel port. We
are currently adding support for the xu1541
(http://www.harbaum.org/till/xu1541/), an USBTINY based ATMega8 device
to replace the parallel port.

If you want the current sources, you can get it through CVS with:

    cvs -z3 -d:pserver:ano...@op...:/cvsroot/opencbm co -r v0_4_0_48-xu1541_branch -P opencbm

and the xu1541 firmware (and the drivers) with
    cvs -z3 -d:pserver:ano...@op...:/cvsroot/opencbm co xu1541


The author of the xu1541 implemented support for Linux based systems
using libusb. It is now my task to port it over to Windows by using
libusb-win32.


Unfortunately, I discovered a problem. If I use libusb-win32 0.1.10.1,
everything works fine. If I use the version 0.1.10.2 (as it is called in
the download section; the INF calls itself 0.1.12.0), I have a big
problem, which results in crashes sometimes, and in access violations in
other times.


I think some notes on the architecture might be appropriate. The tool
(i.e., cbmctrl) uses a DLL (opencbm.dll), which itself uses another DLL
as plugin (in this case, opencbm-xu1541.dll). That DLL imported
libusb0.dll.

Note that opencbm-xu1541.dll is dynamically loaded by opencbm.dll by
using LoadLibrary(), GetProcAddress(), and FreeLibrary afterwards.

Additionally note that I am using the MS DDK compiler (3790.1830, 2003
SP1 DDK) and the DDK build tools for the project.

Thus, the outline is as follows:

    cbmctrl
       ^
       |    statically linked
       v 
    opencbm.dll
       ^
       |    dynamically linked (LoadLibrary, GetProcAddress())
       v 
    opencbm-xu1541.dll
       ^
       |    statically linked
       v 
    libusb0.dll



Interestingly, the crashes almost always occur on program exit. To me,
it seems the heap gets corrupted, which results in the crash.

Note that my problems are with XP SP2. Another tester has the same
problem. Somehow weird, another tester using Windows 2000 reports no
problems at all!


I started some debugging and found something. At least, I could
make go away that crash. Now, all commands which only use "reading" from
the USB work - that is, calls to _usb_io_sync() where "requesttype &
USB_ENDPOINT_IN" is true.

What did I change? I looked into the sources file for libusb-win32 and
found out that it uses MSVCRT.DLL as run-time library (instead of a
statically linked run-time library).

I changed opencbm.dll and opencbm-xu1541.dll to use the same DLL, and
these crashes are gone now.

Note that I am no expert on DLLs. Is this the expected behaviour? Should
it be the same? Why should "my" DLLs use the same run-time library as
you?



Unfortunately, the other problem (with access violations) remains for
the commands where _usb_io_sync() "requesttype & USB_ENDPOINT_IN" is
false.

I debugged it as follows in WinDBG: I compiled libusb0.dll myself (using
make_dll in the ddk_make/ directory, and utilizing the LIB, DLL and PDB
I got).

Have a look at libusb/src/windows.c
(http://libusb-win32.svn.sourceforge.net/viewvc/libusb-win32/trunk/libusb/src/windows.c?revision=148&view=markup),
function usb_control_msg() (lines 659ff):

Everything works fine as long as the if() in line 770 (if(!(requesttype
& USB_ENDPOINT_IN))) is not true. Line 783ff (_usb_io_sync()) succeeds,
and everytings works.

This is the reason why the access violation depends upon the requesttype
& USB_ENDPOINT_IN value.

If the if() in line 770 is true, lines 771-781 malloc()s a new memory
area for the request req. req is a local variable, which is copied over
the newly allocated memory. Additionally, the output data is copied
after that request.

I can see that the malloc() succeeds, and the copying works as expected
(at least as I expect it), too. Nevertheless, the call in line 783
(_usb_io_sync()) fails now!

Have a look into _usb_io_sync(), line 1170 of windows.c: It fails with
error code 0xc0000005 (access violation). I can clearly see in the
debugger that the memory referenced by the pointer out is valid, as is
the data referenced by the pointer in. I can follow the OS function
DeviceIoControl() until the SYSENTER. All pointers up to there are valid
and correctly used. Doing the next step, I get the access violation.

To be honest, I don't have a clue what is really going on, and how this
can be solved.

Unfortunately, I cannot debug this better, as

1. I don't have a second XP based machine to debug the call into
   kernel-mode, too, and

2. somehow, I have not managed to get libusb-win32 claim the device
   inside of VMWare (which would allow me to debug the kernel calls,
   too).


I am not sure if this is somehow related to restrictions of a DLL. I
remembered that malloc() and printf() are problematic in a DLL. I came
up with the following article in MSDN:

   http://support.microsoft.com/?scid=kb%3Ben-us%3B94248&x=15&y=15

I thought I might have some problem with the initialization of the CRT.
For this, I changed all DLL entry points in my code to DllEntry();
anyway, the problems did not go away.


Any hints and ideas are welcome.

Regards,
   Spiro.

-- 
Spiro R. Trikaliotis                              http://opencbm.sf.net/
http://www.trikaliotis.net/                     http://www.viceteam.org/

Re: [Libusb-win32-devel] =?iso-8859-15?q?libusb-win32_weird_problem_w?= =?iso-8859-15?q?ith_0=2E1=2E12=2E0_on_XP_SP1=2C_but_works_with_0=2E1=2E10?= =?iso-8859-15?q?=2E1?=

[Stephan M. <ste...@we...>]

> [...]
>=20
>=20
> Unfortunately, I discovered a problem. If I use libusb-win32 0.1.10.1,
> everything works fine. If I use the version 0.1.10.2 (as it is called in=

> the download section; the INF calls itself 0.1.12.0), I have a big
> problem, which results in crashes sometimes, and in access violations in=

> other times.

The correct version is 0.1.12.0. I fixed it.

>=20
>=20
> I think some notes on the architecture might be appropriate. The tool
> (i.e., cbmctrl) uses a DLL (opencbm.dll), which itself uses another DLL
> as plugin (in this case, opencbm-xu1541.dll). That DLL imported
> libusb0.dll.
>=20
> Note that opencbm-xu1541.dll is dynamically loaded by opencbm.dll by
> using LoadLibrary(), GetProcAddress(), and FreeLibrary afterwards.

It should make no difference whether the DLLs are "statically linked" or
loaded at runtime.

> [...]
>=20
> Interestingly, the crashes almost always occur on program exit. To me,
> it seems the heap gets corrupted, which results in the crash.
>=20
> Note that my problems are with XP SP2. Another tester has the same
> problem. Somehow weird, another tester using Windows 2000 reports no
> problems at all!
>=20
>=20
> I started some debugging and found something. At least, I could
> make go away that crash. Now, all commands which only use "reading" from=

> the USB work - that is, calls to =5Fusb=5Fio=5Fsync() where "requesttype &
> USB=5FENDPOINT=5FIN" is true.
>=20
> What did I change=3F I looked into the sources file for libusb-win32 and
> found out that it uses MSVCRT.DLL as run-time library (instead of a
> statically linked run-time library).
>=20
> I changed opencbm.dll and opencbm-xu1541.dll to use the same DLL, and
> these crashes are gone now.
>=20
> Note that I am no expert on DLLs. Is this the expected behaviour=3F Should=

> it be the same=3F Why should "my" DLLs use the same run-time library as
> you=3F

I'm no DLL expert either but it's usually good practice to link all system=

libraries such as msvcrt.dll dynamically because:

* this saves memory and disk space
* speeds up application startup
* ensures that all components of an application use exactly the same=20
   library/DLL versions

>=20
> Unfortunately, the other problem (with access violations) remains for
> the commands where =5Fusb=5Fio=5Fsync() "requesttype & USB=5FENDPOINT=5FIN" is
> false.
>=20
> I debugged it as follows in WinDBG: I compiled libusb0.dll myself (using=

> make=5Fdll in the ddk=5Fmake/ directory, and utilizing the LIB, DLL and PDB
> I got).
>=20
> Have a look at libusb/src/windows.c
> (http://libusb-win32.svn.sourceforge.net/viewvc/libusb-win32/trunk/libus=
b/src/windows.c=3Frevision=3D148&view=3Dmarkup),
> function usb=5Fcontrol=5Fmsg() (lines 659ff):
>=20
> Everything works fine as long as the if() in line 770 (if(!(requesttype
> & USB=5FENDPOINT=5FIN))) is not true. Line 783ff (=5Fusb=5Fio=5Fsync()) succeeds,
> and everytings works.
>=20
> This is the reason why the access violation depends upon the requesttype=

> & USB=5FENDPOINT=5FIN value.
>=20
> If the if() in line 770 is true, lines 771-781 malloc()s a new memory
> area for the request req. req is a local variable, which is copied over
> the newly allocated memory. Additionally, the output data is copied
> after that request.
>=20
> I can see that the malloc() succeeds, and the copying works as expected
> (at least as I expect it), too. Nevertheless, the call in line 783
> (=5Fusb=5Fio=5Fsync()) fails now!
>=20
> Have a look into =5Fusb=5Fio=5Fsync(), line 1170 of windows.c: It fails with
> error code 0xc0000005 (access violation). I can clearly see in the
> debugger that the memory referenced by the pointer out is valid, as is
> the data referenced by the pointer in. I can follow the OS function
> DeviceIoControl() until the SYSENTER. All pointers up to there are valid=

> and correctly used. Doing the next step, I get the access violation.

Have you verified that all parameters passed to DeviceIoControl() are
valid and correct=3F Please check the 'ol' parameter. Is it corrently initia=
lized
and is 'ol.hEvent' valid=3F

>=20
> To be honest, I don't have a clue what is really going on, and how this
> can be solved.
>=20
> Unfortunately, I cannot debug this better, as
>=20
> 1. I don't have a second XP based machine to debug the call into
>    kernel-mode, too, and
>=20
> 2. somehow, I have not managed to get libusb-win32 claim the device
>    inside of VMWare (which would allow me to debug the kernel calls,
>    too).

Where does the crash and the access violation occur=3F In user-space
or in kernel space=3F Do you get a BSOD=3F
If the access violation occurs in user-space then debugging in kernel-spac=
e
won't make much sense.

>=20
>=20
> I am not sure if this is somehow related to restrictions of a DLL. I
> remembered that malloc() and printf() are problematic in a DLL. I came
> up with the following article in MSDN:
>=20
>    http://support.microsoft.com/=3Fscid=3Dkb%3Ben-us%3B94248&x=3D15&y=3D15
>=20
> I thought I might have some problem with the initialization of the CRT.
> For this, I changed all DLL entry points in my code to DllEntry();
> anyway, the problems did not go away.

Thanks for the link. I changed the DLL's entry point to =5FDllMainCRTStartup=
@12=20
as recommended by the article. A new version is attached. Please give it a=
 try.


>=20
>=20
> Any hints and ideas are welcome.
>=20
> Regards,
>    Spiro.
>=20
> --=20
> Spiro R. Trikaliotis                              http://opencbm.sf.net/=

> http://www.trikaliotis.net/                     http://www.viceteam.org/=

>=20
> ------------------------------------------------------------------------=
-
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share =
your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php=3Fpage=3Djoin.php&p=3Dsourceforge&CID=3DDEVDE=
V
> =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
> Libusb-win32-devel mailing list
> Lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel
>=20


=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Viren-Scan f=FCr Ihren PC! Jetzt f=FCr jeden. Sofort, online und kostenlos.
Gleich testen! http://www.pc-sicherheit.web.de/freescan/=3Fmc=3D022222

Re: [Libusb-win32-devel] libusb-win32 weird problem with 0.1.12.0 on XP SP1, but works with 0.1.10.1

[Spiro T. <an-...@sp...>]

Hello Stephan,

* On Sat, Mar 17, 2007 at 11:20:40AM +0100 Stephan Meyer wrote:
 
> The correct version is 0.1.12.0. I fixed it.

Thank you. I expected it that way, but I was not sure.


> > Note that opencbm-xu1541.dll is dynamically loaded by opencbm.dll by
> > using LoadLibrary(), GetProcAddress(), and FreeLibrary afterwards.
> 
> It should make no difference whether the DLLs are "statically linked" or
> loaded at runtime.

I believe this, too. Anyway, I also thought that it should not make any
difference regardless of which run-time library is used for which part,
but it is not. Thus, I reported this for completeness.
 
> I'm no DLL expert either but it's usually good practice to link all system
> libraries such as msvcrt.dll dynamically because:
> 
> * this saves memory and disk space
> * speeds up application startup
> * ensures that all components of an application use exactly the same 
>    library/DLL versions

Well, at least the DDK samples do not use MSVCRT.DLL, but use the
default (single-threaded statically linked library). That's the reason
why I used this approach, too.


> > Have a look into _usb_io_sync(), line 1170 of windows.c: It fails with
> > error code 0xc0000005 (access violation). I can clearly see in the
> > debugger that the memory referenced by the pointer out is valid, as is
> > the data referenced by the pointer in. I can follow the OS function
> > DeviceIoControl() until the SYSENTER. All pointers up to there are valid
> > and correctly used. Doing the next step, I get the access violation.
> 
> Have you verified that all parameters passed to DeviceIoControl() are
> valid and correct? Please check the 'ol' parameter. Is it corrently initialized
> and is 'ol.hEvent' valid?

This was a very good hint!

I looked again and found that the input buffer "in" and "in_size" is not
null if _usb_io_sync() is called. I set them to NULL in case we want to
output anything on the USB bus, cf. the patch attached.  With this,
everything works fine!

Note that the in buffer in is initialized with the input parameter bytes
in usb_control_msg(). Now, the "bytes" parameter I give is just a
constant string parameter. It seems my compiler puts it into a memory
area which is not writable; thus, the 0xc0000005 "no access" occurs.

I am not sure if my fix is the right solution for everyone. I don't know
if usb_control_msg() can give back data even if this is an output call.
Anyway, at least for all my tests, it solved the problem for me.


> > To be honest, I don't have a clue what is really going on, and how this
> > can be solved.
> > 
> > Unfortunately, I cannot debug this better, as
> > 
> > 1. I don't have a second XP based machine to debug the call into
> >    kernel-mode, too, and
> > 
> > 2. somehow, I have not managed to get libusb-win32 claim the device
> >    inside of VMWare (which would allow me to debug the kernel calls,
> >    too).
> 
> Where does the crash and the access violation occur? In user-space
> or in kernel space? Do you get a BSOD?

As I told, the crash is gone if I link my DLL against MSVCRT.DLL, which
I am doing currently.

The access violation is reported as a normal GetLastError() result. It
occurs after the call to SYSENTER, thus, I am perfectly sure that it
happens in kernel-mode.

I have done some more debugging: I added some outputs in the dispatch()
routine in the driver libusb0.sys. It is not called from _usb_io_sync()
if I get the access violation. So, I conclude that it is not your driver
which reports this access violation, but the operating system.

Still, I don't know yet where exactly this happens, and which parameter
is the culprit.


So: With MSVCRT, I do not get any crash. Even if I do not use MSVCRT, I
don't get a BSOD. I just get that pop-up window, telling me that there
was a memory access (read) that should not have been there.

> If the access violation occurs in user-space then debugging in kernel-space
> won't make much sense.

Yes, of course.

> > I thought I might have some problem with the initialization of the CRT.
> > For this, I changed all DLL entry points in my code to DllEntry();
> > anyway, the problems did not go away.
> 
> Thanks for the link. I changed the DLL's entry point to _DllMainCRTStartup@12 
> as recommended by the article. A new version is attached. Please give it a try.

I am sorry, the attachment is empty here.


So, to summarize: IF I link my DLL against MSVCRT, and I use the
attached patch, everything works fine for me (now).

Regards,
   Spiro.

-- 
Spiro R. Trikaliotis                              http://opencbm.sf.net/
http://www.trikaliotis.net/                     http://www.viceteam.org/

Re: [Libusb-win32-devel] libusb-win32 weird problem with 0.1.12.0 on XP SP1, but works with 0.1.10.1

[Spiro T. <an-...@sp...>]

Hello again Stephan,

another investigation:

* On Sat, Mar 17, 2007 at 03:19:46PM +0100 I wrote:
 
> I am not sure if my fix is the right solution for everyone. I don't know
> if usb_control_msg() can give back data even if this is an output call.
> Anyway, at least for all my tests, it solved the problem for me.

To me, it seems my fix is absolutely correct.

I looked into the SVN and found out that the relevant change was done
with revision 115
(http://libusb-win32.svn.sourceforge.net/viewvc/libusb-win32/trunk/libusb/src/windows.c?r1=109&r2=115),
where you changed from direct calls to DeviceIoControl() to calls to
usb_io_sync().

The line 690ff and 731ff (old implementation, revision 109) uses NULL
and 0 for the equivalent to the in and in_size parameters. The new
implementation (line 741, revision 115) does not reset in and in_size,
which results in the problem.

Interestingly, revision 115 was checked in on February 9, 2006 - which
perfectly fits with my observation that 0.1.10.1 (from 2005-03-09) works
perfectly, while 0.1.12 (from 2007-02-25) does not.

Regards,
   Spiro.

-- 
Spiro R. Trikaliotis                              http://opencbm.sf.net/
http://www.trikaliotis.net/                     http://www.viceteam.org/