libusb-devel

[Libusb-devel] Slow USB bulk transfer

[mohammed a. <abu...@gm...>]

Hello,

I have the following code that takes more than a min to finisg:

-------------------------------------------------------
#include <stdio.h>
#include <string.h>
#include <usb.h>

// Search token for the cypress chip.
#define CYP_VENDOR_ID   0x04b4
#define CYP_PRODUCT_ID 0x8613

struct usb_device* cypress_init();
void sighandler(int );


// Global vars
struct usb_device* cypress_dev;
struct usb_dev_handle* cypress_dev_handle;
char* bytes;
struct usb_interface_descriptor interface ;

int verbose = 0;
int c,i,a,ret;

int main(int argc, char* argv[]){
// First, init the cypress chip..
cypress_dev = cypress_init();
// Second, open it ...
if (cypress_dev) cypress_dev_handle = usb_open(cypress_dev);
else {
    printf("Fatel Error: No Cypress CY7C68xx devices connected. Exiting
\n");
    return -9;
}

if (cypress_dev_handle){
// Now, detatch the usbtest kernel driver assigned to the cypress chip upon
//hotpluging it into the system
usb_detach_kernel_driver_np(cypress_dev_handle ,0);

usb_set_configuration(cypress_dev_handle,
cypress_dev->config->bConfigurationValue);
interface = cypress_dev->config[0].interface[0].altsetting[1];

usb_claim_interface(cypress_dev_handle,interface.bInterfaceNumber);
usb_set_altinterface(cypress_dev_handle,interface.bAlternateSetting);

//assign the endpoint addresses....
int epo   = interface.endpoint[0].bEndpointAddress;
int epi   = interface.endpoint[1].bEndpointAddress;

usb_clear_halt(cypress_dev_handle, epi );

//printf ("ep_out is %0x while ep_in is %0x\n",epo,epi);
bytes = (char*) malloc(512);
memset(bytes,0,sizeof(bytes));

/*write 40 Mbytes worth of data to the bulk endpoint*/
/*i am calculating this by: 512 (packet size) * 2 = 1 KB, * 1000 = 1MB, * 40
= 40 MB*/
for (i = 0; i < 2 * 1000 * 40 ; i++){
ret = usb_bulk_write(cypress_dev_handle, epi,bytes, 512, 0 /*the timrout is
ignored for bulk transfer*/);
//printf ("i is:%d\t ret is %d\t bytes is %s\n",i, ret,bytes);
}

}
else {
    printf("Could not open the Cypress chip. Quiting \n");
    return -1;
    }

usb_release_interface(cypress_dev_handle,0);
usb_close(cypress_dev_handle);

return 0;
}

struct usb_device* cypress_init(){

struct usb_bus *bus =NULL;
struct usb_device *dev =NULL;

usb_init();
usb_find_busses();
usb_find_devices();
usb_set_debug(9);

for (bus = usb_busses; bus;  bus = bus->next)
   for (dev = bus->devices; dev;  dev = dev->next)
       if ((dev->descriptor.idVendor ==  CYP_VENDOR_ID)    &&
             (dev->descriptor.idProduct ==  CYP_PRODUCT_ID)) return
dev;

return NULL;
}

void sighandler(int sig){

//usb_release_interface(cypress_dev_handle,0);
usb_close(cypress_dev_handle);

}

-------------------------------------------------------

As root, I ran "time ./a.out" and got

# time ./a.out
real    1m9.793s
user    0m0.118s
sys     0m0.490s

# time ./a.out

real    1m15.078s
user    0m0.095s
sys     0m0.438s
root@elive[/home/me/cypress]#


The lsusb -vv output of the cypress FX2 device is at:

http://abuhijle.googlepages.com/libusb-vvoutput

I am trying to achieve that max possible speed to write to the device. There
are no other processes I am running along with a.out. My goal is to reduce
this > 1 minute to less than one second!! It is a USB 2, I know I should not
expect the standard's  60 MBytes  transfer rate because of overhead and all
other issues so I am shooting for 40.


Any errors in my code, please..? Any tipe on making this faster....

here are some more info:

# libusb-config --version
0.1.12

# uname -a
Linux elive 2.6.15.7 #2 Tue Dec 5 06:48:06 CET 2006 i686 GNU/Linux

gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v
--enable-languages=c,c++,fortran,objc,obj-c++,treelang --prefix=/usr
--enable-shared --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix --enable-nls
--program-suffix=-4.1 --enable-__cxa_atexit --enable-clocale=gnu
--enable-libstdcxx-debug --enable-mpfr --with-tune=i686
--enable-checking=release i486-linux-gnu
Thread model: posix
gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)











-M

Re: [Libusb-devel] Slow USB bulk transfer

[Dan E. <Dan...@di...>]

You'll do much better if you write more than 512 bytes at a time, e.g.
64kB. At the moment you're waiting for each 512 bytes to be transmitted
before the next one is sent. This is going to take at least 125us per
call, and probably quite a bit longer (e.g. 500us).
=20
Dan.

________________________________

From: lib...@li...
[mailto:lib...@li...] On Behalf Of
mohammed abuhijle
Sent: 28 February 2007 14:19
To: lib...@li...
Subject: [Libusb-devel] Slow USB bulk transfer


Hello,

I have the following code that takes more than a min to finisg:

-------------------------------------------------------
#include <stdio.h>
#include <string.h>
#include <usb.h>=20

// Search token for the cypress chip.
#define CYP_VENDOR_ID   0x04b4
#define CYP_PRODUCT_ID 0x8613

struct usb_device* cypress_init();
void sighandler(int );

=20
// Global vars
struct usb_device* cypress_dev;=20
struct usb_dev_handle* cypress_dev_handle;=20
char* bytes;
struct usb_interface_descriptor interface ;

int verbose =3D 0;
int c,i,a,ret;

int main(int argc, char* argv[]){
// First, init the cypress chip..=20
cypress_dev =3D cypress_init();=20
// Second, open it ...
if (cypress_dev) cypress_dev_handle =3D usb_open(cypress_dev);
else {
    printf("Fatel Error: No Cypress CY7C68xx devices connected. Exiting
\n");=20
    return -9;
}

if (cypress_dev_handle){
// Now, detatch the usbtest kernel driver assigned to the cypress chip
upon=20
//hotpluging it into the system
usb_detach_kernel_driver_np(cypress_dev_handle ,0);=20

usb_set_configuration(cypress_dev_handle,
cypress_dev->config->bConfigurationValue);
interface =3D cypress_dev->config[0].interface[0].altsetting[1];

usb_claim_interface(cypress_dev_handle,interface.bInterfaceNumber );
usb_set_altinterface(cypress_dev_handle,interface.bAlternateSetting);

//assign the endpoint addresses....
int epo   =3D interface.endpoint[0].bEndpointAddress;
int epi   =3D interface.endpoint[1].bEndpointAddress;=20

usb_clear_halt(cypress_dev_handle, epi );

//printf ("ep_out is %0x while ep_in is %0x\n",epo,epi);
bytes =3D (char*) malloc(512);
memset(bytes,0,sizeof(bytes));

/*write 40 Mbytes worth of data to the bulk endpoint*/=20
/*i am calculating this by: 512 (packet size) * 2 =3D 1 KB, * 1000 =3D =
1MB,
* 40 =3D 40 MB*/
for (i =3D 0; i < 2 * 1000 * 40 ; i++){
ret =3D usb_bulk_write(cypress_dev_handle, epi,bytes, 512, 0 /*the =
timrout
is ignored for bulk transfer*/);=20
//printf ("i is:%d\t ret is %d\t bytes is %s\n",i, ret,bytes);
}

}
else {
    printf("Could not open the Cypress chip. Quiting \n");
    return -1;
    }

usb_release_interface(cypress_dev_handle,0);=20
usb_close(cypress_dev_handle);

return 0;=20
}   =20

struct usb_device* cypress_init(){
   =20
struct usb_bus *bus =3DNULL;=20
struct usb_device *dev =3DNULL;   =20

usb_init();=20
usb_find_busses();=20
usb_find_devices();
usb_set_debug(9);

for (bus =3D usb_busses; bus;  bus =3D bus->next)=20
   for (dev =3D bus->devices; dev;  dev =3D dev->next)=20
       if ((dev->descriptor.idVendor =3D=3D  CYP_VENDOR_ID)    &&=20
             (dev->descriptor.idProduct =3D=3D  CYP_PRODUCT_ID)) return =
dev;

  =20
return NULL;
}

void sighandler(int sig){
   =20
//usb_release_interface(cypress_dev_handle,0);
usb_close(cypress_dev_handle);   =20
   =20
}

-------------------------------------------------------

As root, I ran "time ./a.out" and got

# time ./a.out
real    1m9.793s
user    0m0.118s
sys     0m0.490s

# time ./a.out

real    1m15.078s
user    0m0.095s
sys     0m0.438s
root@elive[/home/me/cypress]#


The lsusb -vv output of the cypress FX2 device is at:=20

http://abuhijle.googlepages.com/libusb-vvoutput

I am trying to achieve that max possible speed to write to the device.
There are no other processes I am running along with a.out. My goal is
to reduce this > 1 minute to less than one second!! It is a USB 2, I
know I should not expect the standard's  60 MBytes  transfer rate
because of overhead and all other issues so I am shooting for 40.=20


Any errors in my code, please..? Any tipe on making this faster....

here are some more info:=20

# libusb-config --version
0.1.12

# uname -a
Linux elive 2.6.15.7 #2 Tue Dec 5 06:48:06 CET 2006 i686 GNU/Linux

gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v
--enable-languages=3Dc,c++,fortran,objc,obj-c++,treelang --prefix=3D/usr
--enable-shared --with-system-zlib --libexecdir=3D/usr/lib
--without-included-gettext --enable-threads=3Dposix --enable-nls
--program-suffix=3D- 4.1 --enable-__cxa_atexit --enable-clocale=3Dgnu
--enable-libstdcxx-debug --enable-mpfr --with-tune=3Di686
--enable-checking=3Drelease i486-linux-gnu
Thread model: posix
gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21 )











-M

Re: [Libusb-devel] Slow USB bulk transfer

[Tim R. <ti...@pr...>]

mohammed abuhijle wrote:
>
> I have the following code that takes more than a min to finisg:

That's right.


> /*write 40 Mbytes worth of data to the bulk endpoint*/
> /*i am calculating this by: 512 (packet size) * 2 = 1 KB, * 1000 =
> 1MB, * 40 = 40 MB*/
> for (i = 0; i < 2 * 1000 * 40 ; i++){
> ret = usb_bulk_write(cypress_dev_handle, epi,bytes, 512, 0 /*the
> timrout is ignored for bulk transfer*/);
> //printf ("i is:%d\t ret is %d\t bytes is %s\n",i, ret,bytes);
> }

Remember that USB is a "scheduled" bus.  The USB host controller has to
have a frame all planned out before the frame begins.  In your case, you
are sending exactly one packet down, which then gets scheduled into a
frame, and because usb_bulk_write is synchronous, you then wait until
the transfer is complete, thereby wasting the rest of the frame.  That
means you cannot do any better than one packet per frame, 512k bytes per
second, which would make 40MB take 80 seconds.

The USB subsystem will happily chop up a large transfer into
packet-sized pieces as needed, and when it does so, it will pack in as
many packets as it can into a single frame.  You can send megabytes to
usb_bulk_write, and get your 40MB transferred in a second or two.

-- 
Tim Roberts, ti...@pr...
Providenza & Boekelheide, Inc.

Re: [Libusb-devel] Slow USB bulk transfer

[mohammed a. <abu...@gm...>]

Alright then, thanks for the explanation. I was using 512 because I thought
I had to abide by the maxPacketsize of the endpoint I am using.

Now the second problem is that I get a zero as a return value from
usb_bulk_write. It is still the same code but with changing the size value
of usb_bulk_write to 1 MB instead of 512.

Looking at linux.c, this means that zero bytesdone have been returned and I
wrote nothing to the device..!!

What could be going wrong. As you mentioned, bulk is synchronous, so it had
to wait until something is written and then come back..I also have the
timeout value as zero...which according to my understanding is ignored
altogether if we doing bulk read/write...



On 2/28/07, Tim Roberts <ti...@pr...> wrote:
>
> mohammed abuhijle wrote:
> >
> > I have the following code that takes more than a min to finisg:
>
> That's right.
>
>
> > /*write 40 Mbytes worth of data to the bulk endpoint*/
> > /*i am calculating this by: 512 (packet size) * 2 = 1 KB, * 1000 =
> > 1MB, * 40 = 40 MB*/
> > for (i = 0; i < 2 * 1000 * 40 ; i++){
> > ret = usb_bulk_write(cypress_dev_handle, epi,bytes, 512, 0 /*the
> > timrout is ignored for bulk transfer*/);
> > //printf ("i is:%d\t ret is %d\t bytes is %s\n",i, ret,bytes);
> > }
>
> Remember that USB is a "scheduled" bus.  The USB host controller has to
> have a frame all planned out before the frame begins.  In your case, you
> are sending exactly one packet down, which then gets scheduled into a
> frame, and because usb_bulk_write is synchronous, you then wait until
> the transfer is complete, thereby wasting the rest of the frame.  That
> means you cannot do any better than one packet per frame, 512k bytes per
> second, which would make 40MB take 80 seconds.
>
> The USB subsystem will happily chop up a large transfer into
> packet-sized pieces as needed, and when it does so, it will pack in as
> many packets as it can into a single frame.  You can send megabytes to
> usb_bulk_write, and get your 40MB transferred in a second or two.
>
> --
> Tim Roberts, ti...@pr...
> Providenza & Boekelheide, Inc.
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Libusb-devel mailing list
> Lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libusb-devel
>

Re: [Libusb-devel] Slow USB bulk transfer

[Del M. <de...@al...>]

mohammed abuhijle wrote:
> Alright then, thanks for the explanation. I was using 512 because I 
> thought I had to abide by the maxPacketsize of the endpoint I am using.
>
> Now the second problem is that I get a zero as a return value from 
> usb_bulk_write. It is still the same code but with changing the size 
> value of usb_bulk_write to 1 MB instead of 512.

Well, in theory you can say, "here's 1MB; gopher it".  But in practice 
you probably don't want to do that.  First, it's possible Linux has a 
limit on the size of what it buffers.  More likely the device won't like 
something that big, since it too will have to deal with it.  When I 
write, I pick what I consider a "reasonable but still magic" number, 
like 32KB or 64KB.  This keeps the buffers manageable on my end and 
minimizes the likelihood of blowing away some (un)documented buffer 
limit on the device.  If you are using USB 2.0 and know the device well, 
you might want to pick a bigger chunk size, but 64KB (64*1024, not 
64*1000) is probably big enough for many apps to get good throughput and 
small enough to help you catch errors in a timely manner.

It's much like any other packet-driven system: the bigger the packet, 
the better the possible throughput, but the worse the error recovery - 
particularly if you can or have to retransmit the data.

There are other reasons you could be getting nowhere.  Is "epi" the 
input or output endpoint?  What do you see in /var/log/messages?    Do 
you have access to a USB analyzer?  It could be you are getting NAKed or 
STALLed. 

Finally, my read of linux.c's  usb_urb_transfer() suggests that you 
don't want a "0" timeout, since it will force a return before the urb is 
reaped.  I'd make the timeout at least several tens - if not a hundred 
or so - milliseconds.  That will give the kernel a chance to process the 
urb.  You may need to keep tinkering with the timeout and the buffer 
size to get the "best" performance.

-Del

> Looking at linux.c, this means that zero bytesdone have been returned 
> and I wrote nothing to the device..!!
>
> What could be going wrong. As you mentioned, bulk is synchronous, so 
> it had to wait until something is written and then come back..I also 
> have the timeout value as zero...which according to my understanding 
> is ignored altogether if we doing bulk read/write...
>
>
>
> On 2/28/07, *Tim Roberts* <ti...@pr... <mailto:ti...@pr...>> wrote:
>
>     mohammed abuhijle wrote:
>     >
>     > I have the following code that takes more than a min to finisg:
>
>     That's right.
>
>
>     > /*write 40 Mbytes worth of data to the bulk endpoint*/
>     > /*i am calculating this by: 512 (packet size) * 2 = 1 KB, * 1000 =
>     > 1MB, * 40 = 40 MB*/
>     > for (i = 0; i < 2 * 1000 * 40 ; i++){
>     > ret = usb_bulk_write(cypress_dev_handle, epi,bytes, 512, 0 /*the
>     > timrout is ignored for bulk transfer*/);
>     > //printf ("i is:%d\t ret is %d\t bytes is %s\n",i, ret,bytes);
>     > }
>
>     Remember that USB is a "scheduled" bus.  The USB host controller
>     has to
>     have a frame all planned out before the frame begins.  In your
>     case, you
>     are sending exactly one packet down, which then gets scheduled into a
>     frame, and because usb_bulk_write is synchronous, you then wait until
>     the transfer is complete, thereby wasting the rest of the frame.  That
>     means you cannot do any better than one packet per frame, 512k
>     bytes per
>     second, which would make 40MB take 80 seconds.
>
>     The USB subsystem will happily chop up a large transfer into
>     packet-sized pieces as needed, and when it does so, it will pack in as
>     many packets as it can into a single frame.  You can send
>     megabytes to
>     usb_bulk_write, and get your 40MB transferred in a second or two.
>
>     --
>     Tim Roberts, ti...@pr... <mailto:ti...@pr...>
>     Providenza & Boekelheide, Inc.
>
>
>     -------------------------------------------------------------------------
>
>     Take Surveys. Earn Cash. Influence the Future of IT
>     Join SourceForge.net's Techsay panel and you'll get the chance to
>     share your
>     opinions on IT & business topics through brief surveys-and earn cash
>     http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>     <http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV>
>     _______________________________________________
>     Libusb-devel mailing list
>     Lib...@li...
>     <mailto:Lib...@li...>
>     https://lists.sourceforge.net/lists/listinfo/libusb-devel
>     <https://lists.sourceforge.net/lists/listinfo/libusb-devel>
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> ------------------------------------------------------------------------
>
> _______________________________________________
> Libusb-devel mailing list
> Lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libusb-devel
>   

Re: [Libusb-devel] Slow USB bulk transfer

[Tim R. <ti...@pr...>]

Del Merritt wrote:
> mohammed abuhijle wrote:
>> Alright then, thanks for the explanation. I was using 512 because I
>> thought I had to abide by the maxPacketsize of the endpoint I am using.
>>
>> Now the second problem is that I get a zero as a return value from
>> usb_bulk_write. It is still the same code but with changing the size
>> value of usb_bulk_write to 1 MB instead of 512.
>
> Well, in theory you can say, "here's 1MB; gopher it".  But in practice
> you probably don't want to do that.  First, it's possible Linux has a
> limit on the size of what it buffers.  More likely the device won't
> like something that big, since it too will have to deal with it.

No, it won't.  The device will never be asked to transfer more than its
maximum packet size, which for a bulk pipe is no more than 512 bytes.
The chopping and reconstructing is all done in the host.  Even if the
device only transmits 64 bytes at a time, I can still ask for a
megabyte.  It might take a while for it to accumulate, but the USB
driver will handle it.

> It's much like any other packet-driven system: the bigger the packet,
> the better the possible throughput, but the worse the error recovery -
> particularly if you can or have to retransmit the data.

For a bulk pipe, retransmission is automatic, on a packet by packet
basis.  The buffer size is irrelevant to that process.
-- 
Tim Roberts, ti...@pr...
Providenza & Boekelheide, Inc.