From: Dante S. <da...@ma...> - 2008-05-14 13:20:27
|
Hi, I have a Verdex+console-vx+netwifimicro stack that reads data from a SPI device and forwards it to my computer via wireless sockets (I create a pipe with socat and then write to it with fprintf). Right now I am sampling at around 250Hz, but I want to be able to read at least as fast as 1MHz. I'm confident the routine can definitely do it, but the data transmission is what's slowing down my program. Do you think it's worth it to try making the data transmission into a separate thread? I'm not sure if that will actually fix the problem. Also, I'm using gettimeofday() to take a measurement of the sampling time, and every data point returns a different value, approx. within 0.500 ms. Is there a way I can shut off processes running in the background? I'm using a buildroot image. Thanks, Dante -- View this message in context: http://www.nabble.com/Threading-for-real-time-data-transmission-tp17230915p17230915.html Sent from the Gumstix mailing list archive at Nabble.com. |
From: Ryan R. <rjr...@uc...> - 2008-05-14 16:55:40
|
Dante Sanchez wrote: > Hi, > > I have a Verdex+console-vx+netwifimicro stack that reads data from a SPI > device and forwards it to my computer via wireless sockets (I create a pipe > with socat and then write to it with fprintf). > > Right now I am sampling at around 250Hz, but I want to be able to read at > least as fast as 1MHz. I'm confident the routine can definitely do it, but > the data transmission is what's slowing down my program. > > Do you think it's worth it to try making the data transmission into a > separate thread? I'm not sure if that will actually fix the problem. > > Also, I'm using gettimeofday() to take a measurement of the sampling time, > and every data point returns a different value, approx. within 0.500 ms. Is > there a way I can shut off processes running in the background? I'm using a > buildroot image. > > Thanks, > > Dante > For the sampling rate, you should probably use a timer. Linux provides several that work very nicely. You've got several options for data transmission. I think your backup will be in the wifi link. The 50Mbs is a burst rate, kind of like a 3Gbs SATA drive. Sustained rates will be far slower due to packet overhead, handshaking, etc. It also depends on whether you have to have every packet. If dropped packets aren't the end of the world, then you should use a udp socket, or possibly even a raw socket. The other question is how many bits/sample. Your theoretical maximum is 50 bits/sample at 1 MHz. 50 bits is 6 bytes and change, and figure you'll be lucky to get half of that. Also, dump fprintf. It's slowing you down. Use write() to send binary structs or ints/shorts/chars, whichever applies to your sensor. Threading only makes sense when you have one task that's IO bound and another that's CPU bound. Unless you're doing some filtering on the data, you're not CPU bound at all. You have a pure IO problem here, and that's where you should focus. If all the gumstix is doing is acting as a relay, the fastest solution is to modify the SPI driver to pass the data directly to the wifi buffer. Short of that, you should be able to pass the address of the spi buffer to write with an appropriate size and skip all the copying. Ryan |
From: Dante S. <da...@ma...> - 2008-05-14 18:30:12
|
Hi Ryan, Ryan Rapetti wrote: > > For the sampling rate, you should probably use a timer. Linux provides > several that work very nicely. > Great, I'll look into that. I was using Dave Hylands' usleep-drv driver as a 'periodic timer', but if there's something else that is more reliable and also goes down to useconds, I'll take it. Ryan Rapetti wrote: > > If dropped packets aren't the end of the world, then you should use a udp > socket, or possibly even a raw > socket. > Yeah, I am using UDP sockets because I read that TCP sockets resend failed packets, and that didn't sound very good to me. Ryan Rapetti wrote: > > Your theoretical maximum is 50 bits/sample at 1 MHz. 50 bits is 6 bytes > and change, and figure you'll be lucky to get half of that. Also, dump > fprintf. It's slowing you down. Use write() to send binary structs or > ints/shorts/chars, whichever applies to your sensor. > I have almost 40 bytes per sample, and that's what I was sending to fprintf (no fprintf, gotcha!). I can definitely sample the SPI port at 1MHz and pipe the data to wifi at ~50Hz...but then again, how do I do that so as not to get in the way of the data sampling? Ryan Rapetti wrote: > > Unless you're doing some filtering on the data, you're not CPU bound at > all. You have a pure IO problem here, and that's where you should focus. > If all the gumstix is doing is acting as a relay, the fastest solution is > to modify the SPI driver to pass the data directly to the wifi buffer. > Yeah, that sounds easy enough...but I will have to do computations on the data coming in, so I guess I am CPU bound. And, I haven't had the opportunity to develop a SPI driver, so I am accessing the NSSP port through user space. I guess that's not so great either... Thanks for all the pointers, I appreciate it Dante -- View this message in context: http://www.nabble.com/Threading-for-real-time-data-transmission-tp17230915p17237916.html Sent from the Gumstix mailing list archive at Nabble.com. |
From: Dave H. <dhy...@gm...> - 2008-05-14 20:44:42
|
Hi Dante, > > For the sampling rate, you should probably use a timer. Linux provides > > several that work very nicely. > > > Great, I'll look into that. I was using Dave Hylands' usleep-drv driver as a > 'periodic timer', but if there's something else that is more reliable and > also goes down to useconds, I'll take it. Heh - I wouldn't recommend it for that purpose. It was really created to allow code which does bit-banged I/O and needs to wait a small amount of time between bit transitions. For some versions of the kernel, the smallest wait time is on the order of 10 msec, which is way too long for many bit-banged applications. What would be better is to create a driver which uses one of the gumstix timers. As long as you're talking about the SPI clock being at the MHz level, and you're using the SPI hardware, then you're talking about something on the order of 30k samples/sec I think it's probably doable, although it would all need to be done from kernel mode. -- Dave Hylands Vancouver, BC, Canada http://www.DaveHylands.com/ |
From: Ryan R. <rjr...@uc...> - 2008-05-14 21:21:18
|
Dante Sanchez wrote: > Hi Ryan, > > > Ryan Rapetti wrote: > >> For the sampling rate, you should probably use a timer. Linux provides >> several that work very nicely. >> >> > Great, I'll look into that. I was using Dave Hylands' usleep-drv driver as a > 'periodic timer', but if there's something else that is more reliable and > also goes down to useconds, I'll take it. > Microseconds are tricky, you may need to play with the kernel setup to get that kind of resolution. > > Ryan Rapetti wrote: > >> If dropped packets aren't the end of the world, then you should use a udp >> socket, or possibly even a raw >> socket. >> >> > Yeah, I am using UDP sockets because I read that TCP sockets resend failed > packets, and that didn't sound very good to me. > > > Ryan Rapetti wrote: > >> Your theoretical maximum is 50 bits/sample at 1 MHz. 50 bits is 6 bytes >> and change, and figure you'll be lucky to get half of that. Also, dump >> fprintf. It's slowing you down. Use write() to send binary structs or >> ints/shorts/chars, whichever applies to your sensor. >> >> > I have almost 40 bytes per sample, and that's what I was sending to fprintf > (no fprintf, gotcha!). I can definitely sample the SPI port at 1MHz and pipe > the data to wifi at ~50Hz...but then again, how do I do that so as not to > get in the way of the data sampling? > The best way to do this is to have the sensor set the sample frequency. The gumstix kernel isn't true realtime, and getting good timing at 1 MHz is going to be hard. But if the sensor sends a packet at 1MHz, that sets the timing and should be extremely accurate. Then you use the packet arrival as the signal to start everything. Grab the packet, process, send. The other question is whether or not exact timing is required post wifi, because you can't have it. The wifi chip will buffer and send the data on its own schedule. Also, just how much processing do you have to do? At about 1 operation/clock, you have a budget of 600 (XL6P) operations per sample, minus OS overhead, application overhead and other application overhead. If you're doing any floating point ops, 400-500 operations will run out before you've done a whole lot. Basic low order filtering will probably work, but I wouldn't count on much more than that. As far as not stomping on the sampling, you're going to have to be very creative to get the gumstix to set the timing correctly. It's just not realtime enough for that. If you can get the sensor to set the timing, it's mainly a matter of getting the SPI driver to buffer the incoming data and send a signal when a packet arrives. Then you have a signal handler grab the packet, process it and send it. At that point your program is basically in the handler. For multithreading, it all depends on what else the program is doing. If it has other duties, multithreading makes sense, but if all it does is grab, process and send, you shouldn't need a second thread. > > Ryan Rapetti wrote: > >> Unless you're doing some filtering on the data, you're not CPU bound at >> all. You have a pure IO problem here, and that's where you should focus. >> If all the gumstix is doing is acting as a relay, the fastest solution is >> to modify the SPI driver to pass the data directly to the wifi buffer. >> >> > Yeah, that sounds easy enough...but I will have to do computations on the > data coming in, so I guess I am CPU bound. And, I haven't had the > opportunity to develop a SPI driver, so I am accessing the NSSP port through > user space. I guess that's not so great either... > > Thanks for all the pointers, I appreciate it > > Dante > Ryan |