I have gotten quickcam express drivers up and running under Linux. The framerate is REALLY chunky, I am looking into some new drivers as we speak. Anyway thought you would all like to know.... if you have any questions just post.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Brandon, good work on getting the QuickCam's going!
The framerate may be limited by USB. Using full-speed bulk transactions of 64-byte data payloads USB can move a maximum of about 1.2MByte/sec. At about 353x264, each image contains about 93K pixels. At 3 bytes per pixel that is about 280KB for a max of about 4 frames per second. I don't know if they do compression in the camera. If they do, then the bus bandwidth wouldn't be that limiting. If they don't do compression in the camera then you're not going to get a normal framerate unless you go to a very small image.
The QuickCams are about the cheapest cameras out there. The Ibot cameras look a bit better. At least they have Firewire, so frame rate shouldn't be an issue.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Actually I think you'll find that you don't need resolutions as high as 350x260. At such high resolutions the vision algorithms such as region growing and optical flow calculations become so slow that they cannot be done in realtime, even on the fastest processor.
For most of the stuff which I've done on Rodney I've used 160x120 or lower resolutions, which gives a reasonable frame rate and time for visual processing. 160x120 seems adequate for the purposes of image segmentation to recover shape information, and you can get away with much lower resolutions for things like motion or saturated colour detection.
Rodney currently grabs the image from the camera at 160x120, passes it through a gaussian filter and then does segmentation on it. It also subsamples the raw image at 80x60 and uses this for motion detection.
I recently got hold of a copy of RedHat 7.3 in the hope that the quickcams might work with this version of linux, but at the moment I'm struggling to get it to dual boot with windows XP. LILO refuses to boot the XP hard disk (hdd1).
If I remove all the linux partitions XP boots as normal.
- Bob
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
When dealing with robot's and vision, I love/hate to draw parallels to humans but must do so.
How often MUST Bob LOOK ahead? When humans drive cars (a scary thought) they only scan the road for moments (if that long), taking a fix and moving on. Point to make is that when detailed LOOKING is needed, do so. For superficial navigation, a low res, low frame count may be sufficient.
LEGO movie-maker (my-not-so young son) does a very nice job of building I-frames from a couple of pictures/frames.
Just a thought. (hope they are still welcome)
Karl
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I agree that low-res images will be easier/faster for our algorithms to work with. However, from what I saw the color accuracy was HORRIBLE on these QuickCams, which could make it difficult to base decisions on colored objects.
By the way, Bob, I don't think an alreay-installed version of XP will let you later install Linux. I think you have to start fresh with Linux, then install XP afterwards.
--Ryan
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I got the low-res drivers working when I got home from play practice. The rate is about 20 or so but from what I hear on the qce-ga forumes the rate is chunky on 366-400mhz machines because the writer of the driver did something funky when displaying the information or some such... anyways I am sure that will improve.
The color issue is not really a problem, I can change the gain values on the colors manually but it will take some tuning... one thing I am REALLY concerned about is glare, the cameras have a tendency to create blue "foggy" patches of glare under certain lighting conditions which could be misinterpreted as objects...
Anyways I have taken too much time up so far.
Oh and Bob you will need to partition before installing XP because XP generally doesnt like changing it partition size... unless you use one of those fancy 40 PartionMagic programs.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I would say that most of the time you want to be working on a low resulution and *high* frame rate. This will allow the robot to navigate around and avoid things with the minimum latency. On an 80x60 image you could do optical flow calculation to tell whether something is approaching the robot and from what direction. You could also try to find correlations between the robots own movement and the resulting optical flow pattern, in order to cancel out "ego motion".
For detailed inspection/recognition where time is not the critical factor you could have a high resolution and low frame rate.
Nature has actually already solved this problem by using a non-uniform distribution of light sensors across the retina. Within the central fovea region you get a high resolution image, with the periphery only being covered at a low resolution. This makes a good compromise between accuracy and the amount of visual processing required. At MIT on the Cog/Kismet robots they got around the problem by fudging it - using two sets of cameras for each eye, one with a wide and the other with a narrow field of view.
As for colour on the quickcam I would agree that it's not brilliant. Algorithms which look for saturated colour tend only to work under good lighting conditions (ideally natural lighting). What you often get is a noisy, low contrast image and for this reason conventional edge detection usually isn't much use. For detecting regions I've used a texture-based region growing method, which is far more successful on these types of images than the conventional "find the edges and then fill in the areas" method.
Particularly with face detection I found the sort of colour filters which are commonly used for that purpose to be almost useless except under the very best lighting conditions. You really need to combine colour information with other clues such as motion and regions in order to get a reliable result over a variety of conditions.
- Bob
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have gotten quickcam express drivers up and running under Linux. The framerate is REALLY chunky, I am looking into some new drivers as we speak. Anyway thought you would all like to know.... if you have any questions just post.
Brandon, good work on getting the QuickCam's going!
The framerate may be limited by USB. Using full-speed bulk transactions of 64-byte data payloads USB can move a maximum of about 1.2MByte/sec. At about 353x264, each image contains about 93K pixels. At 3 bytes per pixel that is about 280KB for a max of about 4 frames per second. I don't know if they do compression in the camera. If they do, then the bus bandwidth wouldn't be that limiting. If they don't do compression in the camera then you're not going to get a normal framerate unless you go to a very small image.
The QuickCams are about the cheapest cameras out there. The Ibot cameras look a bit better. At least they have Firewire, so frame rate shouldn't be an issue.
Actually I think you'll find that you don't need resolutions as high as 350x260. At such high resolutions the vision algorithms such as region growing and optical flow calculations become so slow that they cannot be done in realtime, even on the fastest processor.
For most of the stuff which I've done on Rodney I've used 160x120 or lower resolutions, which gives a reasonable frame rate and time for visual processing. 160x120 seems adequate for the purposes of image segmentation to recover shape information, and you can get away with much lower resolutions for things like motion or saturated colour detection.
Rodney currently grabs the image from the camera at 160x120, passes it through a gaussian filter and then does segmentation on it. It also subsamples the raw image at 80x60 and uses this for motion detection.
I recently got hold of a copy of RedHat 7.3 in the hope that the quickcams might work with this version of linux, but at the moment I'm struggling to get it to dual boot with windows XP. LILO refuses to boot the XP hard disk (hdd1).
If I remove all the linux partitions XP boots as normal.
- Bob
When dealing with robot's and vision, I love/hate to draw parallels to humans but must do so.
How often MUST Bob LOOK ahead? When humans drive cars (a scary thought) they only scan the road for moments (if that long), taking a fix and moving on. Point to make is that when detailed LOOKING is needed, do so. For superficial navigation, a low res, low frame count may be sufficient.
LEGO movie-maker (my-not-so young son) does a very nice job of building I-frames from a couple of pictures/frames.
Just a thought. (hope they are still welcome)
Karl
I agree that low-res images will be easier/faster for our algorithms to work with. However, from what I saw the color accuracy was HORRIBLE on these QuickCams, which could make it difficult to base decisions on colored objects.
By the way, Bob, I don't think an alreay-installed version of XP will let you later install Linux. I think you have to start fresh with Linux, then install XP afterwards.
--Ryan
I got the low-res drivers working when I got home from play practice. The rate is about 20 or so but from what I hear on the qce-ga forumes the rate is chunky on 366-400mhz machines because the writer of the driver did something funky when displaying the information or some such... anyways I am sure that will improve.
The color issue is not really a problem, I can change the gain values on the colors manually but it will take some tuning... one thing I am REALLY concerned about is glare, the cameras have a tendency to create blue "foggy" patches of glare under certain lighting conditions which could be misinterpreted as objects...
Anyways I have taken too much time up so far.
Oh and Bob you will need to partition before installing XP because XP generally doesnt like changing it partition size... unless you use one of those fancy 40 PartionMagic programs.
I would say that most of the time you want to be working on a low resulution and *high* frame rate. This will allow the robot to navigate around and avoid things with the minimum latency. On an 80x60 image you could do optical flow calculation to tell whether something is approaching the robot and from what direction. You could also try to find correlations between the robots own movement and the resulting optical flow pattern, in order to cancel out "ego motion".
For detailed inspection/recognition where time is not the critical factor you could have a high resolution and low frame rate.
Nature has actually already solved this problem by using a non-uniform distribution of light sensors across the retina. Within the central fovea region you get a high resolution image, with the periphery only being covered at a low resolution. This makes a good compromise between accuracy and the amount of visual processing required. At MIT on the Cog/Kismet robots they got around the problem by fudging it - using two sets of cameras for each eye, one with a wide and the other with a narrow field of view.
As for colour on the quickcam I would agree that it's not brilliant. Algorithms which look for saturated colour tend only to work under good lighting conditions (ideally natural lighting). What you often get is a noisy, low contrast image and for this reason conventional edge detection usually isn't much use. For detecting regions I've used a texture-based region growing method, which is far more successful on these types of images than the conventional "find the edges and then fill in the areas" method.
Particularly with face detection I found the sort of colour filters which are commonly used for that purpose to be almost useless except under the very best lighting conditions. You really need to combine colour information with other clues such as motion and regions in order to get a reliable result over a variety of conditions.
- Bob