From: sebastien m. <seb...@ya...> - 2012-08-17 09:34:59
|
Hi everyone, I'm working on a video processing system for an UAV. The size and weight of the Overo COM is perfect for us, but I saw in the mailing list archives that the FPS with the Caspa cameras are not what the datasheet says (60 FPS), and that the processor load is at 100%. I'd like to know if I could use an Overo IronSTORM COM + Caspa VL camera board and get at least : 30 frames per second at 752x480 that can be used by my C/C++ image processing program with OpenCV (just an example). I heard that the problem could be the driver used with the Caspa board (v4l2 isn't it ?), I'd like to know what is the problem before buying. Thank you in advance for your help ! |
From: Florian V. <flo...@ep...> - 2012-08-29 08:13:12
|
Hello, Le 17/08/2012 11:34, sebastien manin a écrit : > Hi everyone, > > I'm working on a video processing system for an UAV. > The size and weight of the Overo COM is perfect for us, but I saw in > the mailing list archives that the FPS with the Caspa cameras are not > what the datasheet says (60 FPS), and that the processor load is at 100%. > > I'd like to know if I could use an Overo IronSTORM COM + Caspa VL > camera board and get at least : 30 frames per second at 752x480 that > can be used by my C/C++ image processing program with OpenCV (just an > example). > I heard that the problem could be the driver used with the Caspa board > (v4l2 isn't it ?), I'd like to know what is the problem before buying. I streamed images at 752x480 / 30 fps using the Caspa, performing the mpeg4 compression using the DSP. The CPU usage was kept low in this case, however I cannot tell the figure by heart. So the problem is definitely not on the driver side. The driver is using the media controller framework, thus your application will see it as v4l2 api, but you will need additional tools to configure the acquisition channel. See: http://omappedia.org/wiki/Camera-ISP_Driver. Doing image processing in C++ with OpenCV can be pretty heavy, so depending on the algorithm you are running, you will be limited by the CPU, hence limiting your framerate. There is some work to accelerate OpenCV using the DSP, but I cannot tell the current status. I saw a white paper from TI on this. > > Thank you in advance for your help ! > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > > > _______________________________________________ > gumstix-users mailing list > gum...@li... > https://lists.sourceforge.net/lists/listinfo/gumstix-users Cheers, Florian -- Florian Vaussard EPFL - STI - IMT - LSRO1 MEB330 - Station 9 1015 Lausanne / Switzerland tel: +41 21 693 78 39 fax: +41 21 693 78 07 http://lsro.epfl.ch |
From: Simon K. <sim...@pg...> - 2012-08-29 11:16:20
|
Hi, On 29/08/2012, at 8:12 PM, Florian Vaussard wrote: > Doing image processing in C++ with OpenCV can be pretty heavy, so > depending on the algorithm you are running, you will be limited by the > CPU, hence limiting your framerate. There is some work to accelerate > OpenCV using the DSP, but I cannot tell the current status. I saw a > white paper from TI on this. Just to chime in here, I too saw the whitepaper from TI about OpenCV [1]. Unfortunately I then found a post in their forums stating that they have de-emphasised support for OpenCV and aren't distributing their DSP port [2] :( If you aren't tied to OpenCV then you could look into IMGLIB, TI's DSP image processing library. It doesn't have nearly the range of functions of OpenCV but it's a start. It's relatively easy to use via C6EZRun. Simon. [1]: http://focus.ti.com/lit/wp/spry175/spry175.pdf [2]: http://e2e.ti.com/support/embedded/bios/f/355/t/125780.aspx |
From: Steven <gum...@si...> - 2012-08-30 07:50:17
|
>From my experience with the capsa camera about an year ago, the driver overhead was higher than I expected (this might have been fixed though). *If* you're doing video compression @ 60fps then you might get close to 100% (even with the DSP). I also did work with OpenCV using a different USB v4l2 camera and it worked fine, abet without NEON or DSP acceleration. You can definitely capture at 30fps, but no one can tell you if you can process at that rate since OpenCV contains a wide variety of algorithms with wildly different time complexities. One more thing, OpenCV's capture code isn't super well written, so you might have to write your own. Luckily there is a v4l2 example that's pretty easy to follow. http://libav.org/doxygen/master/v4l2_8c_source.html On Fri, Aug 17, 2012 at 5:34 AM, sebastien manin <seb...@ya...> wrote: > Hi everyone, > > I'm working on a video processing system for an UAV. > The size and weight of the Overo COM is perfect for us, but I saw in the > mailing list archives that the FPS with the Caspa cameras are not what the > datasheet says (60 FPS), and that the processor load is at 100%. > > I'd like to know if I could use an Overo IronSTORM COM + Caspa VL camera > board and get at least : 30 frames per second at 752x480 that can be used by > my C/C++ image processing program with OpenCV (just an example). > I heard that the problem could be the driver used with the Caspa board (v4l2 > isn't it ?), I'd like to know what is the problem before buying. > > Thank you in advance for your help ! > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > gumstix-users mailing list > gum...@li... > https://lists.sourceforge.net/lists/listinfo/gumstix-users > |
From: Florian V. <flo...@ep...> - 2012-08-30 09:06:46
|
Hi Brahim (cc'ing to the list), Le 29/08/2012 14:04, Brahim HAMADICHAREF a écrit : > Hi Florian > > Is there a detailed tutorial to acheive this "streamed images at 752x480 > at 30 fps using the Caspa, performing thempeg4 compression using the DSP ? Unfortunately not for now, I had to do this in a hurry for my work. I will outline the procedure, and maybe post a more detailed tutorial in the near future if time permits. > > When you say stream, you mean stream over ethernet ? Yes, over Ethernet, although Wifi was also working. I am using the RTP facility provided by gstreamer. > > I can capture CAPSA at 752x480 at high rate but to display at 30 fps for > example is not possible for me yet. My threaded Qt4 program eats all > CPU usage ... If you want to process the image before streaming it, you will have to adapt my procedure, as I was only broadcasting the compressed multimedia flux. But gstreamer is quite flexible for that. > > Any help will be appreciated Here for the brief outline (details are hanging around in several mailing lists and on the internet, but a lot of stuff is quite out-dated). Sorry, not an easy solution, maybe someone will have something better. But performances were great. - Compile a recent kernel with the tidsp bridge support. I was using a 3.2 at that time, but no reason you should not use a 3.5. Here are the important config flags to look for: CONFIG_OMAP_MBOX_FWK CONFIG_TIDSPBRIDGE TIDSPBRIDGE was in the staging area at that time. - I also experienced problems using gstreamer, as it was expecting some unimplemented APIs. I resolved this using a *very* ugly, yet working, patch [1]. I hopped to never publish such a bad piece of code, but if it could help... - Fetch the DSP binary blob (baseimage.dof mainly) from the TI website, as well as the MPEG4 compression cocdecs (c64xplus_mpeg4enc_02_02_04_production.bin on the website if I remember). But the .dof and all the .dll64P files into /lib/dsp in your Overo rootfs. - When you boot your Overo, you should see a message stating the successful initialization of the DSP in the kernel traces, and a /dev/dsp node. If not, check if the module is correctly loaded. - Get the gst-dsp project [2], cross-compile compile and install it on your Overo. I did native compilation on the Overo. - Set the pipes according to the doc in [3]. Here is mine: media-ctl -r -l '"mt9v032 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]' media-ctl -f '"mt9v032 3-005c":0[SGRBG10 752x480], "OMAP3 ISP CCDC":2[SGRBG10 752x480], "OMAP3 ISP preview":1[UYVY 752x480], "OMAP3 ISP resizer":1[UYVY 752x480]' - Set a proper gstreamer pipeline, to capture the image, encode it using the DSP, put the RTP header and stream it by UDP: HOST='192.168.1.80' # adapt this to match your remote computer PORT=5000 gst-launch v4l2src device=/dev/video6 ! video/x-raw-yuv,width=752,height=480,format='(fourcc)'UYVY ! dspmp4venc ! rtpmp4vpay ! udpsink host=$HOST port=$PORT -v - On your main computer, run something like this, you should see the stream: PORT=5000 gst-launch -v udpsrc port=$PORT caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)MP4V-ES, profile-level-id=(string)5, config=(string) 000001b005000001b50900000100000001200086e0002c352003a00065304ba98517843c1463, payload=(int)96, ssrc=(guint)285587044, clock-base=(guint)3111327438, seqnum-base=(guint)60347" ! rtpmp4vdepay ! ffdec_mpeg4 ! xvimagesink Cheers, Florian [1] From 016527d3f47e8ca96edee80b2bb00497d5535379 Mon Sep 17 00:00:00 2001 From: Florian Vaussard <flo...@ep...> Date: Wed, 14 Mar 2012 16:08:00 +0100 Subject: [PATCH] Adding missing ioctl's --- drivers/media/video/omap3isp/ispvideo.c | 46 +++++++++++++++++++++++++++++++ 1 files changed, 46 insertions(+), 0 deletions(-) diff --git a/drivers/media/video/omap3isp/ispvideo.c b/drivers/media/video/omap3isp/ispvideo.c index 1452b3a..a15e396 100644 --- a/drivers/media/video/omap3isp/ispvideo.c +++ b/drivers/media/video/omap3isp/ispvideo.c @@ -733,6 +733,36 @@ isp_video_get_format(struct file *file, void *fh, struct v4l2_format *format) } static int +isp_video_enum_format(struct file *file, void *fh, struct v4l2_fmtdesc *fmtdesc) +{ + struct isp_video_fh *vfh = to_isp_video_fh(fh); + struct isp_video *video = video_drvdata(file); + + if (fmtdesc->index) + { +// printk(KERN_ERR "%s: invalid index (%d)\n",__func__, fmtdesc->index); + return -EINVAL; + } + + if (fmtdesc->type != video->type) + { +// printk(KERN_ERR "%s: invalid type (%d). Current type: %d\n",__func__, fmtdesc->type, video->type); + return -EINVAL; + } + + // ***Ugly hack*** + fmtdesc->flags = 0; + fmtdesc->description[0] = 'U'; + fmtdesc->description[1] = 'Y'; + fmtdesc->description[2] = 'V'; + fmtdesc->description[3] = 'Y'; + fmtdesc->description[4] = '0'; + fmtdesc->pixelformat = V4L2_PIX_FMT_UYVY; + + return 0; +} + +static int isp_video_set_format(struct file *file, void *fh, struct v4l2_format *format) { struct isp_video_fh *vfh = to_isp_video_fh(fh); @@ -765,6 +795,8 @@ isp_video_try_format(struct file *file, void *fh, struct v4l2_format *format) u32 pad; int ret; +// printk(KERN_ERR "%s: Trying format %d\n",__func__, format->fmt.pix.pixelformat); + if (format->type != video->type) return -EINVAL; @@ -1158,8 +1190,21 @@ isp_video_s_input(struct file *file, void *fh, unsigned int input) return input == 0 ? 0 : -EINVAL; } +static int +isp_video_queryctrl(struct file *file, void *fh, struct v4l2_queryctrl *queryctrl) +{ +// printk(KERN_ERR "%s: querry id (%d)\n",__func__, queryctrl->id); +// printk(KERN_ERR "%s: querry type (%d)\n",__func__, queryctrl->type); + + if (queryctrl->id < V4L2_CID_BASE) + return -EDOM; + + return -EINVAL; +} + static const struct v4l2_ioctl_ops isp_video_ioctl_ops = { .vidioc_querycap = isp_video_querycap, + .vidioc_enum_fmt_vid_cap = isp_video_enum_format, .vidioc_g_fmt_vid_cap = isp_video_get_format, .vidioc_s_fmt_vid_cap = isp_video_set_format, .vidioc_try_fmt_vid_cap = isp_video_try_format, @@ -1180,6 +1225,7 @@ static const struct v4l2_ioctl_ops isp_video_ioctl_ops = { .vidioc_enum_input = isp_video_enum_input, .vidioc_g_input = isp_video_g_input, .vidioc_s_input = isp_video_s_input, + .vidioc_queryctrl = isp_video_queryctrl, }; /* ----------------------------------------------------------------------------- -- 1.7.5.4 [2] https://github.com/felipec/gst-dsp [3] http://omappedia.org/wiki/Camera-ISP_Driver |
From: bhamadicharef <bha...@ho...> - 2012-09-11 15:46:02
|
Thank you, i will try this. I look forward to check if this works on our system. Best regards Brahim -- View this message in context: http://gumstix.8.n6.nabble.com/Overo-COM-Caspa-Camera-for-image-processing-tp4965139p4965360.html Sent from the Gumstix mailing list archive at Nabble.com. |
From: Gutembert <nga...@gm...> - 2012-09-16 21:02:36
|
Hi Florian, I successfully " Streamed video (USB v4l2 WebCam) with Gumstix, using GStreamer and the DSP " but now I would like to broadcast it live using either RTP or RTSP protocol via VLC (url: rtp://Host_IP); or the Web/Mobile phone. Any help or suggestion is welcome Thank you in advance, Gutembert. -- View this message in context: http://gumstix.8.n6.nabble.com/Overo-COM-Caspa-Camera-for-image-processing-tp4965139p4965411.html Sent from the Gumstix mailing list archive at Nabble.com. |
From: bhamadicharef <bha...@ho...> - 2012-09-17 16:21:27
|
Gutembert, could you share the rootfs of this "Streamed video (USB v4l2 WebCam) with Gumstix, using GStreamer and the DSP" system so we can also try ? which kernel is running on 3.0 or 3.2 ? Brahim -- View this message in context: http://gumstix.8.n6.nabble.com/Overo-COM-Caspa-Camera-for-image-processing-tp4965139p4965423.html Sent from the Gumstix mailing list archive at Nabble.com. |
From: Gutembert <nga...@gm...> - 2012-09-19 19:44:46
|
Hi Brahim, Could you please explain what you meant by ' rootfs of this "Streamed video (USB v4l2 WebCam) with Gumstix, using GStreamer and the DSP" '? Regarding the kernel, I started with 3.2 but it didn't work and by further research, 2.6.34 was found to be the most suitable kernel to get the DSP and the Camera working. Also, Ubuntu 10.04 LTS seems to be better for OpenEmbedded compare to others (in my OPINION). Regards, Gutembert. -- View this message in context: http://gumstix.8.n6.nabble.com/Overo-COM-Caspa-Camera-for-image-processing-tp4965139p4965458.html Sent from the Gumstix mailing list archive at Nabble.com. |
From: zoso1234 <hav...@um...> - 2012-09-20 05:09:22
|
I Just bought a Caspa VL and am looking to get into doing image processing as well. I have been using some TI DSP modules for image processing with some success. My main question is how hard would it be to get the Caspa to work with 2.6.37 kernel or 39? The linux-omap_2.6.37 has alot of v4l2 patches and the OMAP3-ISP-driver patch but I don't see anything for the mt9v032. While the 2.6.39 has file://camera/0002-v4l-Add-mt9v032-sensor-driver.patch in the recipe..but I don't see much for v4l2 or ISP...I'm assuming it'd be easier to use 2.6.39 but i'd rather use 37. Has anyone had much success with 2.6.37 and Caspa? I did see some old threads regarding 2.6.39 and caspa soo I have something to work off of. any tips for 2.6.37 would be great -- View this message in context: http://gumstix.8.n6.nabble.com/Overo-COM-Caspa-Camera-for-image-processing-tp4965139p4965461.html Sent from the Gumstix mailing list archive at Nabble.com. |