Firstly thank you for all the effort and work put into this software - its much appreciated.
I'm testing mjpg-streamer out for network enabling various webcams I have around the house for CCTV purposes and recording the streams via Zoneminder (possibly by Motion if I can't get Zoneminder motion detection working to a decent standard).
My webcams are Logitch C510 - fantastic little cameras and support MJPEG compression.
I can capture and stream in both yuv and mjpg formats - obviously the mjpeg if preferred as it has virtually no CPU overhead.
The problem I have is that when I stream with mjpeg enabled on the camera, my syslog fills up with the following :
Jul 17 15:35:14 er1402 zmc_m6[4552]: WAR [Corrupt JPEG data: 1 extraneous bytes before marker 0xd6]
Jul 17 15:35:15 er1402 zmc_m6[4552]: WAR [Corrupt JPEG data: 3 extraneous bytes before marker 0xd1]
Jul 17 15:35:16 er1402 zmc_m6[4552]: WAR [Corrupt JPEG data: 1 extraneous bytes before marker 0xd1]
Jul 17 15:35:17 er1402 zmc_m6[4552]: WAR [Corrupt JPEG data: 2 extraneous bytes before marker 0xd0]
Jul 17 15:35:18 er1402 zmc_m6[4552]: WAR [Corrupt JPEG data: 1 extraneous bytes before marker 0xd6]
Jul 17 15:35:19 er1402 zmc_m6[4552]: WAR [Corrupt JPEG data: 2 extraneous bytes before marker 0xd7]
Jul 17 15:35:20 er1402 zmc_dvideo0[1869]: INF [Monitor-2: 36000 - Capturing at 9.90 fps]
Jul 17 15:35:20 er1402 zmc_m6[4552]: WAR [Corrupt JPEG data: 1 extraneous bytes before marker 0xd0]
Jul 17 15:35:21 er1402 zma_m2[1873]: INF [Monitor-2: 36000 - Processing at 9.90 fps]
Jul 17 15:35:21 er1402 zmc_m6[4552]: WAR [Corrupt JPEG data: 2 extraneous bytes before marker 0xd5]
Jul 17 15:35:22 er1402 zmc_m6[4552]: WAR [Corrupt JPEG data: 2 extraneous bytes before marker 0xd6]
Jul 17 15:35:23 er1402 zmc_m6[4552]: WAR [Corrupt JPEG data: 2 extraneous bytes before marker 0xd7]/code]
If I stream by capturing from the camera in yuv mode then it works perfecly (albeit with the extra load on the camera server).
Any ideas ? Is this the Logitech at fault or something else ? I've tried it with two different C510 webcams.
Ta in advance
Dom
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Well I have faced with the same issue with a Philips SPC1030NC webcam. In JPG mode the frames came directly from the V4L2 API which relies on the data recieieved from the kernel modules. Unfortunatelly I have no camera where the issue can be reproduced to debug it.
If you need it I can create a workaround to it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1. Recompile libjpeg and remove the bit where it moans about the corrupt data
2. Turn off Zoneminder logs
3. Use yuv and use extra processing power
4. Use the application "motion" instead of Zoneminder
I've only had a quick play with motion this morning but it's perfectly happy to accept the mjpg stream from mjgp-streamer and there are no errors in any logs to show its having any issues. Having had a play with motion, I actually prefer it to Zoneminder - it seems to have a better motion detection system, uses less overhead processing power and hasn't argued with me at all yet.
I may well just ditch ZM and write my own small system using motion as my detection/recording system.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In our case there were some rubbish bytes after the end of the JPG image. This data came directly from the kernel. It is relatively easy to workaround because the image should alays ends with the EOI (End Of Image) marker (FF D9) f I remember well. In that case if the buffer not ends with these bytes we can seek back for a few bytes to determine the length of the rubbish at the end of the image. Anyway I have used the Qt framework QImage class for reading image, what could decormpress this image without any problem.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
would a dump of the stream using wget be of any use for you ? I've got limited Linux knowledge but I'm quite happy to grab a few mb of stream and host it up for your perusal ?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks - no desperate hurry as I'm quite liking "motion" but obviously if I'm having this problem then others probably will too.
I've spotted a couple of posts on the Logitech forum from people saying they are having a similar problem with the same warnings although there is no mention of mjpg-streamer so it may just be that the webcams spit out non standard frames in mjpeg mode.
If you need any other logs or even want to sort out a few hours where I'll open my firewall and let you live stream the images then let me know - its always good when you can replicate a problem.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Very pleased to have found this thread. I also had the same issue with a Logitech Quickcam Pro 9000 and Zoneminder. Of course the JPEG errors are not occurring when I pass -y for YUV mode, however this does have quite a performance impact even on a quad core server.
I should be very grateful if somebody could add a feature to mjpg_streamer so that it validates the JPEG images received from kernel/v4l and does not pass on the corrupt ones. It appears that when Zoneminder is fed a corrupted data frame it breaks various parts of the software.
Is there anything I can do to assist with this development? I would also be happy to provide access to the camera/data files if this would be helpful.
Kind regards,
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've been running "motion" with mjpg-streamer for the past couple of weeks without issue now using mjpg compression on my Logitech - I'm actually liking it far more than Zoneminder (and no errors in the logs) - I'll probably ditch ZM now I've found new confidence in motion - it takes a bit of getting used to and setting up but its rock solid software so far.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have found 71 images in your file, but there were not any rubbish bytes after the images neither before them.
Are you sure that the sample stream contains corrupt frames?
If yes, your situation is different what I had with my Philish webcam.
In this case further investigation of the corrupted JPG data is needed.
Thanks,
Miklós
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hmm. After looking a bit deeper to the JPG format I have realized that there are many markers in the file. Writing a code from scratch to find them should not worth trying. If we are lucky than the libjpg would have some kind of image check feature. I will look after it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It appears to be extremely stable in the configuration I'm using it in at the moment - I'm grabbing in mjpg and analysing it with "Motion" which all works fine so although the problem might still be there, for my application its irrelevent as I've dropped ZM - I didn't get on with its motion detection in the end.
I'm quite happy to sort out some more recroded files at some point in the future or even put it on a machine and let you drag down live streams but at the moment I've got a couple of weeks of heavy building to do as I'm sorting out a new workshop :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you for the corrupt marker finder program. I am running in YUV mode however will switch to MJPEG mode overnight and check the output file for corrupt frames. The issue I forsee is that if the corrupt frames occur several hours into the recording this will be a very large binary file and I have no idea how to truncate it to show only the garbage frames. Perhaps I should write a small script to stop the capture and check the output for bad frames, and if none found overwrite the file, then check again a few minutes later repeat until bad frames found.
However martonmiklos seems to suggest that it is not worthwhile to write program to check integrity of JPG images (reinvent the wheel?) and instead we should look for function in libjpeg. Does this mean that I should not try to capture the garbage frames as they are not helpful?
Is this something we should raise with the libjpeg maintainers?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Firstly thank you for all the effort and work put into this software - its much appreciated.
I'm testing mjpg-streamer out for network enabling various webcams I have around the house for CCTV purposes and recording the streams via Zoneminder (possibly by Motion if I can't get Zoneminder motion detection working to a decent standard).
My webcams are Logitch C510 - fantastic little cameras and support MJPEG compression.
I can capture and stream in both yuv and mjpg formats - obviously the mjpeg if preferred as it has virtually no CPU overhead.
The problem I have is that when I stream with mjpeg enabled on the camera, my syslog fills up with the following :
Well I have faced with the same issue with a Philips SPC1030NC webcam. In JPG mode the frames came directly from the V4L2 API which relies on the data recieieved from the kernel modules. Unfortunatelly I have no camera where the issue can be reproduced to debug it.
If you need it I can create a workaround to it.
I've found a few possible solutions so far …….
1. Recompile libjpeg and remove the bit where it moans about the corrupt data
2. Turn off Zoneminder logs
3. Use yuv and use extra processing power
4. Use the application "motion" instead of Zoneminder
I've only had a quick play with motion this morning but it's perfectly happy to accept the mjpg stream from mjgp-streamer and there are no errors in any logs to show its having any issues. Having had a play with motion, I actually prefer it to Zoneminder - it seems to have a better motion detection system, uses less overhead processing power and hasn't argued with me at all yet.
I may well just ditch ZM and write my own small system using motion as my detection/recording system.
In our case there were some rubbish bytes after the end of the JPG image. This data came directly from the kernel. It is relatively easy to workaround because the image should alays ends with the EOI (End Of Image) marker (FF D9) f I remember well. In that case if the buffer not ends with these bytes we can seek back for a few bytes to determine the length of the rubbish at the end of the image. Anyway I have used the Qt framework QImage class for reading image, what could decormpress this image without any problem.
would a dump of the stream using wget be of any use for you ? I've got limited Linux knowledge but I'm quite happy to grab a few mb of stream and host it up for your perusal ?
Hello,
Yes please create a dump just to see that your problem is the similar to mine.
Create a file called nc_request.txt with the following contents:
GET /?action=stream HTTP/1.0
then run the following:
nc context.phys.unideb.hu 8080 < nc_request.txt > nc_out.bin
And of course please replace the context.phys.unideb.hu with your streamer's IP address.
Hi - the NC command just creates a zero byte nc_out.bin file unfortunately :(
I'm using Ubunto 11.04 if that makes any difference.
Have you replaced the hostname with your server's address?
Have you replaced the hostname with your server's address?
Yes - well I think I'm doing it right ……..
nc_out.bin contains zero bytes
I can access 192.168.1.210 from another machine and the mjpg-streamer is running fine
Ahh I have forgotten to mention that you should hit enter at the line end in the nc_request file
Here you go - http://www.picbits.co.uk/nc_out.bin
Can you let me know when you've got a copy and I'll remove it from the server ? Thanks
Done. I will try to analyze it asap.
Thanks - no desperate hurry as I'm quite liking "motion" but obviously if I'm having this problem then others probably will too.
I've spotted a couple of posts on the Logitech forum from people saying they are having a similar problem with the same warnings although there is no mention of mjpg-streamer so it may just be that the webcams spit out non standard frames in mjpeg mode.
If you need any other logs or even want to sort out a few hours where I'll open my firewall and let you live stream the images then let me know - its always good when you can replicate a problem.
logitech quickcam pro 9000 - Corrupt JPEG data:
Very pleased to have found this thread. I also had the same issue with a Logitech Quickcam Pro 9000 and Zoneminder. Of course the JPEG errors are not occurring when I pass -y for YUV mode, however this does have quite a performance impact even on a quad core server.
I should be very grateful if somebody could add a feature to mjpg_streamer so that it validates the JPEG images received from kernel/v4l and does not pass on the corrupt ones. It appears that when Zoneminder is fed a corrupted data frame it breaks various parts of the software.
Is there anything I can do to assist with this development? I would also be happy to provide access to the camera/data files if this would be helpful.
Kind regards,
I've been running "motion" with mjpg-streamer for the past couple of weeks without issue now using mjpg compression on my Logitech - I'm actually liking it far more than Zoneminder (and no errors in the logs) - I'll probably ditch ZM now I've found new confidence in motion - it takes a bit of getting used to and setting up but its rock solid software so far.
bbdom!
Finally I have hooked up a small C program which checks the SOI and EOI markers in the JPG stream:
https://dl.dropbox.com/u/4295670/corrupt_marker_finder.c
I have found 71 images in your file, but there were not any rubbish bytes after the images neither before them.
Are you sure that the sample stream contains corrupt frames?
If yes, your situation is different what I had with my Philish webcam.
In this case further investigation of the corrupted JPG data is needed.
Thanks,
Miklós
Hmm. After looking a bit deeper to the JPG format I have realized that there are many markers in the file. Writing a code from scratch to find them should not worth trying. If we are lucky than the libjpg would have some kind of image check feature. I will look after it.
Thanks for looking at it :-)
It appears to be extremely stable in the configuration I'm using it in at the moment - I'm grabbing in mjpg and analysing it with "Motion" which all works fine so although the problem might still be there, for my application its irrelevent as I've dropped ZM - I didn't get on with its motion detection in the end.
I'm quite happy to sort out some more recroded files at some point in the future or even put it on a machine and let you drag down live streams but at the moment I've got a couple of weeks of heavy building to do as I'm sorting out a new workshop :)
Thank you for the corrupt marker finder program. I am running in YUV mode however will switch to MJPEG mode overnight and check the output file for corrupt frames. The issue I forsee is that if the corrupt frames occur several hours into the recording this will be a very large binary file and I have no idea how to truncate it to show only the garbage frames. Perhaps I should write a small script to stop the capture and check the output for bad frames, and if none found overwrite the file, then check again a few minutes later repeat until bad frames found.
However martonmiklos seems to suggest that it is not worthwhile to write program to check integrity of JPG images (reinvent the wheel?) and instead we should look for function in libjpeg. Does this mean that I should not try to capture the garbage frames as they are not helpful?
Is this something we should raise with the libjpeg maintainers?
My setup outputs the corrupt frames almost immediately - I only needed 10 seconds or so before my syslog started filling up with the warnings.