Hi there, I've just built the libraries for my hardware and I'm running the test_onvif program.
It finds 7 cameras (some are repeated) and it returns the rtsp url for the camera which does not have authentication enabled. However it fails wih the message "ter:NotAuthorized" on cameras which have authentication enabled. I have checked the username and password are correct using OnvifDeviceManager.
Any suggestions for what might be the problem?
Many thanks
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi David, thank you for your message. It's helpful to know the type of camera that you are working with as they all are somewhat different in their implementations. If I have to guess, I would say that it is Axis cameras you are working with. I have seen them repeated on discovery as sometimes they will answer twice to the discovery message. Something I have seen with that camera is that authentication can fail if the Daylight Savings Time setting is not consistent between the camera and the computer. The basic idea of this theory is that the authentication parameters are only valid for a short time and the camera will think that the time is expired due to a mismatch between the computer and camera regarding the Daylight Savings Time settings,
If you can look at the settings for the camera and see if this is indeed that case, that may be a clue to what is going on. you could also try adjusting the clock on the camera to match the computer. I would be curious to see the result, especially if Onvif Device Manager is successfully authenticating in this situation.
Last edit: Stephen Rhodes 2019-06-12
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi Stephen, Many thanks for the repsonse.
The problem with the Axis camera was indeed the date-time and daylight-saving settings which didn't match the PC client, this camera is now working with libonvif.
I have another camera which does not have a daylight-saving setting and which cannot currently be authenticated.
I also have other cameras which are returning "No XML reply" which I wil need to debug.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Establishing communication with the cameras is a multi step process, so I guess the first troubleshooting effort would be identifying where in the process the failure is occurring.
The first step is the broadcast and response via UDP between the PC and the camera, If the libonvif_test program sees the camera and prompts for the username and password, you know this step has been successful.
The next step is for the PC to query the camera for its date and time info. Note that this step does not require authentication. The PC will use the time info collected to construct the password hash used for authentication, which involves computing the time difference between the PC and the camera to give a more accurate timestamp to the hash.
Next comes the process of obtaining the RTSP string, which is a three step process. The PC will ask GetCapabilities then GetProfile then GetStreamURI in order as each step requires information aquired in the previous step. These are all authenticated requests, so they wil not be successful without proper authentication.
It is possible to observe these steps directly by adding some code to the onvif.c file. You may observe the UDP responses from the cameras by adding the line
fprintf(stdout, "%s\n", onvif_session->buf[i]);
to the looping part of the brodcast routine. The code phrase should look like this after you have made the modification
Similarly, you may observe the command and response sequences following discovery by adding code to the sendCommandToCamera routine at the beginning of the routine to show the command
Remember to save the changes in your editor and then re-build the project the same as when you built it the first time. The libonvif_test program should now be showing a dump of all the internal communications between the PC and the camera, which may help with your troubleshooting efforts.
Something I have found working with these cameras is that most do not adhere strictly to the ONVIF standard, so there may be mis-communication. They may also use malformed XML syntax which can throw off the parser. Some cameras require you to explicitly enable ONVIF in their configuration before you can communicate with them. Many cameras will not produce error messages in the event of failure or they may show a non-compliant error message which is not picked up by libonvif. Most cameras do not enforce the time limits on the authentication hash, but a few will.
If you are familiar with Wireshark or tcpdump, you can use these utilities as well. If Onvif Device Manager is successfully communicating with the cameras and you are able to observe the communications, you may be able to make adjustments to the program to emulate the message transaction.
Let me know how things turn out or if you need additional info.
Last edit: Stephen Rhodes 2019-06-13
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi Stephen ,
I'm back on this project now. I added all of your debug stuff and I think I can see the problem with the camera which returns "Error getting camera uri - No XML reply".
In the broadcast response section for a good camera I get this:
So, in the XAddrs section, the link-local IP address, 169.254.81.173, comes first and the rest of the software uses this address rather than the real IP address, 10.26.7.86, which comes after the link-local address. How do I get it to use the real IP address?
Best regards
David
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Good to hear from you, hope you enjoyed your break.
It looks like the axis camera has both a wireless and wired Ethernet connection. The 169 subnet IP address looks to be the unconnected port. Libonvif only checks the first address and uses that one. If you reboot the axis camera in it's current state it should come back with the 10 subnet address as the first address.
Give the axis camera a reboot to try that and see if it works.
It will probably be worthwhile to add some code to libonvif to check for this condition and automatically find the valid port address
Hi Stephen ,
I'm back on this project now. I added all of your debug stuff and I think I can see the problem with the camera which returns "Error getting camera uri - No XML reply".
In the broadcast response section for a good camera I get this:
<soap-env:envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:d="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:soap-enc="http://www.w3.org/2003/05/soap-encoding" xmlns:dn="http://www.onvif.org/ver10/network/wsdl"><soap-env:header><wsa:messageid>urn:uuid:e1279c3b-6659-405b-8fec-fd32d244bc43</wsa:messageid><wsa:relatesto>urn:uuid:fecfb31b-6e7d-9041-56e4-390df1d28082</wsa:relatesto><wsa:to soap-env:mustunderstand="true">http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:to><wsa:action soap-env:mustunderstand="true">http://schemas.xmlsoap.org/ws/2005/04/discovery/ProbeMatches</wsa:action><d:appsequence instanceid="1218790878" soap-env:mustunderstand="true" messagenumber="2159"></d:appsequence></soap-env:header><soap-env:body><d:probematches><d:probematch><wsa:endpointreference><wsa:address>urn:uuid:0007d817-0007-4d81-8709-0007d8170910</wsa:address></wsa:endpointreference><d:types>dn:NetworkVideoTransmitter</d:types><d:scopes>onvif://www.onvif.org/type/video_encoder onvif://www.onvif.org/hardware/VK2-1080VFD onvif://www.onvif.org/name/VISTA_VK2-1080VFD onvif://www.onvif.org/location </d:scopes><d:xaddrs>http://10.26.7.126/onvif/device_service http://169.254.233.251/onvif/device_service</d:xaddrs><d:metadataversion>1</d:metadataversion></d:probematch></d:probematches></soap-env:body></soap-env:envelope>
but for the Axis M3007 camera I get this:
<soap-env:envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:d="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:soap-enc="http://www.w3.org/2003/05/soap-encoding" xmlns:dn="http://www.onvif.org/ver10/network/wsdl"><soap-env:header><wsa:messageid>uuid:a60fa4a4-b763-11e9-bb04-00408ce1d601</wsa:messageid><wsa:relatesto>urn:uuid:fecfb31b-6e7d-9041-56e4-390df1d28082</wsa:relatesto><wsa:to soap-env:mustunderstand="true">http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:to><wsa:action soap-env:mustunderstand="true">http://schemas.xmlsoap.org/ws/2005/04/discovery/ProbeMatches</wsa:action><d:appsequence instanceid="1559918212" soap-env:mustunderstand="true" messagenumber="640"></d:appsequence></soap-env:header><soap-env:body><d:probematches><d:probematch><wsa:endpointreference><wsa:address>urn:uuid:4454341c-b02d-11e7-b4a2-00408ce1d601</wsa:address></wsa:endpointreference><d:types>dn:NetworkVideoTransmitter</d:types><d:scopes>onvif://www.onvif.org/type/video_encoder onvif://www.onvif.org/Profile/Streaming onvif://www.onvif.org/type/ptz onvif://www.onvif.org/hardware/M3007 onvif://www.onvif.org/name/AXIS%20M3007 onvif://www.onvif.org/location/ </d:scopes><d:xaddrs>http://169.254.81.173/onvif/device_service http://10.26.7.86/onvif/device_service</d:xaddrs><d:metadataversion>1</d:metadataversion></d:probematch></d:probematches></soap-env:body></soap-env:envelope>
So, in the XAddrs section, the link-local IP address, 169.254.81.173, comes first and the rest of the software uses this address rather than the real IP address, 10.26.7.86, which comes after the link-local address. How do I get it to use the real IP address?
Hi Stephen,
Ok, so I've written some code to take the second Ip address if the first one is a 169 subnet address and that fixed that particular issue. Other cameras return an Ipv6 address in addition to the Ipv4 address so I may also have to check for this too.
Another camera on the network was responding too slowly for libonvif and the responses were being ignored. I changed this line in the sendCommandToCamera function in onvif.c to wait for a response.
valread = recv( sock, buffer, 4096, MSG_WAITALL);
This fixed the problem but I'm not sure if this is the best way forward here.
I now have the problem where cameras are responding twice with ProbeMatch responses and so they are being counted twice. I'm not sure why this is as ODM doesn't do it. Maybe I just have to check on the camera list if I already received a ProbeMatch for that ip address.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The issue of cameras responding twice is easily fixed by keeping track of the xaddrs sent by the camera in the main program using libonvif. This is really more of a higher level function which is why it is not addressed by libonvif itself.
It's interesting that there is a camera that takes too long to respond. Your approach may be correct, it's hard to tell without testing. I guess if your fix doesn't affect otherwise properly functioning cameras then you should be good to go.
Checking for the 169 address will probably get you a good result for most cases. It seems like that will be the address used if the port is inactive. I think libonvif will filter out ipv6 addresses but I would have to double check that.
All in all I would say nicely done, and if you don't mind I would like to add these changes to the next version.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have implemented a list of the xaddrs in my application without changing libonvif. However, this is not quite so easy as you also have to keep a record of the camera index, because prepareOnvifData() requires the index to return the OnvifData structure for that camera. Basically, I build a list containing xaddrs and their corresponding libonvif indices.
Sure, feel free to use any or all of this in the next version.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi there, I've just built the libraries for my hardware and I'm running the test_onvif program.
It finds 7 cameras (some are repeated) and it returns the rtsp url for the camera which does not have authentication enabled. However it fails wih the message "ter:NotAuthorized" on cameras which have authentication enabled. I have checked the username and password are correct using OnvifDeviceManager.
Any suggestions for what might be the problem?
Many thanks
Hi David, thank you for your message. It's helpful to know the type of camera that you are working with as they all are somewhat different in their implementations. If I have to guess, I would say that it is Axis cameras you are working with. I have seen them repeated on discovery as sometimes they will answer twice to the discovery message. Something I have seen with that camera is that authentication can fail if the Daylight Savings Time setting is not consistent between the camera and the computer. The basic idea of this theory is that the authentication parameters are only valid for a short time and the camera will think that the time is expired due to a mismatch between the computer and camera regarding the Daylight Savings Time settings,
If you can look at the settings for the camera and see if this is indeed that case, that may be a clue to what is going on. you could also try adjusting the clock on the camera to match the computer. I would be curious to see the result, especially if Onvif Device Manager is successfully authenticating in this situation.
Last edit: Stephen Rhodes 2019-06-12
Hi Stephen, Many thanks for the repsonse.
The problem with the Axis camera was indeed the date-time and daylight-saving settings which didn't match the PC client, this camera is now working with libonvif.
I have another camera which does not have a daylight-saving setting and which cannot currently be authenticated.
I also have other cameras which are returning "No XML reply" which I wil need to debug.
Establishing communication with the cameras is a multi step process, so I guess the first troubleshooting effort would be identifying where in the process the failure is occurring.
The first step is the broadcast and response via UDP between the PC and the camera, If the libonvif_test program sees the camera and prompts for the username and password, you know this step has been successful.
The next step is for the PC to query the camera for its date and time info. Note that this step does not require authentication. The PC will use the time info collected to construct the password hash used for authentication, which involves computing the time difference between the PC and the camera to give a more accurate timestamp to the hash.
Next comes the process of obtaining the RTSP string, which is a three step process. The PC will ask GetCapabilities then GetProfile then GetStreamURI in order as each step requires information aquired in the previous step. These are all authenticated requests, so they wil not be successful without proper authentication.
It is possible to observe these steps directly by adding some code to the onvif.c file. You may observe the UDP responses from the cameras by adding the line
fprintf(stdout, "%s\n", onvif_session->buf[i]);
to the looping part of the brodcast routine. The code phrase should look like this after you have made the modification
if (onvif_session->len[i] > 0) { onvif_session->buf[i][onvif_session->len[i]] = '\0'; fprintf(stdout, "%s\n", onvif_session->buf[i]); i++; }Similarly, you may observe the command and response sequences following discovery by adding code to the sendCommandToCamera routine at the beginning of the routine to show the command
xmlDocPtr sendCommandToCamera(char *cmd, char *xaddrs) { fprintf(stdout, "%s\n", cmd);and at the end of the routine to show the repsonse
fprintf(stdout, "%s\n", xml_reply); xmlDocPtr reply = xmlParseMemory(xml_reply, xml_length); char error_msg[1024] = {0}; return reply;Remember to save the changes in your editor and then re-build the project the same as when you built it the first time. The libonvif_test program should now be showing a dump of all the internal communications between the PC and the camera, which may help with your troubleshooting efforts.
Something I have found working with these cameras is that most do not adhere strictly to the ONVIF standard, so there may be mis-communication. They may also use malformed XML syntax which can throw off the parser. Some cameras require you to explicitly enable ONVIF in their configuration before you can communicate with them. Many cameras will not produce error messages in the event of failure or they may show a non-compliant error message which is not picked up by libonvif. Most cameras do not enforce the time limits on the authentication hash, but a few will.
If you are familiar with Wireshark or tcpdump, you can use these utilities as well. If Onvif Device Manager is successfully communicating with the cameras and you are able to observe the communications, you may be able to make adjustments to the program to emulate the message transaction.
Let me know how things turn out or if you need additional info.
Last edit: Stephen Rhodes 2019-06-13
Thanks again, I have to take a break from this work for a few weeks but I'll let you know when I get back to it
Hi Stephen ,
I'm back on this project now. I added all of your debug stuff and I think I can see the problem with the camera which returns "Error getting camera uri - No XML reply".
In the broadcast response section for a good camera I get this:
<soap-env:envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:d="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:soap-enc="http://www.w3.org/2003/05/soap-encoding" xmlns:dn="http://www.onvif.org/ver10/network/wsdl"><soap-env:header><wsa:messageid>urn:uuid:e1279c3b-6659-405b-8fec-fd32d244bc43</wsa:messageid><wsa:relatesto>urn:uuid:fecfb31b-6e7d-9041-56e4-390df1d28082</wsa:relatesto><wsa:to soap-env:mustunderstand="true">http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:to><wsa:action soap-env:mustunderstand="true">http://schemas.xmlsoap.org/ws/2005/04/discovery/ProbeMatches</wsa:action><d:appsequence instanceid="1218790878" soap-env:mustunderstand="true" messagenumber="2159"></d:appsequence></soap-env:header><soap-env:body><d:probematches><d:probematch><wsa:endpointreference><wsa:address>urn:uuid:0007d817-0007-4d81-8709-0007d8170910</wsa:address></wsa:endpointreference><d:types>dn:NetworkVideoTransmitter</d:types><d:scopes>onvif://www.onvif.org/type/video_encoder onvif://www.onvif.org/hardware/VK2-1080VFD onvif://www.onvif.org/name/VISTA_VK2-1080VFD onvif://www.onvif.org/location </d:scopes><d:xaddrs>http://10.26.7.126/onvif/device_service http://169.254.233.251/onvif/device_service</d:xaddrs><d:metadataversion>1</d:metadataversion></d:probematch></d:probematches></soap-env:body></soap-env:envelope>but for the Axis M3007 camera I get this:
<soap-env:envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:d="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:soap-enc="http://www.w3.org/2003/05/soap-encoding" xmlns:dn="http://www.onvif.org/ver10/network/wsdl"><soap-env:header><wsa:messageid>uuid:a60fa4a4-b763-11e9-bb04-00408ce1d601</wsa:messageid><wsa:relatesto>urn:uuid:fecfb31b-6e7d-9041-56e4-390df1d28082</wsa:relatesto><wsa:to soap-env:mustunderstand="true">http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:to><wsa:action soap-env:mustunderstand="true">http://schemas.xmlsoap.org/ws/2005/04/discovery/ProbeMatches</wsa:action><d:appsequence instanceid="1559918212" soap-env:mustunderstand="true" messagenumber="640"></d:appsequence></soap-env:header><soap-env:body><d:probematches><d:probematch><wsa:endpointreference><wsa:address>urn:uuid:4454341c-b02d-11e7-b4a2-00408ce1d601</wsa:address></wsa:endpointreference><d:types>dn:NetworkVideoTransmitter</d:types><d:scopes>onvif://www.onvif.org/type/video_encoder onvif://www.onvif.org/Profile/Streaming onvif://www.onvif.org/type/ptz onvif://www.onvif.org/hardware/M3007 onvif://www.onvif.org/name/AXIS%20M3007 onvif://www.onvif.org/location/ </d:scopes><d:xaddrs>http://169.254.81.173/onvif/device_service http://10.26.7.86/onvif/device_service</d:xaddrs><d:metadataversion>1</d:metadataversion></d:probematch></d:probematches></soap-env:body></soap-env:envelope>So, in the XAddrs section, the link-local IP address, 169.254.81.173, comes first and the rest of the software uses this address rather than the real IP address, 10.26.7.86, which comes after the link-local address. How do I get it to use the real IP address?
Best regards
David
Hi David,
Good to hear from you, hope you enjoyed your break.
It looks like the axis camera has both a wireless and wired Ethernet connection. The 169 subnet IP address looks to be the unconnected port. Libonvif only checks the first address and uses that one. If you reboot the axis camera in it's current state it should come back with the 10 subnet address as the first address.
Give the axis camera a reboot to try that and see if it works.
It will probably be worthwhile to add some code to libonvif to check for this condition and automatically find the valid port address
Sent from my iPhone
Hi Stephen,
Ok, so I've written some code to take the second Ip address if the first one is a 169 subnet address and that fixed that particular issue. Other cameras return an Ipv6 address in addition to the Ipv4 address so I may also have to check for this too.
Another camera on the network was responding too slowly for libonvif and the responses were being ignored. I changed this line in the sendCommandToCamera function in onvif.c to wait for a response.
valread = recv( sock, buffer, 4096, MSG_WAITALL);
This fixed the problem but I'm not sure if this is the best way forward here.
I now have the problem where cameras are responding twice with ProbeMatch responses and so they are being counted twice. I'm not sure why this is as ODM doesn't do it. Maybe I just have to check on the camera list if I already received a ProbeMatch for that ip address.
The issue of cameras responding twice is easily fixed by keeping track of the xaddrs sent by the camera in the main program using libonvif. This is really more of a higher level function which is why it is not addressed by libonvif itself.
It's interesting that there is a camera that takes too long to respond. Your approach may be correct, it's hard to tell without testing. I guess if your fix doesn't affect otherwise properly functioning cameras then you should be good to go.
Checking for the 169 address will probably get you a good result for most cases. It seems like that will be the address used if the port is inactive. I think libonvif will filter out ipv6 addresses but I would have to double check that.
All in all I would say nicely done, and if you don't mind I would like to add these changes to the next version.
I have implemented a list of the xaddrs in my application without changing libonvif. However, this is not quite so easy as you also have to keep a record of the camera index, because prepareOnvifData() requires the index to return the OnvifData structure for that camera. Basically, I build a list containing xaddrs and their corresponding libonvif indices.
Sure, feel free to use any or all of this in the next version.