Menu
▾
▴
#806 "not new enough" file when using CURL_TIMECOND_IFMODSINCE
Hi,
I'm using libcurl 7.18.2-8 in a C program, and I get the message "The requested document is not new enough" even if the server sends a file with response 200 OK. The problem I have with this behaviour is that the program depends on the HTTP response code to know if there was a more recent file or not on the server, and expects to receive a file when getting 200 OK. But in this case, curl doesn't modify the http headers, and makes a 0-byte file.
This behaviour happens if the server doesn't have a correct time, and is in the past. If the client sends a request with an If-Modified-Since header with a future date for the server, then the server can send a 200 OK response. I'm joining a file to show one transfer like this.
In my opinion, curl shouldn't check the file time in addition to the check done by the server, except if there is a good reason. I have checked curl code, and it seems that if there is a 304 http response received, libcurl already behaves correctly, and there is an additional code to check the filetime returned by the server (line 498 of lib/transfer.c on revision 7.19.3)
Thanks !
libcurl log showing the transfer
I beg to differ.
The option you pass to libcurl is about requesting a document modified since a specific date, and libcurl takes measures to only deliver the document if newer (or older) than the specified time. Why would you like to get the document that hasn't changed even though you explicitly asked you didn't want it then?
What I think could be argued is how this is signaled to the application, as just returning CURLE_OK doesn't feel quite right to me as that would imply a normal transfer which with it isn't exactly...
Well, there are two ways of seeing things:
- we trust everything that the server sends, so libcurl doesn't have to check the time, and only returns what the server sent (http response code anf file if one was sent)
- we don't trust the server, so libcurl do another check on time. In this case, I think that libcurl shouldn't mislead the application in thinking that the server sent a 0-byte file.
If I understand your point, you prefer the second possibility. In this case, shouldn't libcurl sends http response code 304 instead of an error ?
If you prefer not sending CURLE_OK, that's fine - I suppose that the application will still be able to understand what happened by looking at the error code.
Thinking about this further, we have the exact same problem with FTP but for that protocol there is no 304 or similar so in order to signal this back to the app we need something new.
I'm now more thinking of a new info-variable to be readable after a transfer that was done with a time-condition, as I think it isn't quite an error so returning an error-code would be a bit weird methinks. It would then presumably also get set when a 304 is returned on a time-conditional request.
What do you think about that idea?
I suppose that means that the application won't have to check the http response code, but only this new information variable. This variable would then be almost like a "valid" bit, telling the application if the server's file is more recent than the local one, and if libcurl has updated the file or not. If that is what you propose, I think that it is a good idea
There is still the problem on how to handle already existing applications, as they will get the bug I reported. I suppose that you want the programmer to modify the application, and use only a recent version of libcurl (or use #ifdef to know which libcurl version he is using).
If libcurl fails to download the file because of the time condition, will there be a guarantee that the target file won't be modified ?
CURLINFO_CONDITIONAL test shot
Yes, that's what I mean. I've just uploaded condition.patch which introduces the CURLINFO_CONDITIONAL info variable that returns 1 if libcurl didn't transfer a file due to a condition you specified.
Existing libcurl versions of course already exist and need to be dealt with, but then you would have to no matter how this gets fixed. And yes, you'd have to do something conditional to make your app works with both this new stuff and older libcurls.
> If libcurl fails to download the file because of the time condition, will there be a guarantee that the target file
> won't be modified ?
Mostly libcurl doesn't touch target files at all, as your app would deal with that. But yes, if the condition prevents a file to get transfered it will not call the write callback (not even the built-in if no custom one has been set).
Apart from the name of the new variable that I don't really like, your patch looks good :)
A better name could be CURLINFO_CONDITIONAL_FAILED or CURLINFO_UNMET_TIME_CONDITION, or even CURLINFO_FAILED_TIMECOND (which is a name similar to the option one)
How about CURLINFO_CONDITION_UNMET ? I want to keep it open as "condition" as then it could serve for other (related) conditions in the future or so.
If you tell me your real name I'll give you proper credit for this work in the changelog!
CURLINFO_CONDITION_UNMET was committed just now anyway, so if you'd like to try it out and give it a check I'll let this entry remain open for a while more.
Thanks for the report!
I have tried your new code, and it seems to work correctly with my application. Only one question: is CURLINFO_CONDITION_UNMET info variable supposed to be set to 0 if I'm not using CURL_TIMECOND_IFMODSINCE ? And where is this info variable cleared in the library code ?
It looks like that the info variable is always set when I don't set CURL_TIMECOND_IFMODSINCE.
For reference, my real name is Jocelyn Jaubert.
Thanks.
Please forget what I have written earlier: I had linked with the old library instead of the new library. Now that I'm using the correct library, I can see that CURLINFO_CONDITION_UNMET info variable is set to FALSE when I'm not using CURL_TIMECOND_IFMODSINCE.
Thanks, case closed!