From: Tim O. <ti...@br...> - 2007-08-24 21:51:18
|
I've noticed some problems with davfs if I use a Subversion autoversioning server and leave locking on. If I write to a file A and then rename it to B, I will get two files, A & B with 0 size. If I turn off locks, then everything works fine. (I get file B of proper size). Why does this happen? Is this a bug / missing feature in davfs or subversion? If I turn off locking, will davfs do conditional PUTs ? I am using davfs 1.2.2, subversion 1.4.2, and apache 2.2.3 Thanks for your help! -Tim |
From: Tim O. <ti...@br...> - 2007-08-24 22:29:07
|
btw, the two empty files are lock-empty files. Wireshark & apache's log show the interaction to look like the following: -> PROPFIND /svn/autov/tim/test/ <- 207 -> HEAD /svn/autov/tim/test/A <- 404 -> LOCK /svn/autov/tim/test/A <- 200 -> UNLOCK /svn/autov/test/A <- 204 -> LOCK /svn/autov/tim/test/B <- 200 -> HEAD /svn/autov/tim/test/B <- 200 -> UNLOCK /svn/autov/tim/test/B <- 204 But there are no PUT or MOVE requests. Thanks, Tim On Fri, 24 Aug 2007 17:51:13 -0400 Tim Olsen <ti...@br...> wrote: > I've noticed some problems with davfs if I use a Subversion > autoversioning server and leave locking on. > > If I write to a file A and then rename it to B, I will get two files, > A & B with 0 size. > > If I turn off locks, then everything works fine. (I get file B of > proper size). > > Why does this happen? Is this a bug / missing feature in davfs or > subversion? > > If I turn off locking, will davfs do conditional PUTs ? > > I am using davfs 1.2.2, subversion 1.4.2, and apache 2.2.3 > > Thanks for your help! > > -Tim > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a > browser. Download your FREE copy of Splunk now >> > http://get.splunk.com/ _______________________________________________ > Dav-linuxfs mailing list > Dav...@li... > https://lists.sourceforge.net/lists/listinfo/dav-linuxfs |
From: Tim O. <ti...@br...> - 2007-08-24 23:26:44
|
I think I've figured it out. See below. On Fri, 24 Aug 2007 18:28:28 -0400 Tim Olsen <ti...@br...> wrote: > > btw, the two empty files are lock-empty files. Wireshark & apache's > log show the interaction to look like the following: > > -> PROPFIND /svn/autov/tim/test/ > <- 207 > > -> HEAD /svn/autov/tim/test/A > <- 404 > > -> LOCK /svn/autov/tim/test/A > <- 200 > > -> UNLOCK /svn/autov/test/A > <- 204 > > -> LOCK /svn/autov/tim/test/B > <- 200 > > -> HEAD /svn/autov/tim/test/B > <- 200 Herein lies the problem. Davfs expects locknull behavior here (HEAD should return 404). Because a 200 is returned (lockempty behavior), davfs thinks somebody has put a file while it wasn't looking (Note that this is possible even with locknull behavior because davfs is not sending If-None-Match: * with the LOCK!) and silently fails. So I guess the answer is that davfs, subversion, and locking are just incompatible for now. Cheers, Tim > > -> UNLOCK /svn/autov/tim/test/B > <- 204 > > But there are no PUT or MOVE requests. > |
From: Werner B. <wer...@on...> - 2007-08-25 11:06:42
|
Hello Tim, thanks for your detailed report and your investigations. You encountered a really serious problem. Part of it was already known to me, but there seems to be a new problem with apache/subversion too. Conditional requests: --------------------- As you mentioned, davfs2 should (and would like to) use conditional requests on LOCK and PUT. This would be the proper way to avoid the "lost update problem". In case of PUT: If the file does not yet exist, it should send PUT /foo If-None-Match: * If the file already exists on the server: PUT /foo If-Match: "validator" "validator" is the Etag of the file that was downloaded from the server and then edited. Unforunately the implementation of conditional requests is badly broken in Apache/mod_dav. This is bad. But it is worse: This bug in Apache/mod_dav was first reported 2003-01-30 (http://issues.apache.org/bugzilla/show_bug.cgi?id=16593) and then again 2005-12-24 (http://issues.apache.org/bugzilla/show_bug.cgi?id=38034). It is not yet fixed. On 2007-07-26 I sent a patch to fix this bug: No response. So Apache/mod_dav seems to be unmaintained. This is really frustrating. Especially as I like free software and microsoft's IIS gets it right. If you know anybody involved in the development of Apache, please talk to them about this problem. HEAD requests: -------------- davfs2 tries to work around this problem using HEAD requests to check for changes on the server before uploading or requesting a lock for a new file. This works most of the time, but is ugly and not really secure, as there may be race conditions with two requests instead of one. Locked Null resources: ---------------------- Whether we use conditional requests or HEAD to check for changes on the server, the client must know whether the file already exists on the server or not. When a client locks a not-yet-existing file, it must know, whether the server creates a new empty file (new style) or a locked null resource (old style). This is quite easy: when the server creates a new file, it MUST respond with 201 CREATED. The combination Apache/subversion, you use, seems to violate the spec in creating a new file and responding with 200. This makes dealing with the lost update problem almost impossible. The only sensible behaviour for a client would be "don't care". But davfs2 wants to care about. Although this might not be a problem for subversion (as it creates a new version and does not delete anything), it is a big problem for all clients, that are not specialized on just one buggy server. Please, report this as a bug to the maintainer of the server software. Missing MOVEs: -------------- davfs2 delays uploading of files that are closed by the user for some seconds (exact time depends on the load) for two reasons: - many applications use temporary files, that will be deleted immediately after closing. Delaying the uploads avoids unnecessary traffic. - davfs2 is single threaded. Delaying large uploads improves (most time) the response time of davfs2 to calls from the application. So, if you create a new file A and rename it into B: when file A is not yet uploaded, davfs2 will just unlock A, and create and upload file B. Note: Some people think this feature is a bug. I intend to make it a configuration option. But this will not be in the near future. Cheers Werner |
From: Tim O. <ti...@br...> - 2007-08-27 15:42:23
|
On Sat, 25 Aug 2007 13:06:10 +0200 Werner Baumann <wer...@on...> wrote: > Unforunately the implementation of conditional requests is badly > broken in Apache/mod_dav. This is bad. > > But it is worse: > This bug in Apache/mod_dav was first reported 2003-01-30 > (http://issues.apache.org/bugzilla/show_bug.cgi?id=16593) and then > again 2005-12-24 > (http://issues.apache.org/bugzilla/show_bug.cgi?id=38034). It is not > yet fixed. On 2007-07-26 I sent a patch to fix this bug: No response. > > So Apache/mod_dav seems to be unmaintained. This is really > frustrating. Especially as I like free software and microsoft's IIS > gets it right. If you know anybody involved in the development of > Apache, please talk to them about this problem. I'll bug people on the httpd-dev mailing list. These bugs are violations of HTTP/1.1 which is something I would hope they would take seriously. > > HEAD requests: > -------------- > davfs2 tries to work around this problem using HEAD requests to check > for changes on the server before uploading or requesting a lock for a > new file. This works most of the time, but is ugly and not really > secure, as there may be race conditions with two requests instead of > one. > > Locked Null resources: > ---------------------- > Whether we use conditional requests or HEAD to check for changes on > the server, the client must know whether the file already exists on > the server or not. > > When a client locks a not-yet-existing file, it must know, whether > the server creates a new empty file (new style) or a locked null > resource (old style). This is quite easy: when the server creates a > new file, it MUST respond with 201 CREATED. > > The combination Apache/subversion, you use, seems to violate the spec > in creating a new file and responding with 200. This makes dealing > with the lost update problem almost impossible. The only sensible > behaviour for a client would be "don't care". But davfs2 wants to > care about. Although this might not be a problem for subversion (as > it creates a new version and does not delete anything), it is a big > problem for all clients, that are not specialized on just one buggy > server. > > Please, report this as a bug to the maintainer of the server software. > I believe the MUST requirement for a 201 response is new with RFC 4918 which specifies lockempty resources. So I think if the server is creating locknull resources, they can legitimately return a response of 200. Of course, in this case, subversion is returning 200 for a lockempty resource; hence, this is definitely a bug. I will report it to the subversion developers. There is still the problem, however, that davfs expects a 404 response to its HEAD request after performing a LOCK. Although a server MUST return a 201 when using lockempty resources, it would be nice if davfs could handle a 200 response to a HEAD request on a lockempty resource and not silently fail. Should I report this as a bug on davfs's bug tracker? > Missing MOVEs: > -------------- > davfs2 delays uploading of files that are closed by the user for some > seconds (exact time depends on the load) for two reasons: > - many applications use temporary files, that will be deleted > immediately after closing. Delaying the uploads avoids unnecessary > traffic. > - davfs2 is single threaded. Delaying large uploads improves (most > time) the response time of davfs2 to calls from the application. > > So, if you create a new file A and rename it into B: when file A is > not yet uploaded, davfs2 will just unlock A, and create and upload > file B. > > Note: Some people think this feature is a bug. I intend to make it a > configuration option. But this will not be in the near future. In this particular situation, I have no problem with this. I could see other people having a problem when you start to mix in BIND and DeltaV functionality where a MOVE would retain DAV:resource-id and/or version history. Thanks for your help! -Tim > > Cheers > Werner |
From: Werner B. <wer...@on...> - 2007-08-27 19:15:42
|
Hello Tim! Tim Olsen wrote: > I believe the MUST requirement for a 201 response is new with RFC 4918 > which specifies lockempty resources. So I think if the server is > creating locknull resources, they can legitimately return a response of > 200. Of course, in this case, subversion is returning 200 for a > lockempty resource; hence, this is definitely a bug. I will report > it to the subversion developers. As far as I know, the old RFC did not allow to create an empty file on LOCK and therefore 200 OK was the correct response. RFC 4918 changed this. Now the server MUST create an empty file and respond with 201 CREATED. Creating an empty file and responding with 200 OK never was an option. I think HTTP requires a 201 response, when a file is created in response to a request. > There is still the problem, however, that davfs expects a 404 response > to its HEAD request after performing a LOCK. Although a server MUST > return a 201 when using lockempty resources, it would be nice if davfs > could handle a 200 response to a HEAD request on a lockempty resource > and not silently fail. Should I report this as a bug on davfs's bug > tracker? davfs2 does treat this: it moves the local copy into lost+found. The problem: When davfs2 gets a 200 response on LOCK, and later gets 200 on HEAD, there are two possibilities: - the server sent the wrong response code (200 instead of 201) - somebody else created a file in between It is almost impossible to distinguish this two cases (that is why this server bug is that bad). So davfs2 treats it save. It does not overwrite the file somebody else may have created, but stores the local changes in lost+found. Second problem: davfs2 is a daemon and can only interact with the user via the file system interface. As this problem usually occurs when the file is closed and davfs2 tries to upload it after some delay, it can't interact with the user at all. Even if it would not delay the upload, it only could respond with EIO to the close request, which will be ignored by most applications. Do you have any idea, what davfs2 could do in this case? It should be save and not too complicated. > In this particular situation, I have no problem with this. I could see > other people having a problem when you start to mix in BIND and DeltaV > functionality where a MOVE would retain DAV:resource-id and/or version > history. Good point. I did not think of this. Dead properties might get lost too. So davfs2 should always send a MOVE request, when the file exists on the server. I will add this to the TODO list. Cheers Werner |
From: Werner B. <wer...@on...> - 2007-08-27 19:26:45
|
Hello again. Maybe I misunderstood port of your last email. So to clarify: When a server responds with 201 on a LOCK request, davfs2 knows that an empty file has been created, and does not expect 404 but 200 on a HEAD request. Indeed it will only send a HEAD request before uploading if it has an Etag or Last-Modified date of the empty file (to check for changes on the server). Werner |
From: Tim O. <ti...@br...> - 2007-08-27 19:46:18
|
On Mon, 27 Aug 2007 21:26:35 +0200 Werner Baumann <wer...@on...> wrote: > Hello again. > > Maybe I misunderstood port of your last email. > > So to clarify: > When a server responds with 201 on a LOCK request, davfs2 knows that > an empty file has been created, and does not expect 404 but 200 on a > HEAD request. Indeed it will only send a HEAD request before > uploading if it has an Etag or Last-Modified date of the empty file > (to check for changes on the server). ahh.. I see. Then I would agree that you're giving the server a decent chance to say that it's using lockempty by returning a 201 response. One option, however, is to look at the Content-Size of the HEAD request. If it's zero, then you're not really losing anything if you overwrite the file. -Tim > > Werner |
From: Werner B. <wer...@on...> - 2007-08-28 10:05:03
|
Hello Tim! Tim Olsen wrote: > ahh.. I see. Then I would agree that you're giving the server a decent > chance to say that it's using lockempty by returning a 201 response. > One option, however, is to look at the Content-Size of the HEAD > request. If it's zero, then you're not really losing anything if you > overwrite the file. I agree. It's on the TODO-list now. Cheers Werner |
From: Werner B. <wer...@on...> - 2007-11-03 12:06:25
|
Hello Tim, I have fixed the MOVE-bug and also added a check for length 0 to work around the SVN-bug. The changes are in our CVS-Repository. Before doing a new release, I would be thankful for any testing. As I don't have write access to a SVN-server, testing with SVN is much needed. BTW: On w3c...@w3... you wrote: > For example, Apache still does not understand If-None-Match * > correctly! (One of my team members, Paritosh Shah, has submitted a > patch to the Apache mailing list. Hopefully it will be accepted) I could not find this patch on the Apache mailing lists. As I sent a patch for this too, I would be very interested. Could you give me a hint, where to find it? Cheers Werner |
From: Tim O. <ti...@br...> - 2007-11-14 04:16:21
|
Hi Werner, Sorry for the delayed reply. I currently have this list sorted a mail folder which I don't read very often. I should setup my sieve filter to copy any emails personally addressed to me into my INBOX. On Nov 3, 2007, at 8:05 AM, Werner Baumann wrote: > Hello Tim, > > I have fixed the MOVE-bug and also added a check for length 0 to > work around the SVN-bug. The changes are in our CVS-Repository. > > Before doing a new release, I would be thankful for any testing. As > I don't have write access to a SVN-server, testing with SVN is much > needed. Sure thing. I would be glad to test the latest code. I am at ApacheCon right now but will gladly test it when I return to the office next week. > > > BTW: On w3c...@w3... you wrote: > > > For example, Apache still does not understand If-None-Match * > > correctly! (One of my team members, Paritosh Shah, has submitted a > > patch to the Apache mailing list. Hopefully it will be accepted) > > I could not find this patch on the Apache mailing lists. As I sent a > patch for this too, I would be very interested. Could you give me a > hint, where to find it? Look for the email thread titled "thoughts on ETags and mod_dav" on the httpd-dev list. The thread started on Oct 11. Paritosh posted his latest patch on Oct 19th. Cheers, Tim > > > Cheers > Werner |