rsnapshot

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: Matt McCutchen <matt@ma...> - 2009-02-02 18:54 On Mon, 2009-02-02 at 07:23 -0500, Charles Marcus wrote: > Next I created a second config file (rsnapshot_ext.conf), and when I run: > > rsnapshot -c /ext/rsnapshot_ext.conf -t daily > > The results don't show --link-dest being used: > > myhost ~ # rsnapshot -c /etc/rsnapshot_ext.conf -t daily > echo 1508 > /var/run/rsnapshot_ext.pid > mv /mnt/qnap1/snapshots/daily.0/ /mnt/qnap1/snapshots/daily.1/ > /usr/bin/rsync -a --delete --numeric-ids --delete-excluded \ > /backups/.rsnapshots/daily.0/myhost/ /mnt/qnap1/snapshots/daily.0/moria/ > touch /mnt/qnap1/snapshots/daily.0/ > myhost ~ # > > I'm assuming this means that rsnapshot doesn't think that the remote > filesystem support hard links? This is a bug in the test mode. Rsnapshot passes --link-dest on condition that the vious snapshot (in this case daily.1) exists. It fails to take into account that daily.1 would exist at that point had the mv actually been performed, so it omits the --link-dest option. A real rsnapshot run (rsnapshot -c /etc/rsnapshot_ext.conf -v daily) should be unaffected. -- Matt

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: Charles Marcus <CMarcus@Me...> - 2009-02-02 19:47 On 2/2/2009 1:54 PM, Matt McCutchen wrote: > This is a bug in the test mode. Rsnapshot passes --link-dest on > condition that the vious snapshot (in this case daily.1) exists. It > fails to take into account that daily.1 would exist at that point had > the mv actually been performed, so it omits the --link-dest option. > > A real rsnapshot run (rsnapshot -c /etc/rsnapshot_ext.conf -v daily) > should be unaffected. Oh, cool, thats good news... testing now. Thanks Matt! -- Best regards, Charles

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: Charles Marcus <CMarcus@Me...> - 2009-02-02 20:48 On 2/2/2009, Charles Marcus (CMarcus@Media-Brokers.com) wrote: >> This is a bug in the test mode. Rsnapshot passes --link-dest on >> condition that the vious snapshot (in this case daily.1) exists. It >> fails to take into account that daily.1 would exist at that point had >> the mv actually been performed, so it omits the --link-dest option. >> >> A real rsnapshot run (rsnapshot -c /etc/rsnapshot_ext.conf -v daily) >> should be unaffected. > Oh, cool, thats good news... testing now. Worked perfectly just as you said... Should I open a bug report for this or is it already fixed in cvs? Thanks again Matt. -- Best regards, Charles

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: Matt McCutchen <matt@ma...> - 2009-02-04 05:33 On Mon, 2009-02-02 at 15:48 -0500, Charles Marcus wrote: > On 2/2/2009, Charles Marcus (CMarcus@Media-Brokers.com) wrote: > >> This is a bug in the test mode. Rsnapshot passes --link-dest on > >> condition that the vious snapshot (in this case daily.1) exists. It > >> fails to take into account that daily.1 would exist at that point had > >> the mv actually been performed, so it omits the --link-dest option. > Should I open a bug report for this or is it already fixed in cvs? It's not fixed. You can open a bug report. <rant> Personally, I find this kind of bug demoralizing. We try to emulate a test mode (or the analogous --dry-run mode in rsync) in a simple way, but there will always be inaccuracies due to unanticipated interactions that have to be fixed in an ad-hoc way. In my view, the real solution would be to run the entire program and filesystem on a lazy functional evaluation engine and force evaluation of the output but not the final state of the filesystem. </rant> -- Matt

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: Charles Marcus <CMarcus@Me...> - 2009-02-04 14:08 On 2/4/2009 12:33 AM, Matt McCutchen wrote: >>>> This is a bug in the test mode. Rsnapshot passes --link-dest on >>>> condition that the vious snapshot (in this case daily.1) exists. It >>>> fails to take into account that daily.1 would exist at that point had >>>> the mv actually been performed, so it omits the --link-dest option. >> Should I open a bug report for this or is it already fixed in cvs? > It's not fixed. You can open a bug report. > > <rant> > Personally, I find this kind of bug demoralizing. We try to emulate a > test mode (or the analogous --dry-run mode in rsync) in a simple way, > but there will always be inaccuracies due to unanticipated interactions > that have to be fixed in an ad-hoc way. In my view, the real solution > would be to run the entire program and filesystem on a lazy functional > evaluation engine and force evaluation of the output but not the final > state of the filesystem. > </rant> So... you're saying you don't there should even be a -t option? It does come in handy sometimes... Anyway, thanks again, and bug report submitted: http://tiny.pl/vnqq -- Best regards, Charles

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: Matt McCutchen <matt@ma...> - 2009-02-06 04:57 On Wed, 2009-02-04 at 09:08 -0500, Charles Marcus wrote: > On 2/4/2009 12:33 AM, Matt McCutchen wrote: > > <rant> > > Personally, I find this kind of bug demoralizing. We try to emulate a > > test mode (or the analogous --dry-run mode in rsync) in a simple way, > > but there will always be inaccuracies due to unanticipated interactions > > that have to be fixed in an ad-hoc way. In my view, the real solution > > would be to run the entire program and filesystem on a lazy functional > > evaluation engine and force evaluation of the output but not the final > > state of the filesystem. > > </rant> > > So... you're saying you don't there should even be a -t option? It does > come in handy sometimes... I didn't make that clear. I do want an easy way to do test runs. The solution I was proposing involves running the "real" rsnapshot code on a special sandboxed computation engine, hence the better accuracy; the user could do that him/herself, or rsnapshot could have a -t option to do it. In any case, this is only theoretical as I don't know of any currently available tool that would be suitable for this process. For now, I am thinking of submitting a fix for the inaccuracy you found (since the situation that triggers it isn't particularly unusual and I think I have a simple fix), but I wanted to argue that fixing all inaccuracies in the test mode as it is currently implemented may not be a worthwhile or realistic goal. Then again, one could say the same of any software. -- Matt

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: Matt McCutchen <matt@ma...> - 2009-02-06 05:17 Attachments: fix-test-link-dest.diff      On Mon, 2009-02-02 at 13:54 -0500, Matt McCutchen wrote: > On Mon, 2009-02-02 at 07:23 -0500, Charles Marcus wrote: > > Next I created a second config file (rsnapshot_ext.conf), and when I run: > > > > rsnapshot -c /ext/rsnapshot_ext.conf -t daily > > > > The results don't show --link-dest being used: > > > > myhost ~ # rsnapshot -c /etc/rsnapshot_ext.conf -t daily > > echo 1508 > /var/run/rsnapshot_ext.pid > > mv /mnt/qnap1/snapshots/daily.0/ /mnt/qnap1/snapshots/daily.1/ > > /usr/bin/rsync -a --delete --numeric-ids --delete-excluded \ > > /backups/.rsnapshots/daily.0/myhost/ /mnt/qnap1/snapshots/daily.0/moria/ > > touch /mnt/qnap1/snapshots/daily.0/ > > myhost ~ # > > > > I'm assuming this means that rsnapshot doesn't think that the remote > > filesystem support hard links? > > This is a bug in the test mode. Rsnapshot passes --link-dest on > condition that the vious snapshot (in this case daily.1) exists. It > fails to take into account that daily.1 would exist at that point had > the mv actually been performed, so it omits the --link-dest option. The attached patch should fix this. Charles, please make sure it works for you. Then, with David Keegel's approval, I will commit it. -- Matt

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: Matt McCutchen <matt@ma...> - 2009-02-06 13:35 On Fri, 2009-02-06 at 07:27 -0500, Charles Marcus wrote: > On 2/6/2009 12:17 AM, Matt McCutchen wrote: > > The attached patch should fix this. Charles, please make sure it works > > for you. Then, with David Keegel's approval, I will commit it. > > I'll be happy to, but if you recall, I'm the one who needed your help > with a patched version of rsync (the case-insensitive patch)... ;) > > But, I'd like to do this myself this time (its about time I learned how > to do basic patching)... > > So... after some googling, I must be missing something... would it > really be as simple as: > > patch -b < fix-test-link-dest.diff > > ? Do I have to copy the patch to the directory where the file to be > patched resides? Or just run the patch command from the directory where > the patch resides? > > Hmmm... looking at the patch, it is patching rsnapshot-program.pl, but > find doesn't find that file anywhere on my system (gentoo linux)... The source tree contains the bulk of the code in a file called "rsnapshot-program.pl". The configure script makes a few small changes to it to generate "rsnapshot", which can then be installed in /usr/bin . Thus, the "right" way to apply the patch would be to check out rsnapshot from CVS: cvs -d:pserver:anonymous@rsnapshot.cvs.sourceforge.net:/cvsroot/rsnapshot co rsnapshot apply the patch in that directory, "make rsnapshot", and install the resulting "rsnapshot" script in your $PATH. But the differences between "rsnapshot-program.pl" and "rsnapshot" are so minor that you can probably just apply the patch directly to your /usr/bin/rsnapshot by explicitly specifying the file to patch: patch -b /usr/bin/rsnapshot < fix-test-link-dest.diff (Of course, that would need to run as root unless you make a copy of /usr/bin/rsnapshot first.) -- Matt

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: Matt McCutchen <matt@ma...> - 2009-02-07 03:23 On Fri, 2009-02-06 at 00:17 -0500, Matt McCutchen wrote: > On Mon, 2009-02-02 at 13:54 -0500, Matt McCutchen wrote: > > On Mon, 2009-02-02 at 07:23 -0500, Charles Marcus wrote: > > > Next I created a second config file (rsnapshot_ext.conf), and when I run: > > > > > > rsnapshot -c /ext/rsnapshot_ext.conf -t daily > > > > > > The results don't show --link-dest being used: > > > > > > myhost ~ # rsnapshot -c /etc/rsnapshot_ext.conf -t daily > > > echo 1508 > /var/run/rsnapshot_ext.pid > > > mv /mnt/qnap1/snapshots/daily.0/ /mnt/qnap1/snapshots/daily.1/ > > > /usr/bin/rsync -a --delete --numeric-ids --delete-excluded \ > > > /backups/.rsnapshots/daily.0/myhost/ /mnt/qnap1/snapshots/daily.0/moria/ > > > touch /mnt/qnap1/snapshots/daily.0/ > > > myhost ~ # > > > > > > I'm assuming this means that rsnapshot doesn't think that the remote > > > filesystem support hard links? > > > > This is a bug in the test mode. Rsnapshot passes --link-dest on > > condition that the vious snapshot (in this case daily.1) exists. It > > fails to take into account that daily.1 would exist at that point had > > the mv actually been performed, so it omits the --link-dest option. > > The attached patch should fix this. Charles, please make sure it works > for you. Charles, I'll add: please make sure you test the case that originally triggered the bug, in which the snapshot root contains a snapshot "daily.0" and nothing else. I realize your production snapshot root probably has additional stuff in it that you won't want to delete, but you could set up a separate one for testing. -- Matt

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: David Keegel <djk@ke...> - 2009-02-08 09:31 On Fri, Feb 06, 2009 at 12:17:11AM -0500, Matt McCutchen wrote: > On Mon, 2009-02-02 at 13:54 -0500, Matt McCutchen wrote: > > > > This is a bug in the test mode. Rsnapshot passes --link-dest on > > condition that the vious snapshot (in this case daily.1) exists. It > > fails to take into account that daily.1 would exist at that point had > > the mv actually been performed, so it omits the --link-dest option. > > The attached patch should fix this. Charles, please make sure it works > for you. Then, with David Keegel's approval, I will commit it. Matt, I'm happy to trust your judgement on this sort of thing. In the unlikely event that there turn out to be problems with it (or objections to it), it can always be reverted later. So you can treat that as approval. -- ___________________________________________________________________________ David Keegel <djk@cybersource.com.au> http://www.cyber.com.au/users/djk/ Cybersource P/L: Linux/Unix Systems Administration Consulting/Contracting

Re: [rsnapshot-discuss] No --link-dest with remote nfs filesystem on QNAP NAS From: Matt McCutchen <matt@ma...> - 2009-03-08 06:32 On Sun, 2009-02-08 at 20:31 +1100, David Keegel wrote: > On Fri, Feb 06, 2009 at 12:17:11AM -0500, Matt McCutchen wrote: > > On Mon, 2009-02-02 at 13:54 -0500, Matt McCutchen wrote: > > > > > > This is a bug in the test mode. Rsnapshot passes --link-dest on > > > condition that the vious snapshot (in this case daily.1) exists. It > > > fails to take into account that daily.1 would exist at that point had > > > the mv actually been performed, so it omits the --link-dest option. > > > > The attached patch should fix this. Charles, please make sure it works > > for you. Then, with David Keegel's approval, I will commit it. > > Matt, I'm happy to trust your judgement on this sort of thing. > In the unlikely event that there turn out to be problems with it > (or objections to it), it can always be reverted later. > > So you can treat that as approval. I have committed the fix. -- Matt

2 messages have been excluded from this view by a project administrator.