From: Les M. <le...@fu...> - 2007-03-27 16:49:04
|
Evren Yurtesen wrote: >> What is wall clock time for a run and is it >> reasonable for having to read through both the client and server copies? > > I am using rsync but the problem is that it still has to go through a > lot of hard links to figure out if files should be backed up or not. From the perspective of the backup directories, it doesn't matter whether or not there are additional links to a file. It just looks at the directory entry to find the file it wants. It may matter that the inodes and file contents end up splattered all over the place because they were written at different times, though. >>> But I have to disagree, again :) When you make writes and filesystem >>> is mounted with 'sync' option which is default on FreeBSD then all >>> operations are done sync. Your program must have been doing something >>> between truncate and the actual write operation which caused file to >>> be empty for a period of time. >> Yes, it was writing a small amount to the file and the data write was >> always deferred with this kind of problem: >> http://www.complang.tuwien.ac.at/anton/sync-metadata-updates.html > > You shouldnt be trusting this article. The article is flawed in certain > ways. The fact is that 90+% of the time, my data files were empty after a crash while they always appeared to have data when running. That tells me that the truncate happened on disk immediately while the data write happened when a buffer was flushed which was consistent with the way the bsd sync-metadata was described. > For example it says "as demonstrated by soft updates (see below), which > don't even require fsck upon crashing." but this information is wrong. > Softupdates means that you can run the fsck in background but you must > run it on a crash (system does it automatically anyhow). This was before softupdates was an option. I don't think reiserfs was around then either. > Also, about the problem in this article, if async updates would have > been used then the file would have been lost anyhow as the writes would > have been buffered. But then you'd have about a 99% chance of still having the previous contents which would have been much more useful than an empty file. > In either case you could just enable sync or async updates on UFS to > change this behaviour. You didnt have to change the filesystem because > of a deficiency in UFS which is causing the problem. I just got tired of the freebsd people telling me that their way was better and it was correct behavior to lose my data and moved on. >>> The only difference is that FreeBSD uses sync by default because they >>> claim(there is no point to argue if it is true or not, there are >>> enough threads about those if you google :) ) that it is safer while >>> Linux uses async. >> It is safer for the file system if you are concerned that fsck can't fix >> it. It is not safer for the data of any particular file. Anyway >> crashes are pretty much a thing of the past and I protect against most >> kinds of failures by raid-mirroring to an external drive that is rotated >> off-site weekly. I'm running a journaled reiserfs because it is very >> fast at creating and deleting files, but ext3 with the right options >> should also work. > > If you check namesys.com benchmarks, you will see that they only tested > reiserfs against ext2/ext3/xfs/jfs and conveniently forgot to test > against ufs2. > > You can see in the end of the page that slight performance increase in > reiserfs is also bringing twice the cpu usage! (plus extX is faster in > certain operations even) > http://www.namesys.com/benchmarks.html There's a benchmark called 'postmark' that is the best I've seen for showing how well creating/deleting a lot of files works. A few years ago when I changed last, reiserfs was the best. It seems to have disappeared from it's original location from NetApp, but the last time I mentioned it someone said it was packaged for Debian so the source is still available. I'll dig it up again the next time I am interested in changing (maybe when zfs looks mature). >>>>>>> Perhaps, but there is a difference if they are moving 10 times or >>>>>>> 100000 times. > > It is a logical fact that more movement = more wear. Yes, but more wear doesn't matter until you exceed the design limits. Worry about your air conditioner instead - it has more effect on the drive life. I've had as much trouble with drives that have been powered off for much of their lives as ones that stay busy. -- Les Mikesell les...@gm... |