From: Paul F. <pg...@fo...> - 2004-06-09 17:04:11
|
> > > Reiserfs builds balanced trees for the directory entries > > > so they scale up nicely. The directories aren't the problem > > > > does this balanced tree implementation you refer to apply both > > on-disk and in-memory. i confess to not having looked at the > > linux filesystem code -- does each fs supply their own in-core > > directory access routine? > > I belive in memory the inodes are stored in a radix tree (which is *very* > fast); but I could be very far off base here. but that's different data than the directory entries. paul =--------------------- paul fox, pg...@fo... (arlington, ma, where it's 86.4 degrees) |
From: Craig B. <cba...@us...> - 2004-06-29 07:58:26
|
Paul Fox writes: > in looking at the pool on our backup machine yesterday, we got to > wondering... currently the pool has paths like: > /big/backuppc/pool/1/2/3/123456789abcdef0 > > in our case, this leads to leaf directories which are all about > 50K large, mostly containing about 1000 files each. since the > final phase of a path lookup is always a linear search > (correct?), mightn't performance be improved by making one or all > of the intermediate path components "wider", i.e., something > like: > /big/backuppc/pool/1/2/34/123456789abcdef0 > unless i'm missing something, just doing this would reduce the > leaf directories to more like 3K or 4K bytes, with fewer than a > hundred entries to search. > > i'd think that on a ram-limited system this would reduce disk > activity, at the same time that it would reduce cpu time spent > searching the last directory (on any system). i don't claim to > be an expert, but wouldn't better balance to the tree be better? Obviously it depends upon the underlying file system. The change you describe is easy to make (see the function MD52Path() in BackupPC::Lib). BackupPC_nightly will need some changes too. However, you will need to do it on a clean install with an empty pool. If you try the experiment I would be curious about whether you see any significant performance difference. Craig |