Re: Re: [BackupPC-users] replicating backup servers offsite

[<gh_...@be...>]

Craig's proposed solution is very interesting, and it's kind of exciting to see a possiblity for replication on the horizon.  But I have a question that I hope somebody can answer.  How do you tar up a file as a hardlink reference to a pool file already copied to the replication host without actually including the pool file in the tar archive?  As I understand it, tar only follows the hardlinks for files included in the archive.
- gail
> 
> From: Craig Barratt <cba...@us...>
> Date: 2005/03/14 Mon AM 06:40:44 EST
> To: Carl Wilhelm Soderstrom <ch...@re...>
> CC: bac...@li...
> Subject: Re: [BackupPC-users] replicating backup servers offsite
> 
> Carl Wilhelm Soderstrom writes:
> 
> > On 03/03 10:20 , Les Mikesell wrote:
> It wouldn't be too hard to write a script that copies the BackupPC
> store without using too much memory.  But it might be pretty slow:
> 
>   - Copy pool, cpool, conf and log using any method (cp, rsync, tar
>     etc).  Don't worry about hardlinks.
> 
>   - Write a perl script that generates a tar file for the pc directory.
>     Run this script and pipe the output into tar -xf - on the target
>     machine or file system.
> 
> The script exploits the known structure of the hardlinks.
> 
> For any file below pc, the script would do the following:
> 
>   - if it has only 1 link then send the file as normal
>     (this would only apply to the top-level files like
>     the XferLOGs etc that are not hardlinked).
> 
>   - if it has more than 1 link then compute the MD5 checksum
>     (involves uncompressing and reading the first 1MB of the file)
>     and then match the file in the pool by verifying the inodes match.
>     Then send the file as a tar "hardlink".  The tar at the other end
>     will do the right thing, creating a hardlink to the existing
>     pool or cpool file.
> 
>   - to speed things up the script could cache the inode --> checksum
>     data if there are more than 2 links.  The cache entry could be
>     removed once that number of links (minus 1) had been seen.  To
>     save memory the cache could be emptied for each host below the
>     pc directory, since most hard links are local to a single host.
> 
> The trick is that this script can take advantage of knowing where
> the hardlinks point to, and can compute this on-the-fly.  In contrast,
> cp -a and rsync -H aren't so lucky, so they need lots of memory to
> store the reverse inode -> path mapping.
> 
> Volunteers anyone?
> 
> Craig
> 
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> BackupPC-users mailing list
> Bac...@li...
> https://lists.sourceforge.net/lists/listinfo/backuppc-users
> http://backuppc.sourceforge.net/
>