Re: [BackupPC-users] clustered file system and multiple servers

[dan <dan...@gm...>]

I have investigated cluster filesystems for backuppc.  You will have very
poor I/O on all cluster filesystems.  I/O is the most important factor for
backuppc and the performance difference between a local filesystem(or even
iscsi or aoe) and a cluster filesystem can be an order of magnitude slower
I/O performance.  I had hoped to use a cluster filesystem to maintain a
redundant copy of my backuppc data but performance was incredibly bad.  I
find fuse filesystems such as mysqlfs or zfs-fuse to be much faster.



On Mon, Aug 4, 2008 at 10:56 PM, Holger Parplies <wb...@pa...> wrote:

> Hi,
>
> sabujp wrote on 2008-08-04 23:18:41 -0400 [[BackupPC-users]  clustered file
> system and multiple servers]:
> >
> > Can a dev let me know if the files in the pool are FLOCK'd before
> writing,
>
> use the force, read the source. "grep -r flock backuppc-3.1.0" suggests
> that
> flock is used, but not on pool files. Not surprising, considering
> BackupPC_link
> and BackupPC_nightly (or two instances of BackupPC_link) may not run
> concurrently (BackupPC_link is responsible for entering new files into the
> pool).
>
> Come to think of it, the reason for this restriction is of a different
> nature:
> BackupPC_nightly sometimes needs to rename pool files (with a common
> BackupPC
> hash, when one or more files out of the chain are deleted), while
> BackupPC_link
> may insert a new file with the same BackupPC hash. You can't prevent the
> resulting race condition with flock() - at least you don't effectively
> change
> anything in the single-threaded case (you'd need a rather global lock).
>
> > i.e. is there a chance that two servers backing up into the same top
> level
> > data directory could "mangle" a file in the pool in this manner?
>
> I don't think you'd have mixed file contents (or, effectively, a corrupt
> compressed file), but there seems to be a chance of linking a file in a
> backup
> to the wrong pool file (wrong contents altogether).
>
> You could probably use flock() to prevent two instances of
> BackupPC_link/BackupPC_nightly running simultaneously on different servers,
> but there are more things you would want to think about (running
> BackupPC_nightly on more than one server does not make much sense, even if
> they don't run concurrently; limiting simultaneous backups over all
> servers;
> ensuring BackupPC_dump is not run twice simultaneously for the same host
> ...).
>
> In short: sharing a pool between servers is currently not supported.
>
> Regards,
> Holger
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> BackupPC-users mailing list
> Bac...@li...
> List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:    http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>