From: Alex H. <al...@lo...> - 2008-09-05 06:52:37
|
Hi This is my first post to the developers list. I've been a happy BackupPC user for some time. I recently emailed Craig re: a FUSE implementation for BackupPC which I've coded in isolation. It's pretty hacky but functional for my needs at present - basically to about the same level as Stephen's v3 code is (no symlink support at present). I'd be interested in offering that code to form part of an official FUSE implementation, and/or helping out with that project. Best wishes Alex -- Alex Harrington - Network Development Manager Longhill High School t: 01273 391672 e: al...@lo... |
From: Alex H. <al...@lo...> - 2008-09-08 17:07:10
|
>> It's in DEBUG mode so it will send lots of output to that console. > No output when I issue the ls while the script is active. Your other console will need to be logged in as backuppc user too. At the moment, it's set to run in single user mode if DEBUG is enabled. I've not tested running it in such a way that other users can access it - although the code is in there to do it in theory. Also in DEBUG mode, everything will be owned backuppc.backuppc - so that things are accessible. Again, the code is there to make things owned correctly when not running in DEBUG - but it is, to date, untested. I'll try and get some time together and have a bash at that here tonight. Cheers Alex -- Alex Harrington - Network Manager Longhill High School t: 01273 304086 e: al...@lo... -----Original Message----- From: Kenneth Porter [mailto:sh...@se...] Sent: Mon 08/09/2008 17:55 To: Developers discussion Cc: Alex Harrington Subject: RE: [BackupPC-devel] FUSE / BackupPC --On Monday, September 08, 2008 5:23 PM +0100 Alex Harrington <al...@lo...> wrote: > What do you get if you open a second terminal and ls your mountpoint? Before and after running the script: [root@segw2 tmp]# ls -al /mnt/BackupPC total 16 drwxr-xr-x 2 backuppc backuppc 4096 Sep 8 09:52 . drwxr-xr-x 8 root root 4096 Sep 8 09:52 .. [root@segw2 tmp]# ls -al /mnt/BackupPC ls: /mnt/BackupPC: Permission denied > It's in DEBUG mode so it will send lots of output to that console. No output when I issue the ls while the script is active. SELinux is Permissive to avoid interference from it. |
From: Kenneth P. <sh...@se...> - 2008-09-08 18:12:20
|
--On Monday, September 08, 2008 5:59 PM +0100 Alex Harrington <al...@lo...> wrote: > Your other console will need to be logged in as backuppc user too. Aha! Doing that, I can navigate down into the backup share! As you say, the terminal running the script is noisy. (Understandably so.) But proof of concept is there. |
From: Kenneth P. <sh...@se...> - 2009-01-29 01:18:58
|
I saw discussion about this on the users list and hoped to reawaken something here. I just popped up Alex Harrington's implementatation and was able to navigate directories, but the first file I inspected seemed to contain just nulls, so I think the data part of the implementation is stubbed. How hard would it be to flesh that out? That would be enough to do something like: diff -rq /mnt/BackupPC/balrog/2361/D /mnt/Balrog-D (Where Balrog is my Windows host, 2361 is a backup archive number, and D is the share being compared.) What I'd like to do is script the above to run nightly so I can see if there are any trouble spots in the backup. |
From: Jon C. <can...@gm...> - 2009-01-29 02:12:06
|
The data portion will not be trivial, especially if the you are using a compressed pool. The problem lies in the fact that IO is experienced as a series of reads of a given size at a given offset. This means the Fuse driver will receive a series of calls like: read("FILEA", 0 , 8192) read("FILEA", 8192, 8192) read("FILEA",16384,8192) etc. Now the problem is reading the data via the BackupPC API gives you a file handle to the data stream beginning at offset 0. So, if you try to design it stateless (which is the best in terms of resource usage for meta data and overall stability) you will have to open a fresh handle to the data, seek to the position, and then read the requested block of data. This will be very slow and wasteful of resources setting up the handle on each IO. This is true even though you would generally have IO that either read just the head of the file (such as a file typing request like "file FILEA") or read the file from beginning to end. If you make the driver stateful you could cache an open file handle that would service the sequential reading well but brings in headaches like how long to cache before allowing cache to timeout and cleanup. The documentation surrounding the open, read, close calls that are made aren't real clear and you could have multiple overlapping open and closes for any given file (this supports different processes having independent file handles for a given file). All of this begs the question of whether its better to fit a FUSE interface to the current pool/catalog functionality or look at rewriting the whole pool / catalog functionality with deduping written directly into the FUSE filesystem. This would then allow BackupPC to simply write its file to the filesystem and have the filesystem handle all of the magic of deduping / compressing. Jon On Wed, Jan 28, 2009 at 8:18 PM, Kenneth Porter <sh...@se...> wrote: > I saw discussion about this on the users list and hoped to reawaken > something here. I just popped up Alex Harrington's implementatation and was > able to navigate directories, but the first file I inspected seemed to > contain just nulls, so I think the data part of the implementation is > stubbed. How hard would it be to flesh that out? That would be enough to do > something like: > > diff -rq /mnt/BackupPC/balrog/2361/D /mnt/Balrog-D > > (Where Balrog is my Windows host, 2361 is a backup archive number, and D is > the share being compared.) > > What I'd like to do is script the above to run nightly so I can see if > there are any trouble spots in the backup. > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by: > SourcForge Community > SourceForge wants to tell your story. > http://p.sf.net/sfu/sf-spreadtheword > _______________________________________________ > BackupPC-devel mailing list > Bac...@li... > List: https://lists.sourceforge.net/lists/listinfo/backuppc-devel > Wiki: http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > -- Jonathan Craig |
From: Kenneth P. <sh...@se...> - 2008-09-05 23:29:25
|
On Friday, September 05, 2008 7:48 AM +0100 Alex Harrington <al...@lo...> wrote: > I recently emailed Craig re: a FUSE implementation for BackupPC which > I've coded in isolation. It's pretty hacky but functional for my needs > at present - basically to about the same level as Stephen's v3 code is > (no symlink support at present). > > I'd be interested in offering that code to form part of an official FUSE > implementation, and/or helping out with that project. I'd be interested in trying it out. |
From: Alex H. <al...@lo...> - 2008-09-06 11:57:43
|
> I'd be interested in trying it out. I've put the latest version I have here: https://learning.longhill.org.uk/index.phtml?d=34330 I'm using BackupPC from the Ubuntu package for v3.0.0 and have only tested with that at present. If you're not using the Ubuntu install paths, you'll need to edit the libs line to point to your BackupPC libs directory. Make the downloaded file exectuable and as your BackupPC user call ./BackupPCfs mountpoint There's a bug I've just found where attributes are not being returned for tar-based backups. Rsync and SMB backups seem to be OK for me though... Cheers Alex -- Alex Harrington - Network Manager Longhill High School t: 01273 304086 e: al...@lo... |
From: Kenneth P. <sh...@se...> - 2008-09-08 16:02:08
|
--On Saturday, September 06, 2008 12:53 PM +0100 Alex Harrington <al...@lo...> wrote: > I'm using BackupPC from the Ubuntu package for v3.0.0 and have only > tested with that at present. If you're not using the Ubuntu install > paths, you'll need to edit the libs line to point to your BackupPC libs > directory. Using 3.1.0, and tweaking the use lib line to match my CentOS 5 package. I needed to run the script as the backuppc user, and I needed to add that user to the fuse group so it could access /dev/fuse. I ran the script with the output below. As root from another shell I got permission denied trying to list /tmp/BackupPC (the mount point). The mount point is owned by the backuppc user. I exit the script with control-C, which does not umount the filesystem (it still shows up in /etc/mtab), so I have to do that separately. bash-3.2$ ./BackupPCfs /tmp/BackupPC Statement unlikely to be reached at /usr/lib/BackupPC/Lib.pm line 1273. (Maybe you meant system() when you said exec()?) Statement unlikely to be reached at /usr/lib/BackupPC/Lib.pm line 1338. (Maybe you meant system() when you said exec()?) Subroutine BackupPC::FileZIO::Z_OK redefined at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/Compress/Zlib.pm line 1 Subroutine BackupPC::FileZIO::Z_STREAM_END redefined at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/Compress/Zlib.pm line 1 Ambiguous call resolved as CORE::open(), qualify as such or use & at /usr/lib/BackupPC/FileZIO.pm line 408. Ambiguous call resolved as CORE::close(), qualify as such or use & at /usr/lib/BackupPC/FileZIO.pm line 413. BackupPCfs 0.1 - 2008 (c) by Alex Harrington <fas...@gm...> Use of uninitialized value in string eq at /usr/lib/BackupPC/Lib.pm line 118. Use of uninitialized value in string eq at /usr/lib/BackupPC/Lib.pm line 119. Use of uninitialized value in string eq at /usr/lib/BackupPC/Lib.pm line 127. Use of uninitialized value in string ne at /usr/lib/BackupPC/Lib.pm line 379. Useless use of a constant in void context at /usr/lib/BackupPC/Lang/en.pm line 43. Useless use of a constant in void context at /usr/lib/BackupPC/Lang/en.pm line 43. Running against BackupPC version 3.1.0 BackupPC storage is in /var/lib/backuppc unique: 1, opcode: INIT (26), nodeid: 0, insize: 56 INIT: 7.8 flags=0x00000003 max_readahead=0x00020000 INIT: 7.8 flags=0x00000000 max_readahead=0x00020000 max_write=0x00020000 unique: 1, error: 0 (Success), outsize: 40 |
From: Alex H. <al...@lo...> - 2008-09-08 16:27:30
|
> BackupPC storage is in /var/lib/backuppc > unique: 1, opcode: INIT (26), nodeid: 0, insize: 56 > INIT: 7.8 > flags=0x00000003 > max_readahead=0x00020000 > INIT: 7.8 > flags=0x00000000 > max_readahead=0x00020000 > max_write=0x00020000 > unique: 1, error: 0 (Success), outsize: 40 > That looks about right. It should be functional at that point. What do you get if you open a second terminal and ls your mountpoint? It's in DEBUG mode so it will send lots of output to that console. Cheers Alex |
From: Kenneth P. <sh...@se...> - 2008-09-08 16:55:53
|
--On Monday, September 08, 2008 5:23 PM +0100 Alex Harrington <al...@lo...> wrote: > What do you get if you open a second terminal and ls your mountpoint? Before and after running the script: [root@segw2 tmp]# ls -al /mnt/BackupPC total 16 drwxr-xr-x 2 backuppc backuppc 4096 Sep 8 09:52 . drwxr-xr-x 8 root root 4096 Sep 8 09:52 .. [root@segw2 tmp]# ls -al /mnt/BackupPC ls: /mnt/BackupPC: Permission denied > It's in DEBUG mode so it will send lots of output to that console. No output when I issue the ls while the script is active. SELinux is Permissive to avoid interference from it. |