RemoteIndexingGuide

Note: This only applies if you have a network drive from linux via samba on your win32 machine that you want to backup from. Though it may also work if you're backing up from a remote win32 machine, this hasn't been tested. This guide is valid for AnyBackup 1.0.3 and above.

Further note: Version 0.9 introduced adding specific directories to a content drive, this will work with remote indexing but you have to make sure to add the _base remote directory. So if you add a directory from your network drive that corresponds to /shares/myShare/Music on your server (where /share/myShare is the actual samba share) you'd need to make sure to add /share/myShare as the remote path, AnyBackup is smart enough to pass the relative paths to the server indexing script. _

Let's face it, indexing a drive over Samba is painful. I have an 11tb volume (using Greyhole) which can take 20+ minutes to finish indexing! That's not a deal breaker considering I only backup once or twice a month, but it'd be nice to have things run faster!

With that in mind I've created a remote index feature for AnyBackup. At its most basic level it involves running a Python script (server.py) on your linux machine (along with a module called file.py) and then connecting to that server with your AnyBackup client on your windows machine. In my setup that sped up network indexing from 20+ minutes to ~3.5 minutes. Not bad, right? Interested? Read on.

Debian 7

1. If you're using Debian 7, good news, you can download the deb package under downloads as of version 1.0.3 of AnyBackup
2. The remote indexer will be installed as a service called anybackupserver
3. It sources it's config from /etc/anybackupserver.conf
4. It logs to /var/log/AnyBackup.log
5. Requires easy_install and python2.7
6. To install download the appropriate .deb file and run 'dpkg -i anybackupserver-version.deb' as superuser
7. If you don't have Pyro4 already, the install will attempt to grab it
8. Edit /etc/anybackupserver.conf to use the port you want to run the server on (6500 by default)

Getting the remote indexing server script running.

1. Copy server.py, server-generic.py and server.cfg from the repository to your server.
2. In that same server directory create a directory called 'backupManager'
3. Copy enum.py and mixin.py from the repository into 'backupManager'
4. Install Pyro3 on your server (you can get it from pypi -- if you have setup tools installed you can run 'easy_install pyro' and it should grab Pyro 3.14) As of 1.0.0 AnyBackup now uses Pyro4, follow the same steps but with Pyro4
5. Change server.cfg to the port you want to run on (defaults to 6500)
6. Execute server-generic.py -- './server-generic.py'
   • Note this script is written for Python 2.7 just like AnyBackup, so make sure you have a compatible version to run this on!

Remote indexing from AnyBackup

1. Start AnyBackup
2. Go to File -> Preferences
3. Change the Pyro port to whatever you set the server to run on (6500 by default)
4. Right-click on the content drive you want to remote index and select Remote Path
5. Enter the ip or hostname for the linux server
6. Enter the remote directory for the drive you're indexing on the linux server
   • Ex: if the remote share maps to /example/path then this is the path you should provide, even if you're only backing up sub directories for this share
7. Right-click on the content drive and hit refresh
8. When it asks if you'd like to use remote index, hit 'yes'
9. You should see output from the server-generic.py script saying that a request has been received, you should also see feedback in the AnyBackup indexing dialog showing the total number of files discovered

And that should be it! Some things to watch out for:

• If you have non-utf8 filenames in the remote directory, there will be some issues when remote indexing, I'm afraid the only option at present is to stick with indexing the slow way or to rename your files to non-utf8 names, the files will be ignored otherwise.
• If you have invalid modification times on any of your files, you'll need to 'touch' them to fix this.

Note: You may need to update the above scripts (server.py, server-generic.py, server.cfg, mixin.py and enum.py) when a new release comes out incase there have been any changes made. enum.py, mixin.py and init.py should be in a folder called backupManager relative to server.py, server-generic.py and server.cfg.

Version 1.0.4

• The latest version of the AnyBackup msi comes packaged with Pyro4 4.20 which uses a newer protocol. You may have issues if you're not running the same version on your server. To install this specific version you can run 'easy_install pyro4==4.20'

Happy indexing!