From: Jim L. <tr...@ol...> - 2009-08-20 20:08:32
|
Les Mikesell wrote: > This probably isn't quite what you want but you can configure 4 different hosts, > each with one of the shares, and then use the $Conf{ClientNameAlias} option to > point three of them back to the real host. Normally you'd use this only if you > wanted different scheduling or options on the shares but it should permit > concurrency too. I'm doing this now, and while I can be confident that my last share might actually get backed up this time, I'm still measuring unacceptable transfer speeds. Jacob, could you make your specific rsync.exe + cygwin.dll available for testing? I have no experience compiling things in Windows and would most likely screw it up somehow. -- Jim Leonard (tr...@ol...) http://www.oldskool.org/ Help our electronic games project: http://www.mobygames.com/ Or check out some trippy MindCandy at http://www.mindcandydvd.com/ A child borne of the home computer wars: http://trixter.wordpress.com/ |
From: Jacob H. <jhy...@gm...> - 2009-08-21 02:03:32
|
> Jacob, could you make your specific rsync.exe + cygwin.dll available for > testing? I have no experience compiling things in Windows and would > most likely screw it up somehow. Here is a zip of the folder I use for each XP host: http://www.itfresno.com/exe/backuppc.zip Minus the secrets file... This is based on the article here: http://www.goodjobsucking.com/?p=62 A few things were modified on both the backuppc machine as well as the scripts, but only for my environment. Please keep in mind that simple file sharing has to be turned off... a snag which cost me a lot in trouble shooting (not a problem if you're on a domain, this client is 80+ units on peer to peer) If you're not using the VSS deal, just copy the rsync and cygwin files you need. As a side note, this is running on a CentOS 5.3 virtual machine using XenServer. I hope one day to be able to rsync the image of the machine periodically and reinstate it somehow. For now it's just regular image exports for backing up the pool/entire os. -- Jim Leonard (tr...@ol...) http://www.oldskool.org/ Help our electronic games project: http://www.mobygames.com/ Or check out some trippy MindCandy at http://www.mindcandydvd.com/ A child borne of the home computer wars: http://trixter.wordpress.com/ ---------------------------------------------------------------------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ BackupPC-users mailing list Bac...@li... List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ |
From: Jim L. <tr...@ol...> - 2009-08-21 02:45:08
|
Jacob Hydeman wrote: >> Jacob, could you make your specific rsync.exe + cygwin.dll available for >> testing? I have no experience compiling things in Windows and would >> most likely screw it up somehow. > > Here is a zip of the folder I use for each XP host: > http://www.itfresno.com/exe/backuppc.zip Thanks, this was indeed helpful. Performance was much better, so based on this I decided to try it with the actual cygwin 1.7 beta release. The last thing I used was deltacopy, so I copied the cygwin 1.7 binaries and libraries over into the deltacopy directory and that worked as well. Initial tests show 20-30MB/s instead of 5MB/s -- this still isn't anywhere near the performance the machine is capable of, but I am much happier. (For one thing, the cygwin rsync is 3.0.5 and so is the BackupPC server, so checksums and deltas are much more efficient now.) That improvement, coupled with the realization that I can treat each large area on a client as a separate "client" that can go simultaneously, makes this more than viable. If anyone wants to try the exact deltacopy/cygwin binaries I'm using, I put them online at ftp://ftp.oldskool.org/pub/misc/DeltaCopy.rar. > Please keep in mind that simple file sharing has to be turned off... a snag > which cost me a lot in trouble shooting (not a problem if you're on a > domain, this client is 80+ units on peer to peer) I didn't know about this limitation, but thankfully have it off on all of my machines since I attempted smb sharing first. What snags did you run into? -- Jim Leonard (tr...@ol...) http://www.oldskool.org/ Help our electronic games project: http://www.mobygames.com/ Or check out some trippy MindCandy at http://www.mindcandydvd.com/ A child borne of the home computer wars: http://trixter.wordpress.com/ |
From: Jacob H. <jhy...@gm...> - 2009-08-21 05:56:20
|
>> Please keep in mind that simple file sharing has to be turned off... a snag >> which cost me a lot in trouble shooting (not a problem if you're on a >> domain, this client is 80+ units on peer to peer) > I didn't know about this limitation, but thankfully have it off on all > of my machines since I attempted smb sharing first. What snags did you > run into? The simple file sharing issue is only a problem if you're using the scripts to enable VSS, more specifically are using winexe to launch stuff. The other common configuration problem was that file/print sharing wasn't enabled, which prevents winexe from working. There were some Win2k hosts as well (which wont work w/ VSS) and for simplicities sake I didn't want to override anything in the backuppc control menu. I had to modify the scripts somewhat and add a kill program to the backuppc folder. But this isn't an rsync/cygwin issue; it only pertains to running VSS via winexe. -- Jim Leonard (tr...@ol...) http://www.oldskool.org/ Help our electronic games project: http://www.mobygames.com/ Or check out some trippy MindCandy at http://www.mindcandydvd.com/ A child borne of the home computer wars: http://trixter.wordpress.com/ ---------------------------------------------------------------------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ BackupPC-users mailing list Bac...@li... List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ |
From: Michael S. <ms...@ch...> - 2009-08-28 14:45:24
|
After downloading this and trying it out, I noticed that these errors went away: Remote[1]: rsync: readlink "..." (in C) failed: File name too long (91) So whatever version of rsync/cygwin that is, is well worth running for that reason alone. With that, I now have a complete copy of 100% of the files on a box, and have done a few successful bare metal restores. I've also had to pluck a few registry keys out of backups, and am assured that I always have the last sane registry. >> Jacob, could you make your specific rsync.exe + cygwin.dll available for >> testing? I have no experience compiling things in Windows and would >> most likely screw it up somehow. > > Here is a zip of the folder I use for each XP host: > http://www.itfresno.com/exe/backuppc.zip > > Minus the secrets file... > > This is based on the article here: > http://www.goodjobsucking.com/?p=62 > > A few things were modified on both the backuppc machine as well as the > scripts, but only for my environment. > > Please keep in mind that simple file sharing has to be turned off... a > snag > which cost me a lot in trouble shooting (not a problem if you're on a > domain, this client is 80+ units on peer to peer) > > If you're not using the VSS deal, just copy the rsync and cygwin files you > need. > > As a side note, this is running on a CentOS 5.3 virtual machine using > XenServer. I hope one day to be able to rsync the image of the machine > periodically and reinstate it somehow. For now it's just regular image > exports for backing up the pool/entire os. |
From: Jeffrey J. K. <bac...@ko...> - 2009-08-30 02:40:24
|
Jacob Hydeman wrote at about 18:28:54 -0700 on Thursday, August 20, 2009: > > > Jacob, could you make your specific rsync.exe + cygwin.dll available for > > testing? I have no experience compiling things in Windows and would > > most likely screw it up somehow. > > Here is a zip of the folder I use for each XP host: > http://www.itfresno.com/exe/backuppc.zip > > Minus the secrets file... > > This is based on the article here: > http://www.goodjobsucking.com/?p=62 > > A few things were modified on both the backuppc machine as well as the > scripts, but only for my environment. > > Please keep in mind that simple file sharing has to be turned off... a snag > which cost me a lot in trouble shooting (not a problem if you're on a > domain, this client is 80+ units on peer to peer) Why does simple file sharing need to be turned off? If I understand that correctly, it would be a significant limitation on XP Home systems where that is the only type of file sharing that exists. |
From: Jacob H. <jhy...@gm...> - 2009-09-01 04:33:07
|
-----Original Message----- From: Jeffrey J. Kosowsky [mailto:bac...@ko...] Sent: Saturday, August 29, 2009 7:40 PM To: General list for user discussion, questions and support Subject: Re: [BackupPC-users] BackupPC File::RsyncP issues Jacob Hydeman wrote at about 18:28:54 -0700 on Thursday, August 20, 2009: > > > Jacob, could you make your specific rsync.exe + cygwin.dll available for > > testing? I have no experience compiling things in Windows and would > > most likely screw it up somehow. > > Here is a zip of the folder I use for each XP host: > http://www.itfresno.com/exe/backuppc.zip > > Minus the secrets file... > > This is based on the article here: > http://www.goodjobsucking.com/?p=62 > > A few things were modified on both the backuppc machine as well as the > scripts, but only for my environment. > > Please keep in mind that simple file sharing has to be turned off... a snag > which cost me a lot in trouble shooting (not a problem if you're on a > domain, this client is 80+ units on peer to peer) > Why does simple file sharing need to be turned off? > If I understand that correctly, it would be a significant limitation > on XP Home systems where that is the only type of file sharing that exists. The simple file sharing is a requirement of the winexe service. You can read more about it here: http://eol.ovh.org/winexe/ I am looking into the XP Home issue, and some suggest that you can change permissions and such if you boot into safe mode under the admin account, more specifically simple file sharing is "disabled". I won't try it any time soon, but was thinking of making a C$ share in safemode and adding the admin as a user. Jacob ---------------------------------------------------------------------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ BackupPC-users mailing list Bac...@li... List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ |
From: Jeffrey J. K. <bac...@ko...> - 2009-08-28 15:05:45
|
Jim Leonard wrote at about 19:45:57 -0500 on Tuesday, August 18, 2009: > Holger Parplies wrote: > > first of all, where are you seeing these figures, and what are you > measuring? > > Rather than try to convince you of my competence, I will offer up these > benchmarks for the exact same endpoint machines and file (a 2 gigabyte > uncompressable *.avi file that did NOT exist on the target): > > Unix rsync->Unix rsync: 60MB/s > Windows SMB->Unix smbclient: 65MB/s > Windows rsyncd->Unix rsync: 5MB/s > Windows rsyncd->BackupPC_dump: 5MB/s > > As you can see, something is now clearly wrong with the windows rsyncd > source. I confirmed this by profiling actual rsync in Unix and saw that > 77% of its time was spent waiting for data (which mirrors exactly what > File::RsyncP::pollsys was doing, wasting 77% of its time waiting for > data). So the problem isn't BackupPC, it's windows rsyncd. > > I initially used cygwin rsync; for the above test, I switched it out for > DeltaCopy's rsync. BOTH VERSIONS had this kind of crappy speed. Both > versions showed hardly any CPU or filesystem usage; they just simply run > slowly for a reason I can't figure out. The network isn't slow (gigabit > ethernet), the checksums aren't taking a long time (it's a brand new > file that doesn't exist on the target so there's nothing to checksum), > the hard drive isn't slow (raid-0 SATA stripe capable of 130GB/s read > speeds) -- it just simply serves data really really slowly. > I believe that I have noticed similar behavior with rsyncd processes throttled down to near 0% cpu on Windoze machines and backups taking similarly long. > I can't believe this is an isolated incident. Other people have got to > be seeing this. Other than cygwin and DeltaCopy, is there any specific > version of rsyncd I should be using? Any flags I can set in BackupPC > that can improve speed? > > > The primary purpose of the rsync protocol is to save network > bandwidth. So if, > > for example, you are transferring only one tenth the amount of data > for a full > > backup, and that takes the same time as with SMB, your network > throughput will > > These are not incrementals, but full backups, and the speed as > previously mentioned is 1/10th that of SMB. SMB backups are quite fast > on this same infrastructure (around 65MB/s) but I can't use SMB because > of XP/Vista/Win7 permission problems. > > > I believe Craig is researching other alternatives (a fuse FS to handle > > compression and deduplication, so BackupPC could, in fact, use native > rsync). > > I hope that doesn't become mandatory, because that would limit BackupPC > to Unix versions that support FUSE (not all do). > -- > Jim Leonard (tr...@ol...) http://www.oldskool.org/ > Help our electronic games project: http://www.mobygames.com/ > Or check out some trippy MindCandy at http://www.mindcandydvd.com/ > A child borne of the home computer wars: http://trixter.wordpress.com/ > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > BackupPC-users mailing list > Bac...@li... > List: https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki: http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > |
From: Jeffrey J. K. <bac...@ko...> - 2009-09-01 14:42:52
|
Jacob Hydeman wrote at about 21:31:18 -0700 on Monday, August 31, 2009: > The simple file sharing is a requirement of the winexe service. You can read > more about it here: http://eol.ovh.org/winexe/ > > I am looking into the XP Home issue, and some suggest that you can change > permissions and such if you boot into safe mode under the admin account, > more specifically simple file sharing is "disabled". I won't try it any time > soon, but was thinking of making a C$ share in safemode and adding the admin > as a user. > The homepage says: Standard Windows installation with enabled remote sharing and administration (Windows XP Home do not support remote administration) I am not a Windoze expert, but would this be a show-stopper for XP Home or are you suggesting above that there is a way around it? |
From: dan <dan...@gm...> - 2009-09-03 02:05:29
|
I might point out that if you rsync between two windows machine using the same version of cwrsync or deltacopy(which is basically still cwrsync but a different version) the transfer speeds are higher but not anywhere near a smb transfer. One solution might be to create a shadow copy, then explorer using net use to share that copy and have backuppc hit the shadow copy over smb. you will loose the bennies of rsync but if you are saving 50% at the expense of 75% then you have a net loss. I have given in and worked around this issue in a different way. I use cobian9 and pull a backup and send it up to backuppc via FTP. I then have a script that syncs the directory over my base directory as a pre-backup script and backup that folder (then delete the backup from cobian). It uses extra space on the server but solved all of my issues. cobian does VSS backups and incrementals. rsync does the heavy lifting on the server side and it works well with the incremental filesets. it was not planned but I have the files from cobian on a different disk than the backupc backups so the performance in backuppc is pretty good. This does not handle ACL or permissions but for laptop users they dont need any of that anyway, its just their Docs&Settings folder. |