Hi Prem,

rsync only work best with static file.

Even though you mentioned that the file (table) is not being used, but usually MySQL always refreshing the file, maybe doing indexing or something.

That is why proprietary backup software have agent for MySQL to read the binary data, and dump is as static file for the backup.


On Thu, Dec 26, 2013 at 11:55 AM, Prem <squirrelj@yahoo.com> wrote:
Hi Sharuzzaman,

Thank you for the details but isnt rsync clever enough to dump all the files? The thing is the files are not being used hence there should be likely no activities.

As for the error below: I checked the troubleshooting page and its about the $Conf{ClientTimeout} settings, which is at default set to 72000 (around 20 hrs), I have changed to 200000 but have yet to try. I hope no other implications by doing so.
(aborted by signal=ALRM)

From: Sharuzzaman Ahmat Raslan <sharuzzaman@gmail.com>
To: Prem <squirrelj@yahoo.com>; "General list for user discussion, questions and support" <backuppc-users@lists.sourceforge.net>
Sent: Thursday, December 26, 2013 10:54 AM
Subject: Re: [BackupPC-users] Error when backing up 1.4TB files

Hi Prem,

MYD and MYI are files for MySQL. I believe the file is keep changing, thus BackupPC cannot determine that the size and the hash is correct, during and after the transfer.

If you really want to backup this binary database file, I would suggest that you create a static snapshot of the folder/mount point, maybe using LVM tools, provided that the file is inside an LVM logical volume.

Or, you could perform an daily dump of the MySQL table (which is a text file), then use BackupPC to backup this file. You should put exclusion for MYD and MYR file if use this method.


On Thu, Dec 26, 2013 at 9:17 AM, Prem <squirrelj@yahoo.com> wrote:

I am trying to backup a 1.4TB folder in Linux but somehow I am getting some errors and the backup only completes partially.

Appreciate any help on this:

Here are the errors:

attribSet: dir=f%2fmnt%2fdata/mysql/kbmatrix_snomed_10 exists
attribSet(dir=f%2fmnt%2fdata/mysql/kbmatrix_snomed_10, file=matrixipi5.MYD, size=4488979624, placeholder=)
Starting file 3429 (mysql/kbmatrix_snomed_10/matrixipi5.MYI), blkCnt=0, blkSize=2048, remainder=0
mysql/kbmatrix_snomed_10/matrixipi5.MYI: size doesn't match (12861890560 vs 0)
mysql/kbmatrix_snomed_10/matrixipi5.MYI got digests 6137db179868cdc52c7407c56fafaa5e vs 6137db179868cdc52c7407c56fafaa5e
[ skipped 1 lines ]
attribSet: dir=f%2fmnt%2fdata/mysql/kbmatrix_snomed_10 exists
attribSet(dir=f%2fmnt%2fdata/mysql/kbmatrix_snomed_10, file=matrixipi5.MYI, size=12861890560, placeholder=)
Starting file 3430 (mysql/kbmatrix_snomed_10/matrixipi5.frm), blkCnt=0, blkSize=2048, remainder=0
mysql/kbmatrix_snomed_10/matrixipi5.frm: size doesn't match (8782 vs 0)
mysql/kbmatrix_snomed_10/matrixipi5.frm got digests 8d7348ccff2108423f12fca06c39b994 vs 8d7348ccff2108423f12fca06c39b994
[ skipped 1 lines ]
attribSet: dir=f%2fmnt%2fdata/mysql/kbmatrix_snomed_10 exists
attribSet(dir=f%2fmnt%2fdata/mysql/kbmatrix_snomed_10, file=matrixipi5.frm, size=8782, placeholder=)
Starting file 3431 (mysql/kbmatrix_snomed_10/matrixipi6.MYD), blkCnt=0, blkSize=2048, remainder=0
mysql/kbmatrix_snomed_10/matrixipi6.MYD: size doesn't match (4488768644 vs 0)
mysql/kbmatrix_snomed_10/matrixipi6.MYD got digests bb526c0f4a40072b51c09ac9dec23fa8 vs bb526c0f4a40072b51c09ac9dec23fa8
[ skipped 1 lines ]
attribSet: dir=f%2fmnt%2fdata/mysql/kbmatrix_snomed_10 exists
attribSet(dir=f%2fmnt%2fdata/mysql/kbmatrix_snomed_10, file=matrixipi6.MYD, size=4488768644, placeholder=)
Starting file 3432 (mysql/kbmatrix_snomed_10/matrixipi6.MYI), blkCnt=0, blkSize=2048, remainder=0
mysql/kbmatrix_snomed_10/matrixipi6.MYI: size doesn't match (12866581504 vs 0)
Child is sending done
Got done from child
Got stats: 16821515 1936615698 1634885492 0 ('errorCnt' => 0,'ExistFileSize' => 0,'ExistFileCnt' => 31,'TotalFileCnt' => 3383,'ExistFileCompSize' => 0,'TotalFileSize' => '964645288896')
finish: removing in-process file mysql/kbmatrix_snomed_10/matrixipi6.MYI
attribWrite(dir=f%2fmnt%2fdata/mysql/kbmatrix_snomed_10) -> /var/lib/backuppc/pc/indexing-vm/new/f%2fmnt%2fdata/fmysql/fkbmatrix_snomed_10/attrib
attribWrite(dir=f%2fmnt%2fdata/mysql) -> /var/lib/backuppc/pc/indexing-vm/new/f%2fmnt%2fdata/fmysql/attrib
attribWrite(dir=f%2fmnt%2fdata) -> /var/lib/backuppc/pc/indexing-vm/new/f%2fmnt%2fdata/attrib
attribWrite(dir=) -> /var/lib/backuppc/pc/indexing-vm/new//attrib
Child is aborting
Got exit from child
Done: 3383 files, 964645288896 bytes
Executing DumpPostUserCmd: /usr/share/backuppc/bin/endbkpemail.sh prem.kumar 0 indexing-vm full DumpPostUserCmd
Got fatal error during xfer (aborted by signal=ALRM)
Backup aborted by user signal
Saving this as a partial backup, replacing the prior one (got 3383 and 3383 files versus 0)

Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
BackupPC-users mailing list
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Sharuzzaman Ahmat Raslan

Sharuzzaman Ahmat Raslan