Re: [BackupPC-users] server maintenance: reconstruct missing poolCnt; find/delete references to missing pool files

[Craig B. <cba...@us...>]

>
> > BackupPC_refCountUpdate: doing fsck on <host> #1188 since there are no
> poolCnt files
> > BackupPC_refCountUpdate: doing fsck on <host> #1190 since there are no
> poolCnt files
> > ...
> > BackupPC_refCountUpdate: host <host> got 0 errors (took 5 secs)
> The backups in question seem to be fully intact; some are full backups,
> some are incremental. It's just on a minority of backups (appx. 15 out of
> 350 backups), and fortunately on small ones where fsck does not take ages,
> so it does not bother me too much. Nevertheless, can the missing poolCnt
> data be recomputed? fsck seems to do the counting from scratch; can this be
> stored?


This is perfectly ok.  BackupPC 4.0.0alpha3 and prior 4.x versions didn't
store reference counts per backup.  Only the reference counts for the
entire host were maintained (in addition to the pool totals for all
hosts).  In 4.0.0, I changed that so reference counts were also stored
per-backup (which makes it easier to delete backups and to recompute the
per-host ref counts).  So BackupPC_refCountUpdate is simply adding
reference counts to backups done by BackupPC 4.0.0alpha3.  It's a one-time
thing.

There might be an issue that an incremental done by BackupPC 4.0.0alpha3
with no changes will have an empty backup tree, and BackupPC_refCntUpdate
will continually report that there are no poolCnt files for that backup.
That's benign.  In 4.0.0, BackupPC_dump flags that by creating a file
"HOST/NNN/refCnt/noPoolCntOk, which makes BackupPC_refCntUpdate quietly
ignore that backup.  Perhaps I should have BackupPC_refCntUpdate notice
that legacy case and create the noPoolCntOk file...

> BackupPC_refCountUpdate: missing pool file
> 00000000000000000000000000000000 count 30
> > BackupPC_refCountUpdate: missing pool file
> 0601e1b90a7f92ce4cffa588ef2cc9da count 1
> > ...
> > BackupPC_refCountUpdate: missing pool file
> ea1bd7ab2e0000000000000000000000 count 1


This is a bug in rsync-bpc (and BackupPC::XS) that was fixed a couple of
weeks ago.  It happened about 2% of the time the attrib file for a large
directory was written (attrib file sizes >256k, approx 5k files depending
on file name lengths).  If the md5 digest of the last file written to the
256k staging buffer exactly lined up with the end of the buffer, the digest
for that file wasn't written correctly (yes, I had "<" instead of "<="...
doh!).

Future backups with 4.0.0 (assuming the same file exists on the client)
will be updated with the correct digest, but the old backups will still
have the wrong one.  The errors will go away when the corresponding backups
eventually expire.

Craig

On Tue, Mar 7, 2017 at 6:56 AM, Alexander Kobel <a-...@a-...> wrote:

> Dear all,
>
> I have a rather small, private, non-$$$M-mission-critical instance of a
> BackupPC server running for years (that went from several 3.something
> through 4.0.0alpha3 to, recently, 4.0.0).
> After the 4.0.0 migration, I decided to run again some semi-manual
> maintenance (read: fsck and refCountUpdate). Yet, after an (admitted, more
> or less random) sequence of
>   BackupPC_fsck
>   BackupPC_fsck -f
>   BackupPC_fsck -f -s
>   BackupPC_refCountUpdate -m
>   BackupPC_refCountUpdate -m -F -c
>   BackupPC_refCountUpdate -m -F -c -s
>   BackupPC_fixupBackupSummary
>   BackupPC_nightly 0 255
> I'm still stuck with the following messages:
>
> > BackupPC_refCountUpdate: doing fsck on <host> #1188 since there are no
> poolCnt files
> > BackupPC_refCountUpdate: doing fsck on <host> #1190 since there are no
> poolCnt files
> > ...
> > BackupPC_refCountUpdate: host <host> got 0 errors (took 5 secs)
>
> The backups in question seem to be fully intact; some are full backups,
> some are incremental. It's just on a minority of backups (appx. 15 out of
> 350 backups), and fortunately on small ones where fsck does not take ages,
> so it does not bother me too much. Nevertheless, can the missing poolCnt
> data be recomputed? fsck seems to do the counting from scratch; can this be
> stored?
>
> > BackupPC_fsck: building main count database
> > BackupPC_refCountUpdate: missing pool file 00000000000000000000000000000000
> count 30
> > BackupPC_refCountUpdate: missing pool file 0601e1b90a7f92ce4cffa588ef2cc9da
> count 1
> > ...
> > BackupPC_refCountUpdate: missing pool file ea1bd7ab2e0000000000000000000000
> count 1
> > ...
> > BackupPC_refCountUpdate total errors: 70
> > BackupPC_fsck: Calling poolCountUpdate
>
> IIUC, this means that there are reference to files with that hash
> *somewhere* in the backups, but the respective files are missing from the
> cpool. Since the number is very low, I'm not really worried; in particular,
> I'm pretty sure that the pool file with hash 0*32 is a spurious one.
> Nevertheless, I'd like to see which filenames/entries refer to the missing
> pool files, and if possible delete those altogether to get a "clean" state
> without nagging messages. Unfortunately, I'm entirely clueless on how to
> approach that.
>
> FWIW: the server runs on CentOS 7.3, on an ext3 filesystem, with (manually
> installed) BackupPC 4.0.0, compression activated since the dawn of time.
>
> I'd be grateful for any hints to attack any of the above inconveniences.
> Thanks!
>
>
> Best,
> Alexander
>
>
> ------------------------------------------------------------
> ------------------
> Announcing the Oxford Dictionaries API! The API offers world-renowned
> dictionary content that is easy and intuitive to access. Sign up for an
> account today to start using our lexical data to power your apps and
> projects. Get started today and enter our developer competition.
> http://sdm.link/oxford
> _______________________________________________
> BackupPC-users mailing list
> Bac...@li...
> List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:    http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
>