[BackupPC-devel] Separating frequently-accessed LOCK and LOG* from $topDir/pc/$host

[Alexander K. <a-...@a-...>]

Hi,

for a tiny BackupPC instance with on an SSD+HDD-server, I want the HDD 
to be in standby mode (spinned down) most of the time.  Hence, I only 
store __TOPDIR__ on it, and have everything else on the SSD. This almost 
works, except for the fact that unsuccessful pings write a log entry in 
__TOPDIR__/pc/$host/LOG.DDMMYYYY, and LOCK is touched.  Which happens on 
every wakeup when some client is not reachable; in other words, too often.

AFAICS, from what's inside pc/, actually vital for the integrity of the 
backups are the nnn and refCnt dirs (and, to some extent, the backups 
and config.pl files).  Those are only accessed during backup or restore 
jobs, when the "vault" disk is active anyway.
So, in short, I want to bring __TOPDIR__/pc/$host/LOG* and LOCK from the 
HDD to somewhere else.  Note that both LOG* and nnn are moving targets, 
so symlinks only work to a certain extent.
(By the way: LOG.DDMMYYYY is actually LOG.MMYYYY, without DD, on my 
4.1.3 installation.)


My usual approach is anything-sync-daemon (which uses an overlayfs to 
move some directory to tmpfs, and sync backs periodically and/or on user 
request).  However, this only works at directory level, and would 
require moving the entire __TOPDIR__/pc directory to tmpfs.  nnn can 
easily become large, and overlayfs is ill-suited for that.
Another approach would be to move __TOPDIR__/pc to the SSD and rsync 
back each __TOPDIR__/pc/$host/ to the HDD in a DumpPostUserCmd and, 
perhaps, after each nightly.  That's better, but again, the nnn can grow 
large (at least in number of files) and might become ill-suited even for 
the SSD and/or rootfs.


=> Question 1.) Short of patching BackupPC, does anyone have a 
suggestion or alternate solution I didn't think of?


It would be way easier to properly separate the two kinds of data if the 
location of the per-host LOG* and LOCK files would be configurable, e.g. 
to somewhere below __LOGDIR__.  This would match the logic for the 
global LOCK (which resides at __LOGDIR__), even though I'd typically 
expect lock files to live below /run/lock or /var/run/lock.
Full backward compatibility could be kept, e.g., through a 
__PERHOSTLOGDIR__ variable defaulting to __TOPDIR__/pc/.

I'd be willing to look into preparing a patch and pull request, if there 
is any interest.  But I understand that the scenario and motivation are 
negligible in an enterprise setup, and that changing the storage layout 
can a royal pain in the * for existing systems.  Hence,


=> Question 2.) Would such a change have any chance to be integrated in 
BackupPC?  If so, under what constraints for the interface?  Any 
recommendations for locations?  What about 
{Xfer,Restore,Archive}{ERR,LOG,Info}*?



Best,
Alex