Re: [BackupPC-users] The one-at-a-time nightly problem, debugged; DEVS PLEASE READ; long.

[Robin L. P. <rlp...@di...>]

On Sun, Nov 28, 2010 at 10:35:20PM -0800, Robin Lee Powell wrote:
> On Sun, Nov 28, 2010 at 07:01:45PM -0800, Craig Barratt wrote:
> > Robin,
> > 
> > Thanks for the detailed analysis.  I agree - it's pretty broken.
> > 
> > Preventing queuing a second nightly request when one is running
> > will at least avoid the problem.  However, I don't recommend
> > killing the current running nightly.
> 
> When I ask it to run a nightly, it's because I've just manually
> deleted stuff and I really need to free up disk space ASAP.  This
> happens regularily around here.
> 
> Just so you're aware of the use case.

In case you decide to go with a solution like mine, or someone else
wants it, here's the actually-tested version:

- -------------

ut00-s00003 ~ # diff -uw /var/tmp/ /usr/local/bin/BackupPC
--- /var/tmp/BackupPC   2010-11-25 01:52:30.000000000 +0000
+++ /usr/local/bin/BackupPC     2010-11-29 18:18:22.000000000 +0000
@@ -522,6 +522,15 @@
         $req = pop(@CmdQueue);

         $host = $req->{host};
+
+       if( $BackupPCNightlyJobs > 0 && ! $bpc->isAdminJob($host) )
+       {
+            print(LOG $bpc->timeStamp, "Tried to run ".$req->{cmd}." on $host when there are nightlies running.  That's bad.\n");
+
+            unshift(@CmdQueue, $req);
+            return;
+       }
+
         if ( defined($Jobs{$host}) ) {
             print(LOG $bpc->timeStamp,
                        "Botch on admin job for $host: already in use!!\n");
@@ -1362,7 +1371,31 @@
             } elsif ( $cmd =~ /^backup all$/ ) {
                 QueueAllPCs();
             } elsif ( $cmd =~ /^BackupPC_nightly run$/ ) {
+                print(LOG $bpc->timeStamp,
+                 "Running nightlies at user request.\n" );
+               foreach my $host (keys %Jobs) {
+                 if( $bpc->isAdminJob( $host ) ) {
+                   my $pid = $Jobs{$host}{pid};
+                   kill($bpc->sigName2num("INT"), $pid);
+                   delete $Jobs{$host};
+                    print(LOG $bpc->timeStamp,
+                     "Killing nightly job $host with PID $pid to make way for manual run.\n");
+                 }
+               }
+               # Clear all traces of Nightly jobs by name, just in case
+               for ( my $i = 0 ; $i < $Conf{MaxBackupPCNightlyJobs} ; $i++ ) {
+                 my $host = $bpc->adminJob($i);
+                 if( exists $Jobs{$host} ) {
+                   my $pid = $Jobs{$host}{pid};
+                   kill($bpc->sigName2num("INT"), $pid);
+                   delete $Jobs{$host};
+                   print(LOG $bpc->timeStamp,
+                     "Killing nightly job $host with PID $pid to make way for manual run.\n");
+                 }
+               }
                 $RunNightlyWhenIdle = 1;
+               $BackupPCNightlyJobs = 0;
+               $BackupPCNightlyLock = 0;
             } elsif ( $cmd =~ /^backup (\S+)\s+(\S+)\s+(\S+)\s+(\S+)/ ) {
                 my $hostIP     = $1;
                 $host          = $2;


- -------------

Perhaps some redundancy/overkill there, but I've seen enough failure
modes here that I want to be careful.  :)

-Robin

-- 
http://singinst.org/ :  Our last, best hope for a fantastic future.
Lojban (http://www.lojban.org/): The language in which "this parrot
is dead" is "ti poi spitaki cu morsi", but "this sentence is false"
is "na nei".   My personal page: http://www.digitalkingdom.org/rlp/