Re: "JobSet", "Batch", or "Aggregate" Concept / Max tape usage per (WAS: Re: [Bacula-users] Hit me w/ a Clue-by-Four (Amanda user))

[Russell H. <rus...@wr...>]

Brian A. Seklecki wrote:
> On Thu, 2006-02-09 at 10:37 +0000, Russell Howe wrote:
>>Bacula will do this. Check you don't have "Maximum Volume Jobs = 1" in
> 
> 
> I do.  This is because I'm trying to enforce a specific threshold of a
> single-tape-per-day policy. 

Ah, but this is a single-tape-per-job policy.

> I can manage volume capacity myself.  I'd like to override Bacula's
> default behavior of auto-picking a tape based on status/capacity/recycle
> time.
> 
> I'd essentially like to "Micro-Manage" it perhaps.
> 
> I want my tapes to be written to by the scheduled jobs for their daily
> assignment, by all the jobs scheduled to run at that time, and then be
> marked "Used".
> 
> The trick is, if all the jobs run at the same time, don't mark the
> volume "Used" or "Full" until after they've all run.

When it comes to marking tapes as Used or Full, there seem to be two
ways to do it currently:

* Maximum Volume Jobs
* Volume Use Duration

So, you can either say "23 hours after the first job which wrote to this
tape {started,finished}, mark it as used" to get you a new tape every
day (I'm not sure which of those two options applies, but I guess it's
in TFM somewhere)

Or, you can say "After <n> jobs have been written to this tape, close it
off/mark it used"

The latter is what I do. I run 10 jobs every night, most of them on the
same schedule, so they start at the same time, although they don't run
concurrently - I have "Maximum Concurrent Jobs = 1" set for the storage
device, so that jobs get written sequentially.

After the last job completes (the Catalog backup), I get this:

11-Feb 21:43 bacula-dir: Start Backup JobId 26,
Job=BackupCatalog.2006-02-10_23.51.00
11-Feb 21:43 spanky-sd: Volume "PoolA_Weekly_1" previously written,
moving to end of data.
11-Feb 21:47 spanky-sd: Ready to append to end of Volume
"PoolA_Weekly_1" at file=95.
11-Feb 21:47 bacula-dir: Max Volume jobs exceeded. Marking Volume
"PoolA_Weekly_1" as Used.

This does mean that if I change the number of jobs which get written, I
have to remember to update the catalog's record for the media, and
update the Pool definitions in the director's configuration file. I have
this comment in the configuration to remind anyone who might inherit
this setup:

If you're not seeing the above behaviour, I guess there could be some
kind of race condition whereby when two jobs run concurrently, and both
try to incrememnt the "number of jobs" field in the catalog for the
media table, both jobs update it to n+1 instead of the first updating it
to n+1 and the second adding one to that (making it n+2)... I find it
hard to believe that such a bug would go unnoticed, though... maybe
nobody uses "Maximum Volume Jobs" with concurrent backups?!

-- 
Russell Howe
rus...@wr...