Re: [Vchanger-users] bacula requesting nonexistent slots

[Steven A. F. <ste...@gm...>]

On 8/10/23 11:19 AM, Josh Fisher wrote:
> On 8/10/23 09:51, Steven A. Falco wrote:
>> I've had a working setup for years, but suddenly I'm getting errors:
>>
>> Fatal error: 3992 Bad autochanger "load Volume v1_0005_0197 Slot 412, Drive 0": ERR=Child exited with code 1.
>> Results=cannot load drive 0 from invalid slot 412
>>
>> bacula indeed thinks the volume is in slot 412, according to the "list media" command:
>>
>>  1,501 | v1_0005_0197 | Append    |       1 | 23,539,407,831 | 5 |    3,628,800 |       1 |  412 |         1 | File |       1 |        0 | 2023-08-09 00:54:13 | 3,511,018 |
>>
>> I tried doing "vchanger /etc/vchanger/v1.conf REFRESH" which I thought would update bacula's inventory of the slots, but that doesn't seem effective.
>>
>> If I ask vchanger what it has, using "vchanger /etc/vchanger/v1.conf LIST", I get:
>>
>> 198:v1_0005_0197
>>
>> so vchanger thinks the volume is in slot 198.
>>
>> I was able to get bacula to correct its slots via "update slots", and things are working again, but I'd like to understand how the slots/volumes might have gotten out of sync.  Any thoughts on that would be appreciated.
>>
>> Also, is the "update slots" command the right way to recover?
> 
> 
> Yes, 'update slots' is the right way. It gets out of sync every time a filesystem (virtual magazine), say a removable drive, is attached or detached from the system. Vchanger assigns that filesystem's volume files to slots in the order that they are attached. Once a filesystem is detached, the slots it occupied may be reused. Regardless, when it is again attached, it is likely to be assigned different to a different set of slots.
> 
> Ordinarily, vchanger calls out to Bacula via its bconsole command when it detects a change in occupied slots. The REFRESH command simply triggers vchanger to check for slot changes and initiate a sync if needed. The sync is simply a call to bconsole to issue an 'update slots' command.
> 
> So, you need to somehow initiate an 'update slots' command whenever a filesystem is attached or detached. You can call vchanger with the REFRESH command, you can go into bconsole and issue it directly, or you can use udev rules to cause udev to automatically launch vchanger with the REFRESH command whenever a filesystem assigned to vchanger is physically attached or detached.
> 
> To use udev, there is a script (usually) installed at /usr/bin/vchanger-genudevrules that can be used to generate the udev rules by scanning the magazine= lines in the vchanger config file. Whenever a new filesystem is added or removed from the vchanger config file, those udev rules need to be regenerated  It should be used to update a udev rules file, usually in /etc/udev/rules.d. I have mine in /etc/udev/rules.d/96-vchanger.rules.
> 
> If the REFRESH command does not seem to actually change the slot number in Bacula, then something is preventing vchanger from launching the bconsole command. Check the vchanger config file. There is a bconsole= parameter that defaults to "bconsole". If the bconsole parameter is empty/blank, then vchanger will log that the update slots is needed, but will not actually call bconsole. Otherwise, if bconsole is not installed in a directory in the PATH environment variable, then it won't find the bconsole binary. Otherwise, there could be a permissions problem. Set log_level = LOG_DEBUG in the vchanger config to get very detailed logging of the process, which should show where it is failing.
> 
> 

I've checked the log, and I had one instance of:

Aug 09 09:49:09:  [12299]: WARNING! 'update slots' needed in bconsole
Aug 09 09:49:09:  [12299]: bconsole update slots command success

But after doing the manual "update slots", I don't see any more warnings.  I don't know why bconsole couldn't be run that one time, but I'll keep an eye on it.  Maybe I issued it as an unprivileged user rather than as root.

As an aside, looking at bconsole.cpp I'm surprised that the "success" message would be printed along with the WARNING line.  It might be clearer to have the "success" line in an else clause, like:

    if (update_slots) {
       tFormat(cmd, "update slots storage=\"%s\" drive=\"0\"", conf.storage_name.c_str());
       if(issue_bconsole_command(cmd.c_str())) {
          vlog.Error("WARNING! 'update slots' needed in bconsole");
       } else {
          vlog.Info("bconsole update slots command success");
       }
    }

The same would be true of the label_barcodes a few lines later in the file.

I've also attached a trivial patch, because when reading the log I noticed that there is a typo which I fixed via s/preforming/performing/ in vchanger.cpp (around lines 638-672).

	Thanks for the rapid response!
	Steve