From: Mariusz M. <mm...@ke...> - 2013-07-03 11:50:33
|
According to the docs and sd's tape-handling code, neither MTEOM nor FSF(0xffff…) will get used to quickly forward a tape to its end of data unless there's a working MTIOCGET ioctl that does EOF tracking. The reasoning being, that bacula uses files internally to track written data, so in order for append to tape to result in correct catalog entries, bacula must be using correct file numbers. But considering backup tapes are typically exclusively used by a given backup system, isn't it the case that bacula knows *exactly* how many files there are on a given tape without having to rely on the tape drive doing the counting? When I do a 'list volumes' in bconsole, one of the listed colums is 'volfiles'. Is there any reason why that value can't be used after a MTEOM/FSF(0xfff…) for any further writes? Or even a safer version for the paranoid: maxf=get_volfiles() fsf(maxf) read() If there's actual data there, abort (or rewind and do a fsf(1) loop), cause something's wrong. If read() indicates end of tape, we're ok. I can't think of a reason why it wouldn't be safe and I'm pretty sure that it would be a tad faster, then a looped fsf(1). Comments? --mmazur |
From: Kern S. <ke...@si...> - 2013-07-03 16:25:34
|
Hello, Bacula requires the information from the tape driver as a security measure. The tape driver knows if the tape has been moved (operator intervention ...) and Bacula does not know if the tape has been repositioned by some other means. The only way I know to be 100% sure that Bacula does not write in the wrong place on the tape is to use the existing technique. I prefer security over some minor speed improvements. All modern tape drives can quickly space to the end of the tape and correctly report the position. There were previously many drives that could not do this -- I think FreeBSD still cannot, so Bacula has code to do its own counting, but it is slower than modern Linux tape drivers. The problem with making changes to Bacula's low level code is that you have to test it on a pile of operating systems, with a pile of different tape drivers. I did that 10-12 years ago, and it was painful enough that I wouldn't like to do it again. You are, however, free to keep your own changed version if it works for you. Best regards, Kern On 07/03/2013 01:30 PM, Mariusz Mazur wrote: > According to the docs and sd's tape-handling code, neither MTEOM nor > FSF(0xffff…) will get used to quickly forward a tape to its end of data unless > there's a working MTIOCGET ioctl that does EOF tracking. The reasoning being, > that bacula uses files internally to track written data, so in order for > append to tape to result in correct catalog entries, bacula must be using > correct file numbers. > > But considering backup tapes are typically exclusively used by a given backup > system, isn't it the case that bacula knows *exactly* how many files there are > on a given tape without having to rely on the tape drive doing the counting? > > When I do a 'list volumes' in bconsole, one of the listed colums is > 'volfiles'. Is there any reason why that value can't be used after a > MTEOM/FSF(0xfff…) for any further writes? > > Or even a safer version for the paranoid: > > maxf=get_volfiles() > fsf(maxf) > read() > > If there's actual data there, abort (or rewind and do a fsf(1) loop), cause > something's wrong. If read() indicates end of tape, we're ok. I can't think of > a reason why it wouldn't be safe and I'm pretty sure that it would be a tad > faster, then a looped fsf(1). > > Comments? > > --mmazur > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Bacula-devel mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-devel > |
From: Mariusz M. <mm...@ke...> - 2013-07-03 21:16:40
|
On Wed of July 3 2013, Kern Sibbald wrote: > The problem with making changes to Bacula's low level code is that > you have to test it on a pile of operating systems, with a pile of > different tape drivers. I did that 10-12 years ago, and it was painful > enough that I wouldn't like to do it again. You are, however, free to > keep your own changed version if it works for you. I appreciate the fact that bacula tries to do the right thing out of the box in that is use tapes in a fast & secure way and if that fails, btape suggest a slower but still secure way (turning off EOM and fast FSF). What I had in mind was not changing the current behavior (or the current code paths guiding that behavior), but adding more options for admins. Current hardware can do a bit more than the old one. I'm quite certain I can prevent any non-bacula tape operations on my autoloader – the thing has a webgui and user access controls, so nobody's doing anything to the tapes unless I allow it. Having an option "Admin Is Certain Bacula Has Monopoly On Tape Usage" (defaults to No :) that allows me to make full use of the hardware I have, would be nice. Especially considering that a current generation LTO 6 is 2.5T pre-encryption. Even with LTO-5s I use, with a max file size set to 5 gigs, my backups get me an EOF on average every 4 gigs. Trying to append to a tape that's, say, 2.5t full (assuming I get 3t on lto-5 with 2:1 compression) will mean the tape's going to do 625 forwards, stops and reads just to get into position for the new data to get written. I'm not an expert, but that doesn't exactly sound like it'd be good for the tape's longevity. What I'm asking – should a patch exist that implements what I'm suggesting without messing current code, meaning unless one explicitly enables a new config option, tape handling is done fully by the old code, would you merge it? --mmazur |