Menu
▾
▴
[Bacula-bugs] [bacula 0000173]: The number of files mismatch! after verify?
|
From: <bac...@li...> - 2004-12-28 13:56:23
|
A BUGNOTE has been added to this bug. ====================================================================== http://bugs.bacula.org/bug_view_advanced_page.php?bug_id=0000173 ====================================================================== Reported By: gvan Assigned To: ====================================================================== Project: bacula Bug ID: 173 Category: Director Reproducibility: always Severity: major Priority: normal Status: feedback ====================================================================== Date Submitted: 11-22-2004 23:19 PST Last Modified: 12-28-2004 06:01 PST ====================================================================== Summary: The number of files mismatch! after verify? Description: After backup data I do Verify Volume to Catalog. If after that to make again backup data I receive a error, for example: Error: I canot write on Volume "u1-s3-v1" because: The number of files mismatch! Volume=26 Catalog=39 If before new backup I do restart bacula error does not occur and reserve copying passes normally on given volume. ====================================================================== ---------------------------------------------------------------------- kern - 11-23-2004 11:51 PST ---------------------------------------------------------------------- You will need to send me your bacula-dir.conf file (as an attachement), and include the output from your job log -- that is the output from the backup, the subsequent verify, and the output from the job that failed. Please attach that to the bug report as a .txt file. ---------------------------------------------------------------------- ArnoL - 11-23-2004 14:27 PST ---------------------------------------------------------------------- There was a thread on [Bacula-users] Verify error (VolumeToCatalog) after backup starting started Oct. 13. That looks like it's a similar problem - perhaps even the same? Then, Martin Simmons reported something similar with a long discussion with Kern following. That was "Tape with an empty file between sessions". I can't recall how this ended, but I *think* that Martin had some patch and everybody waited if the error was gone. ---------------------------------------------------------------------- gvan - 11-24-2004 01:53 PST ---------------------------------------------------------------------- To kern: I have uploaded necessary files. Inform, if something else is required. In case of need the errror can be tried to repeat once again with other conditions. To ArnoL: If the question is this message http://sourceforge.net/mailarchive/forum.php?thread_id=5755103&forum_id=8833 the given problem completely is solved. But in this case a problem completely another. edited on: 11-24-04 01:53 ---------------------------------------------------------------------- kern - 11-24-2004 09:39 PST ---------------------------------------------------------------------- Thanks for the output. It was *exactly* what I needed to "see" your problem. I haven't been able to duplicate this bug here, and looking at your bacula-sd.conf file, I think the problem is because you have: TWO EOF = yes I'm a bit surprised that btape "test" doesn't detect this. Could you do the following things? 1. Run btape "test" command from version 1.36.1 with your current configuration, and if it works, please send me the output, as it will indicate that my tests are not sufficiently good. 2. Remove the TWO EOF = yes, then run the btape "test" command again, and make sure it succeeds. 3. If http://bugs.bacula.org/bug_view_advanced_page.php?bug_id=0000002 works, remove the TWO EOF = yes from your Ultrium Device conf (and perhaps also from your DAT Device conf), then run the backup, verify, and backup and see if it succeeds. 4. If http://bugs.bacula.org/bug_view_advanced_page.php?bug_id=0000003 works, then do the same test again, but stop and restart Bacula after the first backup, then proceed with the other tests. ---------------------------------------------------------------------- gvan - 11-25-2004 00:07 PST ---------------------------------------------------------------------- For a long time in release notes of one of versions or even in the documentation it is told that for FreeBSD it was desirable to establish TWO EOF = yes. Since then it at me also is established in this value. Also at me it is in addition established for tape in start of server: /usr/bin/mt -f /dev/nsa1 seteotmodel 2 /usr/bin/mt -f /dev/nsa1 blocksize 0 /usr/bin/mt -f /dev/nsa0 seteotmodel 2 /usr/bin/mt -f /dev/nsa0 blocksize 0 >1. Run btape "test" command from version 1.36.1 with your current >configuration, and if it works, please send me the output, as it will indicate >that my tests are not sufficiently good. Test works fine. See result in btape-1.txt >2. Remove the TWO EOF = yes, then run the btape "test" command again, and make >sure it succeeds. See result in btape-2_1.txt. Test ends with one error and with one recomendation - "Backward Space Record = No". After I set recomendation test ends without error because read backwards test is skiped (result in btape-2_2.txt) >3. If 0000002 works, remove the TWO EOF = yes from your Ultrium Device conf >(and perhaps also from your DAT Device conf), then run the backup, verify, and >backup and see if it succeeds. Second backup after verify reproduce error (I have interrupted it then since and so it was already visible, that the error has repeated). Results in backup-3.txt, verify-3.txt, backup_next-3.txt. Modified bacula-sd.conf also is attached as bacula-sd.conf-mod Probably all the same it does not solve a problem in this case. Whether there are any else ideas on change of my configuration? ---------------------------------------------------------------------- kern - 11-25-2004 00:52 PST ---------------------------------------------------------------------- Hmm. Sorry, my error. I failed to notice that you are using FreeBSD, which means that in principle the Device configuration you have is correct. Just the same, for me it is a very poor way to run an LTO drive, which is quite intelligent. I would appreciate it if you would try the following Device configuration, which is also mentioned in the manual: Hardware End of Medium = no BSF at EOM = no Backward Space Record = no Backward Space File = no Fast Forward Space File = yes TWO EOF = no I recommend you start with the btape "test" command, and if that works, try the Backup/Verify sequence. If this doesn't work, I think I have an idea where the problem is, but I'll have to work on it after the release 1.36.1 in a few days. ---------------------------------------------------------------------- kern - 11-25-2004 00:56 PST ---------------------------------------------------------------------- Oh, sorry, I forgot to mention that you need to modify your tape drive parameters as follows prior to running with the new test Device configuration: mt -f /dev/nsa0 seteotmodel 1 and keep the other parameters you have already set. ---------------------------------------------------------------------- gvan - 11-25-2004 03:39 PST ---------------------------------------------------------------------- I have changed a configuration (a new file bacula-sd.conf-mod2). Has changed model: # mt -f /dev/nsa0 seteotmodel 1 /dev/nsa0: old model was 2 filemarks at EOT /dev/nsa0: new model is 1 filemark at EOT # mt -f /dev/nsa0 geteotmodel /dev/nsa0: the model is 1 filemark at EOT Has started the test btape (results in btape-4.txt, errors are not present). And has executed backup, verify and the second backup (see results in backup-4.txt, verify-4.txt and backup_next-4.txt). The error is same and has remained former:( ---------------------------------------------------------------------- kern - 12-03-2004 12:16 PST ---------------------------------------------------------------------- I really do not know what is going on here or why the Director thinks there are 4 files Please try yet another experiment. This time, do the following: 1 Do a "llist volume=Vol-name" where Vol-name is the volume you are going to write to, presumably empty. 2 Do the backup. 3 Do a "llist volume=Vol-name" "sql" "select * from JobMedia where JobId=nnn" " " (blank line) where nnn is the JobId of the backup Job. 4 Run the Verify 5 Re-do step 3 6 Run the second backup 5 Re-do step 3 Send me the output from step 1 and the 3 X step 3. Maybe I can see something in it as it looks like something is really out of sync. It looks like the Verify is adding a file. The above info will confirm or not this idea. ---------------------------------------------------------------------- gvan - 12-09-2004 03:09 PST ---------------------------------------------------------------------- I think, that a problem all the same another. Results of the test are in a file output-last.txt. As a result of often tests by me it has been revealed, what even backup of small sizes and then verify some times can cause the following mistake successively: 08-Dec 10:13 x1-sd: Volume "u1-s8-v1" previously written, moving to end of data. 08-Dec 10:13 x1-sd: Ready to append to end of Volume "u1-s8-v1" at file=2. 08-Dec 10:13 x1-sd: End of Volume "u1-s8-v1" at 3:0 on device /dev/nsa0. Write of 64512 bytes got 0. 08-Dec 10:13 x1-sd: End of medium on Volume "u1-s8-v1" Bytes=1,976,947,844 Blocks=30,645 at 08-Dec-2004 10:13. I.e. somehow occurs out of sync quantities of files after backup and the subsequent verifies. And then there is a mistake of record and bacula considers, that the tape is full, though size of a tape about 200 GB. If after each verifies before next backups to do restart bacula the given mistake does not arise. ---------------------------------------------------------------------- kern - 12-16-2004 14:10 PST ---------------------------------------------------------------------- I haven't yet looked at your latest output, but from your notes, it appears that there is some strange bug with the interaction of your OS tape driver and Bacula. Bacula should *never* get an error (as it did in your example) when trying to append to a tape. I'll still look at the output, but my best bet is a bug in the OS or the LTO tape driver that in some cases prohibits appending to a tape or gets a false end of tape when it is really not the end of the tape. If restarting Bacula clears the problem, then it points even more to the OS driver as being in an unstable state. It doesn't prove it, but it makes me lean in that direction. ---------------------------------------------------------------------- Dan Langille - 12-28-2004 06:01 PST ---------------------------------------------------------------------- Are you backing up to tape? FreeBSD 4.9-RELEASE contains a pthreads bug that will result in more data being written to the tape than can be stored on the tape. This problem arises only if the backup spans more than one tape. Did your backup involve more than one tape? If so, the above pthreads issue might explain why your backup held less data than the catalog expected. This bug is easily patched. See platforms/freebsd/pthreads-fix.txt for details. Please ask if you have any questions. Bug History Date Modified Username Field Change ====================================================================== 11-22-04 23:19 gvan New Bug 11-22-04 23:19 gvan File Added: bacula-sd.conf 11-22-04 23:29 gvan Bug Monitored: gvan 11-23-04 11:51 kern Bugnote Added: 0000472 11-23-04 11:51 kern Status new => feedback 11-23-04 14:27 ArnoL Bugnote Added: 0000475 11-23-04 22:21 gvan File Added: backup.txt 11-23-04 22:21 gvan File Added: verify.txt 11-23-04 22:21 gvan File Added: backup_next.txt 11-23-04 22:22 gvan File Added: bacula-dir.conf 11-23-04 22:29 gvan Bugnote Added: 0000477 11-24-04 01:53 gvan Bugnote Edited: 0000477 11-24-04 09:39 kern Bugnote Added: 0000480 11-25-04 00:02 gvan File Added: btape-1.txt 11-25-04 00:02 gvan File Added: btape-2_1.txt 11-25-04 00:02 gvan File Added: btape-2_2.txt 11-25-04 00:03 gvan File Added: backup-3.txt 11-25-04 00:03 gvan File Added: verify-3.txt 11-25-04 00:04 gvan File Added: backup_next-3.txt 11-25-04 00:04 gvan File Added: bacula-sd.conf-mod 11-25-04 00:07 gvan Bugnote Added: 0000482 11-25-04 00:52 kern Bugnote Added: 0000483 11-25-04 00:56 kern Bugnote Added: 0000484 11-25-04 03:34 gvan File Added: bacula-sd.conf-mod2 11-25-04 03:34 gvan File Added: btape-4.txt 11-25-04 03:34 gvan File Added: backup-4.txt 11-25-04 03:35 gvan File Added: verify-4.txt 11-25-04 03:35 gvan File Added: backup_next-4.txt 11-25-04 03:39 gvan Bugnote Added: 0000485 12-03-04 12:16 kern Bugnote Added: 0000507 12-09-04 03:09 gvan Bugnote Added: 0000536 12-09-04 03:09 gvan File Added: output-last.txt 12-16-04 14:10 kern Bugnote Added: 0000551 12-28-04 06:01 Dan Langille Bugnote Added: 0000591 ====================================================================== |