[Bacula-users] Catastrophic overflow block problems

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Hi,

I am sometimes getting these errors in my bacula backups:

Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0"

and it is more likely on the larger volume backups. It seemingly results 
from bacula trying to write an additional block to a disk drive that is 
already 100% full. How can I stop bacula from believing this is a valid 
thing to do?

Background:  I have bacula setup on my local network to backup a file 
server and a number of workstations. The file server is also the bacula 
director and is running Fedora 15 and 
"bacula-common-5.0.3-28.fc15.x86_64". Bacula is writing backups to an 
iSCSI disk group (not array) over ethernet; there are 6 disks of 1TB to 
2TB size and these are managed using "vchanger" 0.8.6, with 6 magazines 
each with 24 virtual volumes. The file server has 3.5TB of files and 
other workstations add about another 1TB.

More-complete log:

    17-Jan 14:49 helva-sd JobId 3417: Recycled volume "DiskPool1_0006_0017" on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0), all previous data lost.
    17-Jan 14:49 helva-sd JobId 3417: New volume "DiskPool1_0006_0017" mounted on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0) at 17-Jan-2013 14:49.
    17-Jan 14:49 helva-sd JobId 3417: End of Volume "DiskPool1_0006_0017" at 0:216 on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). Write of 64512 bytes got 3879.
    17-Jan 14:49 helva-sd JobId 3417: End of medium on Volume "DiskPool1_0006_0017" Bytes=217 Blocks=0 at 17-Jan-2013 14:49.
    17-Jan 14:49 helva-sd JobId 3417: 3307 Issuing autochanger "unload slot 89, drive 0" command.
    17-Jan 14:49 helva-dir JobId 3417: Using Volume "DiskPool1_0006_0018" from 'Scratch' pool.
    17-Jan 14:49 helva-sd JobId 3417: 3301 Issuing autochanger "loaded? drive 0" command.
    17-Jan 14:49 helva-sd JobId 3417: 3302 Autochanger "loaded? drive 0", result: nothing loaded.
    17-Jan 14:49 helva-sd JobId 3417: 3304 Issuing autochanger "load slot 90, drive 0" command.
    17-Jan 14:49 helva-sd JobId 3417: 3305 Autochanger "load slot 90, drive 0", status is OK.
    17-Jan 14:49 helva-sd JobId 3417: Recycled volume "DiskPool1_0006_0018" on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0), all previous data lost.
    17-Jan 14:49 helva-sd JobId 3417: New volume "DiskPool1_0006_0018" mounted on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0) at 17-Jan-2013 14:49.
    17-Jan 14:49 helva-sd JobId 3417: End of Volume "DiskPool1_0006_0018" at 0:216 on device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). Write of 64512 bytes got 3879.
    17-Jan 14:49 helva-sd JobId 3417: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). ERR=No space left on device17-Jan 14:49 helva-fd JobId 3417: Error: bsock.c:393 Write error sending 65562 bytes to Storage daemon:helva.cam.ivimey.org:9103: ERR=Connection reset by peer
    17-Jan 14:49 helva-fd JobId 3417: Fatal error: backup.c:1024 Network send error to SD. ERR=Connection reset by peer
    17-Jan 14:49 helva-sd JobId 3417: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). ERR=No space left on device17-Jan 14:49 helva-sd JobId 3417: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). ERR=No space left on device17-Jan 14:49 helva-sd JobId 3417: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). ERR=No space left on device17-Jan 14:49 helva-sd JobId 3417: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device "DiskStorage-drive-0" (/var/spool/bacula/vchanger/0/drive0). ERR=No space left on device17-Jan 14:49 helva-sd JobId 3417: Job write elapsed time = 14:30:55, Transfer rate = 12.06 M Bytes/second
    17-Jan 14:49 helva-dir JobId 3417: Error: Bacula helva-dir 5.0.3 (04Aug10): 17-Jan-2013 14:49:32
       Build OS:               x86_64-redhat-linux-gnu redhat
       JobId:                  3417
       Job:                    Helva_Home.2013-01-17_00.17.26_23
       Backup Level:           Full
       Client:                 "helva-fd" 5.0.3 (04Aug10) x86_64-redhat-linux-gnu,redhat,
       FileSet:                "Home" 2010-12-07 13:37:32
       Pool:                   "Normal-Full-18w" (From Job FullPool override)
       Catalog:                "MyCatalog" (From Client resource)
       Storage:                "DiskStorage" (From command line)
       Scheduled time:         17-Jan-2013 00:17:26
       Start time:             17-Jan-2013 00:17:28
       End time:               17-Jan-2013 14:49:32
       Elapsed time:           14 hours 32 mins 4 secs
       Priority:               12
       FD Files Written:       162,621
       SD Files Written:       162,621
       FD Bytes Written:       630,336,973,872 (630.3 GB)
       SD Bytes Written:       630,355,165,137 (630.3 GB)
       Rate:                   12046.8 KB/s
       Software Compression:   7.7 %
       VSS:                    no
       Encryption:             no
       Accurate:               no
       Volume name(s):         DiskPool1_0006_0003|DiskPool1_0001_0020|DiskPool1_0004_0013|DiskPool1_0004_0014|DiskPool1_0004_0023|DiskPool1_0004_0024|DiskPool1_0006_0017|DiskPool1_0006_0018
       Volume Session Id:      417
       Volume Session Time:    1351720132
       Last Volume Bytes:      217 (217 B)
       Non-fatal FD errors:    1
       SD Errors:              1
       FD termination status:  Error
       SD termination status:  Error
       Termination:            *** Backup Error ***

-- 
Software Manager & Engineer
Blog: http://www.ivimey.org/blog
LinkedIn: http://uk.linkedin.com/in/ruthivimeycook/