[Bacula-users] No Job status returned from FD. Backup fails

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Good morning list, here's my setup:

Director 7.4.2 running on OpenBSD 6.0 (hostname "fafnir")
FD 5.2.6 running on Debian/GNU Linux 7.0 on a remote site (hostname
"perseus")
Connected by a TLS tunnel

I have been using this for about a year now, for backing up both on-site
and off-site machines. The director used to be 7.0.5 on OpenBSD 5.8, the
clients are running Linux and Windows.

A few days ago I set up a new director (see above), and moved the old
configuration files from the old to the new machine. It all worked well
and as I expected it - with one exception.

The volume to back up from the above client is large, about 100GB for a
full backup, and consequently takes up to 20 hours to run. I haven't
been able to run a single full backup yet, as at some point the
connection seems to get lost. Yesterday I managed to run a (probably)
full backup, but apparently "finished" message from the client never got
back to the director, and after a few more hours the connection dropped.

This is what the client reports:

 131  Full   1,007,689    119.0 G  OK       06-Apr-17 14:07 perseus-Backup

Here's the job summary:

06-Apr 18:58 fafnir JobId 131: Error: Bacula fafnir 7.4.2 (06Jun16):
  Build OS:               x86_64-unknown-openbsd6.0 openbsd 6.0
  JobId:                  131
  Job:                    perseus-Backup.2017-04-05_15.39.23_29
  Backup Level:           Full (upgraded from Incremental)
  Client:                 "perseus" 5.2.6 (21Feb12)
x86_64-pc-linux-gnu,debian,7.0
  FileSet:                "Unixoid" 2017-03-29 23:05:01
  Pool:                   "Standard" (From Command input)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "Standard-Device" (From command line)
  Scheduled time:         05-Apr-2017 15:39:21
  Start time:             05-Apr-2017 15:39:26
  End time:               06-Apr-2017 18:58:45
  Elapsed time:           1 day 3 hours 19 mins 19 secs
  Priority:               10
  FD Files Written:       0
  SD Files Written:       1,007,689
  FD Bytes Written:       0 (0 B)
  SD Bytes Written:       119,184,574,492 (119.1 GB)
  Rate:                   0.0 KB/s
  Software Compression:   None
  Snapshot/VSS:           no
  Encryption:             no
  Accurate:               no
  Volume name(s):         FVol-0001|FVol-0013|FVol-0014|FVol-0015
  Volume Session Id:      2
  Volume Session Time:    1491397950
  Last Volume Bytes:      9,574,999,912 (9.574 GB)
  Non-fatal FD errors:    1
  SD Errors:              0
  FD termination status:  Error
  SD termination status:  OK
  Termination:            *** Backup Error ***

Here's the output from bacula.log:

05-Apr 15:39 fafnir JobId 131: No prior Full backup Job record found.
05-Apr 15:39 fafnir JobId 131: No prior or suitable Full backup found in
catalog. Doing FULL backup.
05-Apr 15:39 fafnir JobId 131: Start Backup JobId 131,
Job=perseus-Backup.2017-04-05_15.39.23_29
05-Apr 15:39 fafnir JobId 131: Using Device "HESTIA-files" to write.
06-Apr 01:17 Standard-Device JobId 131: End of medium on Volume
"FVol-0001" Bytes=53,687,041,713 Blocks=832,206 at 06-Apr-2017 01:17.
06-Apr 01:17 Standard-Device JobId 131: Volume "FVol-0013" previously
written, moving to end of data.
06-Apr 01:17 Standard-Device JobId 131: Ready to append to end of Volume
"FVol-0013" size=7,424,802,068
06-Apr 01:17 Standard-Device JobId 131: New volume "FVol-0013" mounted
on device "HESTIA-files" (/var/import/hestia/_bacula-sd) at 06-Apr-2017
01:17.
06-Apr 09:50 Standard-Device JobId 131: End of medium on Volume
"FVol-0013" Bytes=53,687,066,040 Blocks=832,206 at 06-Apr-2017 09:50.
06-Apr 09:50 fafnir JobId 131: Created new Volume="FVol-0014",
Pool="Standard", MediaType="50GB-Medium" in catalog.
06-Apr 09:50 Standard-Device JobId 131: Labeled new Volume "FVol-0014"
on file device "HESTIA-files" (/var/import/hestia/_bacula-sd).
06-Apr 09:50 Standard-Device JobId 131: Wrote label to prelabeled Volume
"FVol-0014" on file device "HESTIA-files" (/var/import/hestia/_bacula-sd)
06-Apr 09:50 Standard-Device JobId 131: New volume "FVol-0014" mounted
on device "HESTIA-files" (/var/import/hestia/_bacula-sd) at 06-Apr-2017
09:50.
06-Apr 12:26 Standard-Device JobId 131: End of medium on Volume
"FVol-0014" Bytes=53,687,079,186 Blocks=832,203 at 06-Apr-2017 12:26.
06-Apr 12:26 fafnir JobId 131: Created new Volume="FVol-0015",
Pool="Standard", MediaType="50GB-Medium" in catalog.
06-Apr 12:26 Standard-Device JobId 131: Labeled new Volume "FVol-0015"
on file device "HESTIA-files" (/var/import/hestia/_bacula-sd).
06-Apr 12:26 Standard-Device JobId 131: Wrote label to prelabeled Volume
"FVol-0015" on file device "HESTIA-files" (/var/import/hestia/_bacula-sd)
06-Apr 12:26 Standard-Device JobId 131: New volume "FVol-0015" mounted
on device "HESTIA-files" (/var/import/hestia/_bacula-sd) at 06-Apr-2017
12:26.
06-Apr 14:07 Standard-Device JobId 131: Elapsed time=22:28:30, Transfer
rate=1.473 M Bytes/second
06-Apr 14:07 Standard-Device JobId 131: Sending spooled attrs to the
Director. Despooling 350,356,996 bytes ...
06-Apr 18:57 fafnir JobId 131: Fatal error: Network error with FD during
Backup: ERR=Connection reset by peer
06-Apr 18:58 fafnir JobId 131: Fatal error: No Job status returned from FD.
06-Apr 18:58 fafnir JobId 131: Error: Bacula fafnir 7.4.2 (06Jun16):

I'm a bit lost atm. I'm backing up a few more remote machines like this,
with not quite the same volume of data but still some. They all run,
like they used to for about a year. Just this one isn't. The only
apparent change to is is moving from OpenBSD 5.8 to 6.0, and from Bacula
7.0.5 to 7.4.2.

One of the remote clients with a bigger volume is running precisely the
same OS and FD version.

TIA
Matthias