job dies with ERR=error:0A000119:SSL routines::decryption failed or bad record mac

Hello,

I have several clients which have recently start failing with:

SD says - Error: openssl.c:108 TLS read/write failure.: ERR=error:0A000119:SSL routines::decryption failed or bad record mac
FD says - Error: bsock.c:397 Wrote 43011 bytes to Storage daemon:bacula-sd-04.int.unixathome.org:9103, but only 0 accepted.
SD says - Fatal error: append.c:327 Network error reading from FD. ERR=Unknown error: 9919

My search for these errors isn't finding anything to try.

Full job output appears below.

Relevant background:

* Bacula 15.0.2 installed on clients and servers
* FreeBSD 14.x
* the failing clients have been around for years, with successful backups
* the gateway in my basement was recently replaced, new firewall rules and OpenVPN configuration
* the OpenVPN topology went from net40 to subnet
* this affects all the VPN hosts; local hosts are unaffected

Given the Bacula configuration hasn't changed and these jobs had been running successfully for years, the problem must be with OpenVPN and/or the firewall rules.  However, I cannot find the cause.

Here is a failed job:

02-Mar 19:06 bacula-dir JobId 373736: Start Backup JobId 373736, Job=r720-02_basic.2025-03-02_19.06.22_22
02-Mar 19:06 bacula-dir JobId 373736: Connected to Storage "bacula-sd-04-FullFile" at bacula-sd-04.int.unixathome.org:9103 with TLS
02-Mar 19:06 bacula-dir JobId 373736: There are no more Jobs associated with Volume "FullAuto-04-15375". Marking it purged.
02-Mar 19:06 bacula-dir JobId 373736: All records pruned from Volume "FullAuto-04-15375"; marking it "Purged"
02-Mar 19:06 bacula-dir JobId 373736: Recycled volume "FullAuto-04-15375"
02-Mar 19:06 bacula-dir JobId 373736: Using Device "vDrive-FullFile-4" to write.
02-Mar 19:06 bacula-dir JobId 373736: Connected to Client "r720-02-fd" at r720-02.vpn.unixathome.org:9102 with TLS
02-Mar 19:06 r720-02-fd JobId 373736: Connected to Storage at bacula-sd-04.int.unixathome.org:9103 with TLS
02-Mar 19:06 bacula-sd-04 JobId 373736: Recycled volume "FullAuto-04-15375" on File device "vDrive-FullFile-4" (/usr/local/bacula/volumes/FullFile), all previous data lost.
02-Mar 19:06 bacula-dir JobId 373736: Max Volume jobs=1 exceeded. Marking Volume "FullAuto-04-15375" as Used.
02-Mar 19:06 bacula-sd-04 JobId 373736: Error: openssl.c:108 TLS read/write failure.: ERR=error:0A000119:SSL routines::decryption failed or bad record mac
02-Mar 19:06 r720-02-fd JobId 373736: Error: bsock.c:397 Wrote 43011 bytes to Storage daemon:bacula-sd-04.int.unixathome.org:9103, but only 0 accepted.
02-Mar 19:06 bacula-sd-04 JobId 373736: Fatal error: append.c:327 Network error reading from FD. ERR=Unknown error: 9919
02-Mar 19:06 r720-02-fd JobId 373736: Fatal error: backup.c:1057 Network send error to SD. ERR=Input/output error
02-Mar 19:06 bacula-sd-04 JobId 373736: Elapsed time=00:00:01, Transfer rate=16.71 M Bytes/second
02-Mar 19:06 r720-02-fd JobId 373736: Error: bsock.c:276 Socket has errors=1 on call to Storage daemon:bacula-sd-04.int.unixathome.org:9103
02-Mar 19:06 bacula-dir JobId 373736: Error: Director's connection to SD for this Job was lost.
02-Mar 19:06 bacula-dir JobId 373736: Error: Bacula bacula-dir 15.0.2 (21Mar24):
  Build OS:               amd64-portbld-freebsd14.1 freebsd 14.1-RELEASE
  JobId:                  373736
  Job:                    r720-02_basic.2025-03-02_19.06.22_22
  Backup Level:           Full
  Client:                 "r720-02-fd" 15.0.2 (21Mar24) amd64-portbld-freebsd14.1,freebsd,14.1-RELEASE
  FileSet:                "basic backup" 2019-10-28 03:05:00
  Pool:                   "FullFile-04" (From Job FullPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "bacula-sd-04-FullFile" (From Pool resource)
  Scheduled time:         02-Mar-2025 19:06:19
  Start time:             02-Mar-2025 19:06:25
  End time:               02-Mar-2025 19:06:26
  Elapsed time:           1 sec
  Priority:               10
  FD Files Written:       91
  SD Files Written:       0
  FD Bytes Written:       17,352,576 (17.35 MB)
  SD Bytes Written:       0 (0 B)
  Rate:                   17352.6 KB/s
  Software Compression:   100.0% 1.0:1
  Comm Line Compression:  45.9% 1.8:1
  Snapshot/VSS:           no
  Encryption:             no
  Accurate:               no
  Volume name(s):         FullAuto-04-15375
  Volume Session Id:      56
  Volume Session Time:    1740841409
  Last Volume Bytes:      16,733,601 (16.73 MB)
  Non-fatal FD errors:    3
  SD Errors:              0
  FD termination status:  Error
  SD termination status:  Error
  Termination:            *** Backup Error ***


-- 
  Dan Langille
  [email protected]