Re: removable HD instead of tape how-to

On 12/23/24 15:13, D. wrote:
> 
> Is there any documentation on how to use USB-attached external hard
> drives in place of tapes?  If yes, where can I find the documentation?
> 
> I would like to rotate the external drives for off-line off-site backups
> just like traditional tapes.
> 
> Any guidance on this subject is greatly appreciated.


I'm using a SAS-attached removable zpool, but the fundamental principle 
for removable disk is the same regardless of the underlying technology.

There's multiple ways to do it, I'm sure.  This is the way I do it.


My SD has two Storage Devices, one internal (actually still hot-swap) 
and one external/removable that gets swapped out after each full backup 
cycle:

Device {
   Name = FileStorage
   Device Type = File
   Media Type = File
   Archive Device = /bpool/bacula
   LabelMedia = yes
   Random Access = Yes
   AutomaticMount = yes
   RemovableMedia = no
   Requires Mount = no
   AlwaysOpen = no
   Maximum Concurrent Jobs = 20
}


Device {
   Name = ArchiveCopy
   Device Type = File
   Media Type = File
   Archive Device = /arcpool
   LabelMedia = yes
   Random Access = Yes
   AutomaticMount = yes
   RemovableMedia = no
   Requires Mount = no
   AlwaysOpen = no
   Maximum Concurrent Jobs = 20
}


You'll notice these look EXACTLY THE SAME.  The first is the internal 
pool, which is a ZFS zpool comprising six 2TB SSDs in RAIDZ.  The second 
is the removable pool, a second ZFS zpool of four 2TB hot-swap hard 
disks also in RAIDZ.

Bacula *doesn't care about that* and never has to care.  All it cares 
about is that when it goes to run Full backups in the FileStorage pool, 
then Copy the full backups to the ArchiveCopy pool, *there is a volume 
mounted at that mountpoint*.  I actually have four sets of disks for 
arcpool, which I use in rotation, and Bacula neither knows that nor cares.

The Pools on this storage are defined like this:

Pool {
   Name = Full-Disk
   Storage = epsilon3-file
   Next Pool = Full-Archive
   Pool Type = Backup
   Recycle = no
   Recycle Oldest Volume = no
   Recycle Current Volume = no
   AutoPrune = yes                     # Prune expired volumes
   Volume Retention = 6 months
   File Retention = 6 months
   Maximum Volume Jobs = 0
   Volume Use Duration = 23h
   Label Format = 
"FULL-$Year${Month:p/2/0/r}${Day:p/2/0/r}-${Hour:p/2/0/r}:${Minute:p/2/0/r}"
   RecyclePool = Scratch
}

Pool {
   Name = Full-Archive
   Storage = epsilon3-archive
   Pool Type = Backup
   Recycle = no
   Recycle Oldest Volume = no
   Recycle Current Volume = no
   AutoPrune = yes                     # Prune expired volumes
   Volume Retention = 6 months
   File Retention = 6 months
   Maximum Volume Jobs = 0
   Volume Use Duration = 23h
   Label Format = 
"ARCHIVE-$Year${Month:p/2/0/r}${Day:p/2/0/r}-${Hour:p/2/0/r}:${Minute:p/2/0/r}"
   RecyclePool = Scratch
}


The removable pool, Archive, is defined exactly the same way, same 
expiration, same EVERYTHING as the Full-Disk pool EXCEPT that it uses 
the other device and has a different Label prefix.

So if a previously used volume is not present on the removable drives at 
any given time, again, Bacula doesn't care, because it's never going to 
try to re-use a Volume anyway.  It creates new Volumes, uses them 
writeable for 23 hours, then marks them Used and creates new ones for 
the next set of jobs.  When each volume expires, it is recycled into the 
scratch pool, then a separate admin job goes through the scratch pool 
and deletes every Volume it finds in it.


This is a different hardware implementation than yours, but the same 
principle should work equally well with a single USB3 removable device. 
All *you* have to make sure of is that there is always a device mounted 
at that mountpoint when Bacula comes to write to it, and that you 
manually delete any expired volumes off the device when you mount it. 
(Because if that *device* was unmounted at the time that Bacula expired 
the volumes stores on it, Bacula doesn't know about them any more and 
doesn't know to delete them before it starts writing to the device.)

I probably COULD create an admin job to delete any expired volumes 
present on the storage device before starting to use it, but honestly 
I've never bothered, since I have to physically load the drives into the 
external chassis and manually import the zpool anyway.



-- 
   Phil Stracchino
   Fenian House Publishing
   [email protected]
   [email protected]
   Landline: +1.603.293.8485
   Mobile:   +1.603.998.6958