Re: Copy Job config help

On 2025/12/08 12:59, Rob Gerber wrote:
> 
> On Sun, Dec 7, 2025, 10:52 PM Rob Gerber <[email protected]> wrote:
> 
>> Copy jobs are tricky to wrap ones head around, as you noted. I know I had
>> trouble at first.
>>
>> The first things I'd try are:
>>
>> 1. The FileCP pool doesn't have a storage resource defined. In this case I
>> think you'd want one pointing to FileCP. The FileCP pool cannot express
>> which storage resource is appropriate for it, so when that pool is
>> referenced the copy control job MigrateDCIM-to-Tape cannot figure out the
>> correct device. It would be nice if the copy control job figured this out
>> by looking through records for JobID 4, but that doesn't appear to be
>> happening here.
>>
>> 2. The tape drive appears not to have a suitable tape labeled and
>> appendable, though it's possible that the misconfiguration that caused all
>> this tied up a suitable tape while attempting to use the same resource as
>> both source and destination (valid if you wanted to copy a tape, but not in
>> your case).
>>
>> 3. You might want selection criterion "PoolUncopiedJobs". I suspect the
>> selection tool criterion you've provided will copy jobs matching the mask
>> you provided, no matter whether they've previously been copied or not. Yes,
>> it did match your target job correctly, but might lead to extra job copies
>> later.
>>
>> 4. Make sure that retention matches for original and copy job definitions.
>> I had an issue where, during testing, I set up longer retention A for my
>> intial source jobs (local storage). I made full backups on the local
>> storage file writers. At some point, I decided to lower the retention
>> period, but I DIDNT forcibly purge the initial full backups with longer
>> retention periods. Then I set up copy jobs to upload local jobs to the
>> cloud, with shorter retention period B (at that time matching all other
>> pools in the system, but this new shorter retention was NOT applied to the
>> existing jobs). The copy jobs worked and I walked away. Then, a few months
>> later, the cloud volume retention expired, the copies of the first full
>> backups were pruned from cloud storage... And were then promptly pruned
>> from the cloud on completion, because they inherited their retention period
>> from the start or end date associated with the original job. Meanwhile, the
>> local full backups weren't set to expire anytime soon. Copy after copy was
>> uploaded to the cloud before I figured out the mistake. Oof.
>>
>> 5. As far as I can tell, when a copy control job like MigrateDCIM-to-Tape
>> launches, any selection criterion like PoolUncopiedJobs operates at time of
>> job launch, even if the job waits for other higher priority jobs to finish
>> first. In other words, if you launch your file backup jobs, then 1 minute
>> later launch your copy jobs with a lower priority while the file jobs
>> complete, the only valid targets for the copy job will be last night's file
>> storage jobs, not the yet-to-be-completed jobs from tonight. The solution I
>> found to this problem was to make an admin job that launches the copy jobs
>> using an echo command to bconsole. So the admin job CopyLauncher sits in
>> the queue waiting for file jobs to finish, then once activated it
>> immediately spawns copy control jobs and exits. Those copy control jobs see
>> all tonights completed jobs that are eligible for a copy, and act
>> accordingly.
>>
>>
>> Robert Gerber
>> 402-237-8692
>> [email protected]
>
> Oops, I made an error.
> 
> In point 1, your FileCP pool should point to its own storage resource,
> which from your source job, appears to be "file2"
> 
> I always define the storage at the pool level. Retention is defined
> there too.


Cheers for that Rob.

Lots of things to try out within the config files, and some things to consider
in general.

One of the things I think has muddied the waters a bit for me, has been that
the examples, within the docs, don't use a consistent set of names for things,
as one moves through to the "more advanced" sections, and so, as I started from
the vanilla config files, there's a load of cruft that gets left around, hence
my attempts to redefine extra stanzas for things.

I'm getting the impression that the re-factoring" of the config files, once one
has something that does what's required, will be another big task.

Ta again, I'll see where insight that gets me,
Kevin Buckley


_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users