Re: Suggestions for secondary cloud backups
Chris Wilkinson <[email protected]> Fri, 3 Jul 2026 17:08:34 +0100
| Newsgroups | gmane.comp.bacula.user |
|---|---|
| Message-ID | <CAHUKsLJh0eA8Y8z7Hy3Li+_ROBovS_TxSiZFq8ic0=p0SQgzhQ@mail.gmail.com> |
--===============8735588554063392005== Content-Type: multipart/alternative; boundary="000000000000bd902b0655b7239e" --000000000000bd902b0655b7239e Content-Type: text/plain; charset="UTF-8" Why a copy job? Is it because you want to backup to cloud when the clients may not be online? I just use a standard scheduled backup job to cloud storage, B2 in my case, with the usual full/diff/incremental pools. My clients are on 24/7. I suppose this means that the main and cloud backups may not be identical but that's not an issue for me. I did find that upload occasionally fails so I use a run after script to upload the cache. Mostly it doesn't upload anything but on the occasions when upload fails, it corrects itself. B2 occasionally reports no slots available so this gets over that. -Chris- On Fri, 3 Jul 2026, 16:02 Andrea Venturoli, <[email protected]> wrote: > Hello. > > I have a setup where I back up several machines to a local NAS. > Now, in order to avoid having data and backups in the same building, I'd > like to add cloud backups (to an S3 storage). > > For most machines backups are monthly Fulls, weekly Diffs and daily Incs > (although a couple only have monthly fulls); I'like to copy Fulls and > Diffs (no Incs) to the cloud. > > > > I came up with a solution using copy jobs like the following > (unimportant options redacted): > > > Pool { > > Name=Full > > } > > Pool { > > Name=Diff > > } > > Pool { > > Name=Inc > > } > > Job { > > Name=BackupFoo > > Schedule=NightlyFDIFullOnSun > > JobDefs="DefaultJob" > > FileSet=Foo > > Client=Foo-fd > > Full Backup Pool=Full > > Differential Backup Pool=Diff > > Incremental Backup Pool=Inc > > } > > Job { > > Name=CloudBackupFoo > > Schedule=Weekly > > JobDefs="DefaultJob" > > FileSet=Foo > > Type=Copy > > Selection Type=SQLQuery > > Selection Pattern="SELECT jobid FROM job WHERE name='BackupFoo' AND > (level='F' OR level='D') AND jobstatus='T' AND poolid!=14 AND poolid!=15 > ORDER BY endtime DESC LIMIT 1" > > Full Backup Pool=Full > > Differential Backup Pool=Diff > > } > > This sort of works, but there are a few problems. > > A) All copy jobs end up in the "CloudFull" pool; no matter if they were > a copy of a full or diff job; "CloudDiff" is never used. > This will prevent keeping fulls for, say, 4 months, and diffs for a > shorter time, thus wasting cloud space. > > B) If host "foo" does a full, the full gets copied; but if it skips the > next diff, the full will be copied twice, thus, again, wasting space for > nothing. > > > > Another approach I though of is using the "Run" parameter in the Job > definition. > Perhaps I don't understand this fully, but: > A) It would run both jobs (local and clouds) on the client and the > client might be a PC or notebook that is turned off/carried away; so I'd > really like to run only the local job on it and let the storage daemon > do the cloud one; > B) it would run the cloud job immediately, thus wasting daytime > bandwidth. It would be a lot better if it ran at night (but, again, it > must be a copy job then, so as not to require the client). > > > > Any suggestion? > > bye & Thanks > av. > > > _______________________________________________ > Bacula-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/bacula-users > --000000000000bd902b0655b7239e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div><div>Why a copy job?</div><div dir=3D"auto"><br></di= v><div dir=3D"auto">Is it because you want to backup to cloud when the clie= nts may not be online?</div><div dir=3D"auto"><br></div><div dir=3D"auto">I= just use a standard scheduled backup job to cloud storage, B2 in my case, = with the usual full/diff/incremental pools. My clients are on 24/7.</div><d= iv dir=3D"auto"><br></div><div dir=3D"auto">I suppose this means that the m= ain and cloud backups may not be identical but that's not an issue for = me.</div><div dir=3D"auto"><br></div><div dir=3D"auto">I did find that uplo= ad occasionally fails so I use a run after script to upload the cache. Most= ly it doesn't upload anything but on the occasions when upload fails, i= t corrects itself. B2 occasionally reports no slots available so this gets = over that.</div><div dir=3D"auto"><br></div><div data-smartmail=3D"gmail_si= gnature">-Chris-</div><br><div class=3D"gmail_quote gmail_quote_container">= <div dir=3D"ltr" class=3D"gmail_attr">On Fri, 3 Jul 2026, 16:02 Andrea Vent= uroli, <<a href=3D"mailto:[email protected]">[email protected]</a>> wrote:<= br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde= r-left:1px #ccc solid;padding-left:1ex">Hello.<br> <br> I have a setup where I back up several machines to a local NAS.<br> Now, in order to avoid having data and backups in the same building, I'= d <br> like to add cloud backups (to an S3 storage).<br> <br> For most machines backups are monthly Fulls, weekly Diffs and daily Incs <b= r> (although a couple only have monthly fulls); I'like to copy Fulls and <= br> Diffs (no Incs) to the cloud.<br> <br> <br> <br> I came up with a solution using copy jobs like the following <br> (unimportant options redacted):<br> <br> > Pool {<br> >=C2=A0 =C2=A0Name=3DFull<br> > }<br> > Pool {<br> >=C2=A0 =C2=A0Name=3DDiff<br> > }<br> > Pool {<br> >=C2=A0 =C2=A0Name=3DInc<br> > }<br> > Job {<br> >=C2=A0 =C2=A0Name=3DBackupFoo<br> >=C2=A0 =C2=A0Schedule=3DNightlyFDIFullOnSun<br> >=C2=A0 =C2=A0JobDefs=3D"DefaultJob"<br> >=C2=A0 =C2=A0FileSet=3DFoo<br> >=C2=A0 =C2=A0Client=3DFoo-fd<br> >=C2=A0 =C2=A0Full Backup Pool=3DFull<br> >=C2=A0 =C2=A0Differential Backup Pool=3DDiff<br> >=C2=A0 =C2=A0Incremental Backup Pool=3DInc<br> > }<br> > Job {<br> >=C2=A0 =C2=A0Name=3DCloudBackupFoo<br> >=C2=A0 =C2=A0Schedule=3DWeekly<br> >=C2=A0 =C2=A0JobDefs=3D"DefaultJob"<br> >=C2=A0 =C2=A0FileSet=3DFoo<br> >=C2=A0 =C2=A0Type=3DCopy<br> >=C2=A0 =C2=A0Selection Type=3DSQLQuery<br> >=C2=A0 =C2=A0Selection Pattern=3D"SELECT jobid FROM job WHERE name= =3D'BackupFoo' AND (level=3D'F' OR level=3D'D') AND= jobstatus=3D'T' AND poolid!=3D14 AND poolid!=3D15 ORDER BY endtime= DESC LIMIT 1"<br> >=C2=A0 =C2=A0Full Backup Pool=3DFull<br> >=C2=A0 =C2=A0Differential Backup Pool=3DDiff<br> > }<br> <br> This sort of works, but there are a few problems.<br> <br> A) All copy jobs end up in the "CloudFull" pool; no matter if the= y were <br> a copy of a full or diff job; "CloudDiff" is never used.<br> This will prevent keeping fulls for, say, 4 months, and diffs for a <br> shorter time, thus wasting cloud space.<br> <br> B) If host "foo" does a full, the full gets copied; but if it ski= ps the <br> next diff, the full will be copied twice, thus, again, wasting space for <b= r> nothing.<br> <br> <br> <br> Another approach I though of is using the "Run" parameter in the = Job <br> definition.<br> Perhaps I don't understand this fully, but:<br> A) It would run both jobs (local and clouds) on the client and the <br> client might be a PC or notebook that is turned off/carried away; so I'= d <br> really like to run only the local job on it and let the storage daemon <br> do the cloud one;<br> B) it would run the cloud job immediately, thus wasting daytime <br> bandwidth. It would be a lot better if it ran at night (but, again, it <br> must be a copy job then, so as not to require the client).<br> <br> <br> <br> Any suggestion?<br> <br> =C2=A0 bye & Thanks<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 av.<br> <br> <br> _______________________________________________<br> Bacula-users mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank" rel= =3D"noreferrer">[email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/bacula-users" rel= =3D"noreferrer noreferrer" target=3D"_blank">https://lists.sourceforge.net/= lists/listinfo/bacula-users</a><br> </blockquote></div></div></div> --000000000000bd902b0655b7239e-- --===============8735588554063392005== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============8735588554063392005== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Bacula-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-users --===============8735588554063392005==--