Re: Questions about delta encoding implementation and virtual files
egoitz--- via Bacula-devel <[email protected]> Mon, 7 Mar 2022 21:59:03 +0100
| Newsgroups | gmane.comp.sysutils.backup.bacula.devel |
|---|---|
| Message-ID | <[email protected]> |
No one 😀 Any help wouldvery highly appreciated… really!! Some parts… are not 100% clear in the doc… and I would be very proud of sharing my work 🙂🙂👍 Cheers! Egoitz, > El 7 mar 2022, a las 12:45, egoitz--- via Bacula-devel <[email protected]> escribió: > > > ATENCION: Este correo se ha enviado desde fuera de la organización. No pinche en los enlaces ni abra los adjuntos a no ser que reconozca el remitente y sepa que el contenido es seguro. > > Hi!, > > > > After digging in plugin building documentation and checking the provided examples, I have some doubts I have not been able to clarify by my own. I describe them below in case you could give me a hand :). I would be very thankful if you could help me a little clarifying this doubts :) > > > > - I'm working in trying to create an open source version of the delta encoding plugin by using the bacula-fd plugin api. When working on it I have seen Bacula's source is aware of delta and delta file sequentiation. I have seen for instance, even a .bvfs command exists for showing deltas of a file id. But, what I have not found is that Bacula works on that Delta files generation (patch generation, signatures, etc...). I assume that Bacula in the non-fd part, acts just as a just delta file holder keeping the files and stores the patch sequentiation just that. Bacula keeps records of deltas in database (and file storages) but only fd works with them (with probably a library like librsync in the delta plugin) in the sense of applying patches over an original file or even generating deltas when backup. Am I wrong?. Was just for understanding the nice work done and what's already written and free in Bacula's source for this purpose. > > > > - By the way, I have one question about virtual files. I have not seen very clear (perhaps my problem as don't understand it) how to work with them. I understand the concept, but have not seen a clear example of how for instance in the backup you create a virtual file, how do you see it in bvfs and finally... what you get after restoring. In page 36/146 of Bacula 11 for developers pdf, you say "This will create a virtual file." but really you are entering in the structure : > > sp->type = FT_REG; > sp->statp.st_mode = 0700 | S_IFREG; > > FT_REG and S_IFREG both are for regular files.... what exactly causes a virtual file to be created?. Perhaps st_size -1?. > > Are they relevant for what I'm trying to do?. It seems Bacula handles delta sequentiation so... perhaps for this purpose I shouldn't need "virtual files"?. > > > > - I'm planning to implement delta encoding by checking the previous day file signature done by librsync. Instead of looking at the filesystem it would be nice if I could take a look at that signature in the last backup done (yesterday backup). Could it be possible in some manner, that if I see a file passed in EventHandleBackupFile() to check if yesterdays signature exists in the backup of yesterday, and then read the yesterday signature from the own backup?. I mean, instead of having to leave the signature in the being backed server's filesystem. > > > > - The last one :) . For restoring, and for the code seen (for instance in insert_missing_delta()) I assume Bacula detects we are restoring a delta compressed file. Then I assume Bacula restores apart from the own initial file, patches to arrive to the day we want to restore to. Am I wrong?. Perhaps later in a post-restore job I could run a shell script that tries to find patches pending to be applied to a parent file. I suppose then I could apply and the backup would become finally restored. Does some other more elegant way you could advise me?. > > > > Thank you so much really, > > Best regards, > > > > > > <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset= > =3DUTF-8" /></head><body style=3D'font-size: 10pt; font-family: Verdana,Gen= > eva,sans-serif'> > <p>Hi!,</p> > <p><br /></p> > <p>After digging in plugin building documentation and checking the provided= > examples, I have some doubts I have not been able to clarify by my own. I = > describe them below in case you could give me a hand :). I would be very th= > ankful if you could help me a little clarifying this doubts :)</p> > <p><br /></p> > <p>- I'm working in trying to create an open source version of the delta en= > coding plugin by using the bacula-fd plugin api. When working on it I have = > seen Bacula's source is aware of delta and delta file sequentiation. I have= > seen for instance, even a .bvfs command exists for showing deltas of a fil= > e id. But, what I have not found is that Bacula works on that Delta f= > iles generation (patch generation, signatures, etc...). I assume that Bacul= > a in the non-fd part, acts just as a just delta file holder keeping the fil= > es and stores the patch sequentiation just that. Bacula keeps records of de= > ltas in database (and file storages) but only fd works with them (with prob= > ably a library like librsync in the delta plugin) in the sense of applying = > patches over an original file or even generating deltas when backup. Am I w= > rong?. Was just for understanding the nice work done and what's already wri= > tten and free in Bacula's source for this purpose.</p> > <p><br /></p> > <p>- By the way, I have one question about virtual files. I have not seen v= > ery clear (perhaps my problem as don't understand it) how to work with them= > =2E I understand the concept, but have not seen a clear example of how for = > instance in the backup you create a virtual file, how do you see it in bvfs= > and finally... what you get after restoring. In page 36/146 of Bacula 11 f= > or developers pdf, you say "This will create a virtual file." but really yo= > u are entering in the structure :</p> > <p>sp->type =3D FT_REG;<br />sp->statp.st_mode =3D 0700 | S_IFREG;</p= >> > <p>FT_REG and S_IFREG both are for regular files.... what exactly causes a = > virtual file to be created?. Perhaps st_size -1?.</p> > <p>Are they relevant for what I'm trying to do?. It seems Bacula handles de= > lta sequentiation so... perhaps for this purpose I shouldn't need "virtual = > files"?.</p> > <p><br /></p> > <p>- I'm planning to implement delta encoding by checking the previous day = > file signature done by librsync. Instead of looking at the filesystem it wo= > uld be nice if I could take a look at that signature in the last backup don= > e (yesterday backup). Could it be possible in some manner, that if I see a = > file passed in EventHandleBackupFile() to check if yesterdays signature exi= > sts in the backup of yesterday, and then read the yesterday signature from = > the own backup?. I mean, instead of having to leave the signature in the be= > ing backed server's filesystem.</p> > <p><br /></p> > <p>- The last one :) . For restoring, and for the code seen (for instance i= > n insert_missing_delta()) I assume Bacula detects we are restoring a delta = > compressed file. Then I assume Bacula restores apart from the own initial f= > ile, patches to arrive to the day we want to restore to. Am I wrong?. Perha= > ps later in a post-restore job I could run a shell script that tries to fin= > d patches pending to be applied to a parent file. I suppose then I could ap= > ply and the backup would become finally restored. Does some other more eleg= > ant way you could advise me?.</p> > <p><br /></p> > <p>Thank you so much really,</p> > <p>Best regards,</p> > <p><br /></p> > <p><br /></p> > > </body></html> > ATENCION > ATENCION > ATENCION!!! Este correo se ha enviado desde fuera de la organizacion. No pinche en los enlaces ni abra los adjuntos a no ser que reconozca el remitente y sepa que el contenido es seguro. > > _______________________________________________ > Bacula-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/bacula-devel > > ATENCION: Este correo se ha enviado desde fuera de la organización. No pinche en los enlaces ni abra los adjuntos a no ser que reconozca el remitente y sepa que el contenido es seguro. > _______________________________________________ Bacula-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-devel