Re: Stand-alone slices

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

[email protected] wrote:
> Denis,
> 
> Thanks for the reply.  First let me restate what I have learned from
> your reply.
> 
> In the development version: - The slicing scheme is recoverable from
> any slice as it is encoded in all of them

yes,

> - The catalogue can now be
> used as a backup to the one found in the last slice (contains offsets
> in virtual file).

yes,

> - The only information needed now is from the last
> slice to attempt a recovery

No, you do not need anymore the last slice: If you have one slice (last
or not) from a given archive and also have an extracted catalogue (using
current development), you can (in theory) restore any file which data is
entirely stored in that slice. In practice dar will ask for the missing
slices, up to the user to provide something that dar accepts as such (if
the filename has an adequate slice header, like for example copying
slice N+1 from slice N, it will work). However CRC check will fail for
the concerned files but contained in slice N+1 but dar will pass them
and continue with next slice)

> 
> I have "emulated" something close to a standalone slice in the
> following way (using the stable release).  I create a sliced archive
> (5 slices), and I rename the 2nd and 3rd slices so that they are not
> found.  This leaves just the 1st, 4th, and last slices.  For each
> file in the original backed-up directory, I attempt a restore using
> "dar ... -g file". 

OK, this assumes that you know where each of the files pointed to by -g
option are stored.

> The slicing scheme is retrieved from the first
> slice, and the catalogue from the last slice.  If the file in
> question resides completely in the 4th slice, then the restore
> proceeds.  Otherwise, dar asks me to provide the xth slice and I
> cancel the operation.  The end result is that I restore all the files
> in a single slice.

Yep, but this is possible only if you know which files not to restore
because they have data located in a missing slice.

> 
> Now... you say that the new version would not require the first slice
> to get the slicing scheme.

Yes, (this is even already implemented and documented, you can give it a
try if you like).  ;-) But avoid using it for real backup.

> So that takes us down to just the last
> slice and the 4th slice.

or just the 4th slice if you have an isolated catalogue.

>  Furthermore, we don't really need the last
> slice in its entireity, but merely the catalogue and the slicing
> scheme that are encoded.  We can get the slicing scheme from the 4th
> slice and isolate the catalogue into a much smaller file. 

yes,

> So let's
> drop the last slice and replace it with the isolated catalogue.  That
> leaves just the 4th and the catalogue.  So far no changes in the code
> at all.

Exactly, all this is already implemented in the current development version.

> 
> My proposition is to replicate my "by hand" method above.  We try to
> restore all the files listed in the catalogue.  When a missing slice
> is needed dar should not prompt the user, but silently fail in the
> restoration and move on to the next file in the catalogue.  This
> could even be done with an external script providing that you could
> set dar to automatically exit rather than ask for a missing slice.

Yes, but even before trying to read data to restore a given file, dar
must first create the directory entry (inode) then tries to copy data
from the archive into the just created file. At this time only it may
fail to proceed due to the lack of slice. which lets in the filesystem a
empty file [I must double check what dar does to the already created
file, but I guess it is let in place uncompleted or empty]

In your idea, you assume that dar tests the data location before
deciding whether to restore or not a given file. But due to the
implementation in layers (low layer manages files/slices, above which
takes place the virtual unsliced data, then above is encryption, then
again encryption, then at last file's data and catalogue) It's not
straight forward to have the possibility to do this way. In particular,
if the start of a Data is possible to locate, due to compression and
encryption, it is not easy instead to know where is located its end
(same slice or next slice that may be missing for example). Thus you may
end with a partially restored file, even having this test implemented
before the restoration of each file.

> 
> A more efficient way would be to allow the catalogue to be trimmed to
> the contents of a specific slice.  That would eliminate the repeated
> attempts and failures, but it would require some extra code and would
> have to be regenerated in the event of a reslice.

Of course that would require extra code, thus that's an idea to think
about. However what I like is the possibility to transparently reslice
(using dar_xform) an archive without having to rebuild an extracted
catalogue (you can also slice or reslice an extracted catalogue and keep
using it against the same original or resliced archive).

IMHO it is much more complicated to correctly trim a extracted catalogue
to fit a given slice than to emulate a missing slice (over the fact it
does not really match the current software design). The only thing that
must be seen more deeply within the current dev. implementation is the
possibility to test the start of data location before restoring a given
file. I guess (and hope) this should be possible to implement cleanly in
coherence with the existing software architecture... Still remains the
case of file that could not be restore entirely and the need of a new
optional feature to let the user decide whether to remove them from the
filesystem or let them uncompleted restored.

> 
> What do you think?
> 
> Bill

Regards,
Denis.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJhhjSpC5CI8gYGlIRAvJoAKCDj17lFzxVT7sbhfSi/Pr/A1Oq1ACbBszp
C4fpcL7gjkceK6g/Mkm3Nag=
=U5/l
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword