Re: Stand-alone slices

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

[email protected] wrote:
> My hard drive just crashed, but luckily I didn't lose any important
> data as it was on another hard drive.  So I'm trying to use dar and
> par2 to create backups that will span multiple dvds.  This is not a
> support question, however.
> 
> Basically, I see that individual slices are pretty much worthless
> without the catalogue.  So from what I understand, the catalogue
> holds a list of all the files in the archive and their offsets in the
> large virtual file.  Then when you go to actually restore a file, the
> virtual offset is translated into a slice number and slice offset.
> 
> Furthermore, I see that you can isolate the catalogue and encode no
> data.
> 
> Finally, I read that you can extract files individually, and you will
> only require the slice(s) where the file actually resides in the
> archive.  You do, however, appear to need the catalogue from the
> first slice to determine what files are in the archive and where they
> fall in the virtual file (and hence the slices).
> 
> So I am wondering if a semi-acceptable solution to the stand-alone
> slice issue could be to provide a means of isolating the catalogue so
> that the user could optionally write it as a separate file to each
> piece of media.  Then, when a certain file was called for, the
> virtual address would translate into a slice offset.  The program
> would prompt the user to provide the correct slice.  That way, if I
> lost all my backup dvd's except for 1, then I would still be able to
> extract any files from that single slice, and I would just be out of
> luck for the rest.
> 
> I'm sure I'm missing something here, but I don't see the problem with
> this.  (of course it wouldn't be as space efficient, but then again
> neither is using par2).

Hello Bill,

The currently under development version of dar already allows one to
extract a catalogue (contents table) of a given (sliced or not) archive
into a separate file. At the difference of the current released
versions, the under development can be told to use that separated
catalogue as backup of the internal catalogue of a given archive (the
new format of extracted catalogue contains the offset of files in the
virtually sliced archive, information which is not present in the
current extracted catalogues).

The second feature that will help here and which is also already
available in the current development, is that dar does no more require
to read the first slice to get the slicing scheme, this information is
now available in any slice (each slice now has this information in the
slice header, which makes them a bit bigger but not that much).

Actually dar reads, the first slice to get the slicing scheme (first
slice size, middle slice size), and the reads the last slice for the
catalogue an to know the archive layout and contents. In current
development version, dar thus only needs the last slice to get both the
slicing scheme and the table of content of an archive.

If I correctly understand you arguments, it would just need an
additional and optional switch for dar to consider a given subset of
slice as lost and rely on the first available slice and an given
extracted catalogue to restore as much file as possible from the remain
slices.

OK, this is for the feature level (its always simple to describe what
should be done, things get less trivial when its time to implement them
over/inside an existing code).

Now, seen the implementation of dar, it would not be too much
complicated in the slicing management code (named sar.hpp/sar.cpp) to
emulate the known to be missing slices by zeroed virtual files. letting
upper layer of the implementation fail upon incoherent CRC for the
files which data is located in the missing slices. However, this would
let the resulting filesystem populated by wrong zeroed files. A
additional and independent feature that would allow dar to remove files
that failed CRC test once restored would solve this issue. However, the
problem with this implementation is its inefficiency. Restoring zeroed
files then removing them afterward ...

Sorry to write as I thing, but an improvement would be to simulate the
missing files by I/O error (any read from theses would make dar generate
a fake I/O error by the slicing part of the implementation --- aka
sar.hpp/sar.cpp). The upper layer (archive layer) would then restore
empty files (which length is zero) for most files except those starting
in a remain file and having the rest of their data in a missing slice.
The additional feature could be adapted to either remove only zero
length generated files upon faked I/O error or remove any file that
failed to be properly restored.

I have no better idea so far to implement this interesting feature, I
agree. Seen the drawbacks, what's your objections, ideas?

> 
> Bill

Regards,
Denis.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJhcHZpC5CI8gYGlIRAqRxAJ4tOMtNLB4GSGmiavMfqJqdq3QGUwCgnwCN
TRKSEjeJv/jJyDVk5e9f9m8=
=JQDm
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword