Re: Stand-alone slices

> 
> > The slicing scheme is retrieved from the first
> > slice, and the catalogue from the last slice.  If the file in
> > question resides completely in the 4th slice, then the restore
> > proceeds.  Otherwise, dar asks me to provide the xth slice and I
> > cancel the operation.  The end result is that I restore all the files
> > in a single slice.
> 
> Yep, but this is possible only if you know which files not to restore
> because they have data located in a missing slice.
> 

I try to restore all of the files in the archive.  If dar prompts me for a slice, then I know that the file I'm trying to restore does not exist wholly in the slice I have.  


> 
> > 
> > My proposition is to replicate my "by hand" method above.  We try to
> > restore all the files listed in the catalogue.  When a missing slice
> > is needed dar should not prompt the user, but silently fail in the
> > restoration and move on to the next file in the catalogue.  This
> > could even be done with an external script providing that you could
> > set dar to automatically exit rather than ask for a missing slice.
> 
> Yes, but even before trying to read data to restore a given file, dar
> must first create the directory entry (inode) then tries to copy data
> from the archive into the just created file. At this time only it may
> fail to proceed due to the lack of slice. which lets in the filesystem a
> empty file [I must double check what dar does to the already created
> file, but I guess it is let in place uncompleted or empty]
> 
> In your idea, you assume that dar tests the data location before
> deciding whether to restore or not a given file. But due to the
> implementation in layers (low layer manages files/slices, above which
> takes place the virtual unsliced data, then above is encryption, then
> again encryption, then at last file's data and catalogue) It's not
> straight forward to have the possibility to do this way. In particular,
> if the start of a Data is possible to locate, due to compression and
> encryption, it is not easy instead to know where is located its end
> (same slice or next slice that may be missing for example). Thus you may
> end with a partially restored file, even having this test implemented
> before the restoration of each file.
> 

I didn't realize that I had gotten so many empty files created in the directory.  This is a problem.  But it does seem like it would be fairly simple to let the user specify that the files should be deleted if the recovery is not successful.  

The other problem would be a file that spans multiple slices and gets partially restored.  Would this ever occur more than once per slice?  

> > 
> > A more efficient way would be to allow the catalogue to be trimmed to
> > the contents of a specific slice.  That would eliminate the repeated
> > attempts and failures, but it would require some extra code and would
> > have to be regenerated in the event of a reslice.
> 
> Of course that would require extra code, thus that's an idea to think
> about. However what I like is the possibility to transparently reslice
> (using dar_xform) an archive without having to rebuild an extracted
> catalogue (you can also slice or reslice an extracted catalogue and keep
> using it against the same original or resliced archive).
> 
> IMHO it is much more complicated to correctly trim a extracted catalogue
> to fit a given slice than to emulate a missing slice (over the fact it
> does not really match the current software design). The only thing that
> must be seen more deeply within the current dev. implementation is the
> possibility to test the start of data location before restoring a given
> file. I guess (and hope) this should be possible to implement cleanly in
> coherence with the existing software architecture... Still remains the
> case of file that could not be restore entirely and the need of a new
> optional feature to let the user decide whether to remove them from the
> filesystem or let them uncompleted restored.
> 

The testing of the data location would be good.  Really, couldn't you just have an additional prompt for incomplete files.  "dar: provide the correct slice"   "user: I don't have it... ESC"  "dar:file not complete do you want to delete it?" 

As for the trimming of the file, I have only 1 idea that may not be any good.  I'm not sure how the "on-the-fly"  isolation works.  I have seen that the catalogue file is not written until all slices are finished.  But I must assume that the data is collected as the slices are made.  Therefore, at the end of the xth slice, you would have all the data for the first x slices and no more.  So it might be possible to insert a flag when each slice is finished into an on-the-fly catalogue.  This is more speculative since I don't know how it's implemented. 

Bill

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword