Re: burning changing files

Could you please open a wish on bugs.kde.org regarding the file monitoring 
issue so I won't forget it.

On Monday 25 June 2007 22:56:24 dagurasu dagurasu wrote:
> This is getting off the K3b topic, but I thought I'd share my followup
> findings a bit.  It's possible to stream a tar to an ISO using growsiofs
> --stream-media-size option (see mkisofs man page).  I have problems with
> this with large files... not sure why yet.. maybe filesize limit.  With
> small files this allowed me to mount the disc and navigate through the tar
> with tar capabale browsers, specifically Konqueror.
>
> You can also pipe the tar directly to the dvd.  Konqueror can read the tar
> by opening the device link directly, but for some reason it breaks.  It
> usually only shows me one or two directories in the tar when there are
> many.  I've tried with star too, same result.  However, I do get tars that
> work even for big files... they just have to copied or extracted to use
> them... better than corrupted data.
>
> Now all I need is a good verification method.  You can use the -d switch in
> tar to verify after the fact.  However, it only verifies files that are in
> the tar, not ones that never made it there (which I've had happen).  What I
> decided to do is construct the MD5 as the tar is being piped to disk and
> then check against the disk file later.  This only checks the burn, not the
> read, but I'm not usually worried about my systems read stability, could be
> an issue if you have bad ram or a failing drive though so this verification
> isn't for a purist.  To me the most important thing is for backup to be
> easy.  If I back up often, then even if I miss the bad hd symptoms... I'll
> have data often enough not to lose much.  Maybe I should worry about
> tar/star corrupting data though.. this won't get caught either.
>
> This method has many advantages.  It doesn't have to reread the data from
> the hard disk, so it's faster and doesn't keep tying up your hd while
> you're working.  I'm trying to avoid temporary disk space so I don't have a
> temporary copy of the tar and even if I remade one from disk, the point of
> my problem is that files on disk are changing... so really to check against
> the disk files, the only way is to check md5's one file at a time.  This is
> VERY slow even off a standard ISO image, and when comparing to a tar, not
> so easy to even see how to do without extracting the whole tar first
> (extracting one file at a time is impossibly slow)... and thus defeating
> all the points.
>
> There's a problem though.  The optical device data stream doesn't end
> exactly where the tar pipe ended so an MD5 sum comparisson won't match
> up... unless you dd the correct number of bytes off the device... so you
> have to pay attention while piping the tar to disk and somehow get the
> exact byte length.  dd outputs this, but I must admit my ignorance that I
> don't know how to intercept it, because the output is not standard output
> and is not standard error either.  Instead I used star's standard error
> reporting (too bad since using dd would allow choice of archiver... I'll
> work on it).    Finally, you get a simple pass or fail.  Even if you did
> compare files one at a time after the burn, you'd have to check by eye that
> the ones that did change are one's you would expect to change.
>
> here's the scriptlet:
>
> # The tar will pipe to growisofs, but will also tee to a named pipe that
> will be used # to read the md5.. Make the named pipe:
> rm tfifo
> mkfifo tfifo
>
> # first start md5sum to read off the named pipe
> # also dump the star standard error ouput to disk so we can use the byte
> length info from it md5sum -b tfifo  > md5original & star -c $@ 2>
> tarfilesize | tee tfifo \
>
> |growisofs  -Z $dev=/dev/fd/0
>
> #parse the star output to get the byte length; toolazy to remember gawk
> right now. length=`head -n 1 tarfilesize|cut -d"=" -f2|cut -d"." -f1|cut
> -d" " -f2` echo file length $length kB
> rm tarfilesize
>
> #now read back the right number of bytes from the burned disk to check the
> MD5 dd if=$dev bs=1024 count=$length |md5sum >md5copy
>
> echo doing MD5 check:
> mdo=`head -n 1 md5original|cut -d" " -f1`
> echo "original: $mdo"
> mdc=`head -n 1 md5copy|cut -d" " -f1`
> echo "copy: $mdc"
> echo
> #compare the disk md5 to the tar pipe:
> if [[ $mdo == $mdc ]]; then
> echo passed
> else
> echo failed
> fi
> rm tfifo
>
> _________________________________________________________________
> Live Earth is coming.  Learn more about the hottest summer event - only on
> MSN. http://liveearth.msn.com?source=msntaglineliveearthwlm
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> K3b-user mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/k3b-user



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/