Re: Remove unique files from destination, and a few other issues

On 12/02/2021 23:04, tgrolier1992 via Dar-support wrote:
> Hi,

Hi,

> 
> I recently installed dar under Ubuntu Server 20 and was trying to image
> an NTFS Windows 10 HDD with it. I mounted the drive under Ubuntu and ran
> dar to create a full backup of the disk. On some files, I get an I/O
> error saying certain items in '/win10hdd/Program
> Files/WindowsApps/Microsoft.OneConnect_1.1607.0_x64__8wekyb3d8bbwe/',
> like 'clrcompression.dll' The disk has no bad sectors, and chkdsk under
> Windows says it's fine. I can view the files just fine under Windows,
> but not Ubuntu.
> 
> Oddly enough, on things like 'OneConnect.exe' (also in that directory
> mentioned above), the full dar backup I made says it copied the symlink
> just fine. Doing cat OneConnect.exe gives "file not found", but under
> Windows it's fully usable and looks like a regular file.
> 
> The full dar backup I made exited with code 5. With that being said, is
> the dar image still good for restoring? I would like to note that every
> file dar could not copy, I do not need anyway, I am only keeping them so
> Windows does not break.

Difficult to answer this question. My guess is that the best answer will
be given by proceeding to a restoration in a temporary server or virtual
machine or equivalent and see whether the missing files dar could not
save due to these I/O error met under Ubuntu lead to a viable windows
system or not...

> 
> Moving on to the other question I have, as a test, I renamed some files
> on the Windows 10 HDD and also deleted some, made new directories (one
> empty and one full with a new file), and made a new file in the Windows
> folder. I tried restoring the full dar image made earlier to the drive
> to see what would happen, and it appears that every file was
> overwritten. 
This is not the default behavior, you have to tell dar to do so for
overwriting to take place.

> Is it necessary for dar to overwrite unchanged files? 
Well, I understand your question as to not restore a file if it has not
changed, right? To know if a file has changed or not you have to read it
and at the same time compare it with the copy stored in the backup
(which may lead to uncompressing and unciphering such data). If the file
had not changed you will have done the same amount of read+write
operations as if you have not tested it before restoring... But if the
file had changed, thing will be even slower, as at the first difference
met, you will have to restart not reading but restoring this time (and
uncompressing/unciphering again what has been done in the previous step).

You have the -r option that let dar overwriting only more recent files.
But, more generally, that's up to you to decide what dar will overwrite
or not (as said, dar does never overwrites by default), you have filter
mechanismes (-I,-X,-P,-g, -[, -] options), -r option also and other ways
to decide what is to be considered for restoration, so you can knowning
the context (something dar alone cannot guess) reduce the restoration time.

> This
> takes a very long time to do and would quickly wear out an SSD. Or, am I
> wrong, and dar is really hashing the files on the disk and comparing
> them with the ones in the image, to see what needs to be restored and
> what can be skipped? (I prefer this option)

Doing so would take more time than simply overwriting an existing file.

> 
> Additionally, I noticed the new files and directories I had created were
> not removed from the destination after the image was restored. I looked
> all over the man pages, tutorial, mini-howto and could not figure out
> how to do this.

A full backup is expected to be restored on an empty volume/disk, a
differential backup is expected to be restored over this volume after
the full backup has been restored.

> 
> Does dar allow this to work? The only other alternative I have thought
> to this, that I have not done, would be to remove _all_ files from the
> HDD and then restore the full image back, which again would be stressful
> on an SSD.

I assume you have a *full* backup to restore on your SSD and this one
has some file located on it. You don't want to erase this SSD because
you fear to shorten its life, this is understandable, but probably a bit
paranoid ;^)

What you could try is this:

- make a snapshot of the SSD before the restoration:

    dar -c snapshot -A + -R /mnt/SSD

- merge the full backup and the snapshot keeping the files from the
backup upon file conflict (unless the file in place is more recent of
whatever makes sens for your context, see overwriting policy for more
complex use cases) let's call the result the "merge1" backup:

   dar -+ merge1 -A full -@ snapshot -ak -/ 'Pp'

- merge again using merge1 with the same snapshot, but this time any
conflict where the "in place" entry is not a saved inode (thus the file
comes from the snapshot and was absent from the original backup),
replace it by a "deleted" entry. Let's call the result merge2 backup

  dar -+ merge2 -A merge1 -@ snapshot -ak -/ '{!S}[Rr]'

- restore merge2 with [-w option] or without overwiting [-n option to
avoid the user interactions in case of conflict] depending on your
context/need. Dar will then restore files and remove all files that were
present before the restoration and that were not present in the original
backup:

  dar -x merge2 -R /mnt/SSD

This is the theory, you have to try it first with a small set of files
and tweek the overwriting polic if necessary to take into account your
context and needs!

> 
> 
> This is my first time using dar, so I apologize if some of these
> questions are cluttered or noob-sounding.

The only wrong questions are those that one does not ask! ;-)

> 
> Thank you,
> Thomas
> 
> 

Cheers,
Denis

_______________________________________________
Dar-support mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dar-support