Re: HD synch?

Paul wrote:
> Robert Baer wrote:
>>   Must be a law, data grows.
>>
>>   I never image a drive for backup; I clone it.
>>   AFAIK the process takes the same amount of time.
>>   But usage differs greatly; a clone is immediately usable, like the 
>> original.
>>   An image must be "restored" to yet another place/media, and _that_ 
>> restoration then is usable.
>>
>>   Now as data grows, time needed for backup grows, now eating 
>> TERRORbytes of space and eating hourzz of time.
>>
>>
>>   I would love an edge by syncing the copy to the original on a 
>> regular basis instead of cloning on a regular basis.
>>
>>   A program in the background could be ideal; copy incremental data as 
>> and when needed, or chunk away on a regular basis.
>>
>>   Suggestions, please?
>>
>>
>>     Thanks, R. Baer
> 
> When you Image a drive (convert a whole drive to a .container file),
> there is
> 
>     Full
>     Differential
>     Incremental
> 
> It is the incremental option,
> 
>     Full Incr Incr Incr...    Full Incr Incr Incr...
> 
> that gives the small addition to the Full each day.
> 
> During restoration to the third day, you'd use three files
> 
>     Full Incr Incr
> 
> to get to the "state" of the second Incr. Because an Incr
> is never rewritten, your file "notes.txt" could be stored
> in all three files, and the metadata knows that the
> third one is the one to keep during a restore.
> 
> On a backup product that has Incremental Forever, there
> don't have to be quite as many file groups. The Full
> and a set of Incr, can be squashed together (synthetically)
> to make a new Full, ready to have more Incr collected.
> 
> *******
> 
> The other type is Differential. Backups are measured
> against the Full. Each Differential tends to stand on its
> own two legs.
> 
>     Full Diff1 Diff2 Diff3...  Full Diff1 Diff2 Diff3...
> 
> This gives a more bulky operation each day, but still
> well short of what a Full takes.
> 
> During restoration to the third day, you'd use two files
> 
>     Full       Diff2
> 
> and that would return you to the state of Diff2.
> 
> *******
> 
> On Windows, both of those sequences can be done "hot"
> during the backup. VSS (Volume Shadow Service) allows
> tracking of the work to be done. This is unlike the
> old "Ghost" program, where a reboot and an offline
> backup was done. Certain files, like pagefile.sys
> and Windows.edb are not captured, but this does not
> affect system operation. Windows.edb cannot be captured,
> because the subsystem that has it open (Search Indexer)
> refuses to quiesce it on demand.
> 
> Incremental backup costs money. Differential backup
> can be acquired for free.
> 
> On a Windows 10 system, an Incremental might be 1GB per
> day, as there is some maintenance activity that creates
> file differences. Your "notes.txt" file is a small
> contributor to the size of the Incremental file collected.
> 
> *******
> 
> An example of cloning, where only new files are written,
> would be something like Linux RSYNC. On Windows, Robocopy
> or Richcopy might do it. The problem with some of these
> solutions, is whether they handle the metadata properly
> or not. The backup softwares, generally they seem to do a
> good job. That's because the metadata on a volume, is also
> a file, and so the metadata can be backed up as an incremental
> file change.
> 
> But for programs which might choose to record files
> individually, the onus would be on the developer to make
> sure that during restoration or recovery, all aspects of
> the file (ownership, permissions, ACLs) are restored properly.
> And for crappy situations like Windows 10 with reparse points,
> this requires the developer understand everything that
> Microsoft has done. At one time, Retrospect did file by file
> recording to tape, as an example, then later, it could
> also use a disk drive for the copy it kept. The first time
> I used Retrospect, it took me a solid two days (16 hours)
> of work, scripting everything, and even then, a couple partitions
> never seemed to back up properly and would throw some error.
> 
> That doesn't mean every product works like that.
> 
> For example, the version of Acronis a few years back,
> I ran a trial of it, and the workflow was particularly
> easy to get it set up. The current version seems to be
> getting a bit bloated and silly (they're trying to
> trick customers into a subscription model of sorts).
> 
> *******
> 
> All of the ideas above, have mounters, and allow recovery
> of single files. You don't have to restore the capture
> entirely, to get at a single file. Most of the imaging
> .container formats have the file directory at the end
> of the recording, during "mount" that is consulted for
> a file map, and when you navigate through the virtual
> partition, all the files show up in File Explorer for you.
> Even Acronis would do that. So would Macrium.
> 
> (Full and Differential are Free, Incremental requires a license)
> 
> https://www.macrium.com/reflectfree
> 
>     Backup at Home
> 
>     Reflect 7 Free
> 
>     Download Free  <===
> 
> When you install that, it will prompt you to create
> rescue media. The rescue media boots and it is necessary
> for "bare metal restore", when the drive C: used to be
> on is ruined or dead. You make the CD so you have something
> to boot from. It's a good idea to make that right away,
> as the files to make the media will not be on Microsoft
> servers forever. Some of the early versions of the
> WinPE files, like for WinPE 3, are no longer available.
> The process of making the rescue media is automated,
> but there are some options the user can select.
> 
> If you want to, you can also boot the CD and do the backup
> if you want. The backup runs from C:\Program Files for
> sure, but the emergency boot media is also fully functional.
> 
> By selecting an option on the left hand side, you can
> have all disk drives on the computer recorded into
> a single container file. I don't necessarily recommend
> this, from a file reliability point of view however.
> Macrium has a Verify function, and if you keep old
> backups around, sometimes running a Verify indicates
> the .container file has become corrupted. In one case, this
> was a function of the computer having bad RAM, and
> that's how I detected there was a RAM problem, when
> the computer could no longer make "good" backups.
> 
> *******
> 
> Some versions of Windows have File History, which occasionally
> dumps to an external disk for capture. But this is an
> acquired taste, navigating the thing and tricking it into
> coughing up the file you need. This would mainly be for your
> user profile area, and not for capturing the entire C: partition.
> 
> *******
> 
> For more examples of software, try here. Some of the
> software featured in the reviews here, these were the
> "good" versions, and later versions available today
> aren't quite as nice. For example, the two links underneath
> item #2, are probably better versions than the current ones
> for download from the supplier site.
> 
> https://www.raymond.cc/blog/10-commercial-disk-imaging-software-features-and-backuprestore-speed-comparison/ 
> 
> 
> Page 2 has a table of test data, for the programs.
> Easeus had the fastest incremental. Aomei had the fastest Full.
> You can see for the slowest programs, they're sometimes
> five times slower than the fast ones.
> 
> The programs have several jobs to do:
> 
>     Lightweight compression (like LZ4 maybe) - hard on CPU, not 
> multithreaded.
>     Compute checksum over image file - MD5 maybe, limits peak transfer 
> rate.
>     (Copy the data clusters sequentially...)
> 
> For things like computing a checksum, some processors are
> faster at this than others. Then your backup runs faster.
> 
> The compressors used, do not save a lot of space. They are
> not there to make the absolute tiniest file, instead they
> are there to take compressible material (your "notes.txt")
> and make that smaller. Only the files on the disk are
> backed up, and the white space is not recorded. This means
> if, after a restore, you want to run Recuva UnDelete, that's
> not going to work on a freshly restored volume. No program
> is supposed to "record deleted material", that's logically
> a silly thing to do. If you want deleted material recorded
> too, check the Preferences for "Forensic Copy", and select
> that. Note that Forensic Copy is not tested by the developer
> all that well, and I caught mine not making Forensic copies
> when I told it to. Forensic just records everything and
> is a "giant space waster" :-) But if you insist that
> Recuva work for you, that's one way to do it.
> 
> *******
> 
> Windows File History, probably comes closest to your
> carefree requirement. But the various OSes do it different
> ways (Win10 one might be better than the Win7 one), and
> thus this is an "uneven" solution and depends on what
> you're running, as to whether this option is worth
> entertaining. The good version, it prompts you to
> "connect the backup drive now, so I can dump the
> collection to your backup disk". The backup drive
> does not have to be connected continuously. Windows
> is probably using the VSS shadow capability, to record
> file differences, and when it gets enough of them,
> it'll request an opportunity to dump what has
> been collected.
> 
> Robocopy can do mirroring, such that the state of
> the clone drive matches the source drive. But it
> does not do a good job on C:\Windows and is not
> a replacement for a Full. It would do a good job
> on C:\users\username\Downloads for example. And
> you would want to be exited from Excel and Word,
> when doing a run in that case. There are lots of
> similar things, that will make a mess if you don't
> do them properly. This is one reason I harp on the
> backup programs above, first, because the automation
> tends to handle the rough edges better. If you
> craft a crappy solution like our "xcopy brethren"
> might, the mess you make is yours to clean up :-)
> While it is possible to manually make VSS snapshots,
> to "freeze" file content and make the syncing safe,
> it's above my pay scale to tell you how to do that.
> And you're unlikely to get every detail and nuance
> of the permissions right, if hand crafting a solution
> using bailing wire and binder twine.
> 
> The above just scratches the surface. I'm not an
> expert in this stuff, just a user like you. It takes
> a lot of work, to prove the method you select, actually
> works, and I do recommend you test it. It helps to
> have two drives you can use for scratch purposes, to
> test safely. Not a lot of people have drives just
> sitting around for this sort of work.
> 
>     Paul
   Just figuring out takes research to figure out what to do for what 
you want for end result.

   Could take weeks to make that map. Expensive start-up.


   Thanx.



-- 
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus