Re: Thinking aloud.

Hi Nigel,

as for the checksums, I wouldn't even bother checksumming the whole 
page. Checksumming only every 100th byte within the page might be 
sufficient to give you a good idea whether the page is likely to have 
changed or not. Obviously, this would not speed up a 
memory-bandwidth-limited checksum, but saves some CPU cycles and energy.

Do you have an idea of how many pages "typically" don't get touched in 
between two hibernation cycles? I am aware that there's no such thing as 
typical system or a typical hibernation cycle, yet I suspect it would be 
good to know very roughly how much improvement could be expected.
If you didn't plan to do so already, it might be worthwhile to run a 
simple test on a few systems to find out more. Maybe comparing the 
uncompressed, unencrypted images from two subsequent hibernation cycles 
(written to two separate files or partitions on the disk) would do the job?

Finally, your work has always been very much appreciated by me. I have 
to agree with Pedro very much though. Mainline integration of TOI seems 
to be the most important missing feature and would allow a few million 
more users to appreciate your work.

Good luck anyways.

Regards
Martin

On 22/10/2012 04:29, Nigel Cunningham wrote:
> Hi Ted.
>
> On 10/22/2012 01:27 PM, Theodore Ts'o wrote:
>> On Sat, Oct 20, 2012 at 08:20:28AM +1100, Nigel Cunningham wrote:
>>> My initial thought is to use SHA1 checksums to determine what pages
>>> have changed, keeping those checksums (and the pfns to which they
>>> refer) in memory or on disk between hibernation cycles.
>> Hi Nigel,
>>
>> One thought --- if you are on a 64-bit machine, where we don't have a
>> shortage of page flags, you could set a new flag on all pages that
>> were saved during the last hibernate cycle, and on the restore, mark
>> all pages in the page table read/only. As the pages get modified, on
>> the page fault, clear the flag. This should be a lot faster than
>> burning the memory bandwidth and computational resources needed to
>> calculate SHA1 checksums on all memory pages which are in use.
>>
>> If you can't do this approach, something else that would probably
>> worth doing is using a two-pass checksum; first use a very fast
>> checksum, such as Adler-32 or Fletcher-32, which is memory bandwidth
>> limited. If the fast-but-not-entirely checksum doesn't match, there's
>> no point trying to doing the (much slower) SHA1 checksum over the
>> page; you'll know that the page has changed. If the fast checksum
>> matches, then you can do a second pass with SHA1 to make sure you can
>> really skip doing the disk write.
> Thanks for the great suggestions; I'll give them a try.
>
> Nigel
> _______________________________________________
> TuxOnIce-devel mailing list
> [email protected]
> http://lists.tuxonice.net/listinfo/tuxonice-devel
>