Re: Thinking aloud.

Am 19.10.2012 23:20, schrieb Nigel Cunningham:
> Hi all.
>
> For a few years now, TuxOnIce has been in maintenance mode, having
> reached a point where most of us seem to be pretty happy with the
> functionality it provides.
>
> Recently, however, I've started to think about one more way in which it
> could be improved. I'm not saying I'm going to start work on
> implementing this immediately (and maybe not ever), but I thought it
> might be good to share the idea with you and get it publicly documented
> that the idea was around before the implementation, lest fun things like
> patents come into the picture later (may it never happen!).
>
> Anyway, what I've been thinking about is that fact that each time we
> write the image, we write the complete image. Surely there must be a
> portion of the data that is exactly the same as last time, and therefore
> shouldn't need to be written again. If we could avoid writing that
> portion again, we would definitely reduce wear and tear on the drive(s)
> being written to, and potentially make writing the image [much?] faster.
> Hopefully this could be done without greatly increasing TuxOnIce's
> memory usage or the time taken to read the image.

On the one hand SSDs make hibernating and resuming a lot faster. Your 
approach will probably have more effect when the image is written on 
traditional hard drives. On the other hand, today's machines have more 
RAM, which will make the writing/reading process take longer even on 
SSDs. Therefore I believe such an improvement as you proposed could be a 
very useful investment.

> My initial thought is to use SHA1 checksums to determine what pages have
> changed, keeping those checksums (and the pfns to which they refer) in
> memory or on disk between hibernation cycles. This would also of course
> require us to not free storage between cycles, like with the Keep Image

Your solution would require permanent reservation of storage space. That 
is no problem now, because if you have lots of RAM, it's very unlikely 
you will need swap space. This is something that might change though. 
Nevertheless, it might be better to save that bit of storage space for 
hibernation and benefit from the speed-up, as storage space will always 
increase and be bigger in relation to RAM. Additionally, one might 
perhaps only use a smaller portion of the image for such checksummed data.

> mode. (Is checksumming a full page and then compressing the data and
> writing it if necessary significantly faster (or slower?) than
> compressing a full page and then checksumming the compressed data?)

If you checksum uncompressed pages, you'd save the time needed for 
compression. Therefore I'd say it should be faster. And this way it 
doesn't matter whether you compress pages or not, so even writing an 
uncompressed image would improve.

> A simple initial implementation could be to add the changed pages in
> each pageset to the end of the image, modifying the header so it knows
> of the existence of the extra data and loads it after loading the
> original image data (etc). The whole image could be rewritten when we
> run out of room for the incremental modifications, or after a fixed
> number of cycles, or triggered manually.

> We could also look at overwriting data in the image as contents change.
> It would probably reduce fragmentation if pages were sorted according to
> the number of times they'd changed (put pages that have never changed
> together since they're probably less likely to change in the future and
> so on). This would of course only make sense after you've already done
> some cycles.

Maybe one could also defragment/optimize at a later time, that is 
_after_ resuming is complete, when cpu and storage are idle? Or maybe 
make this function available via sysfs, so something could trigger it.

> What do you think?
>
> Regards,
>
> Nigel

Could prove very useful. Windows 8 improved boot time by using 
hibernation for the kernel. I think you can read some of it up here: 
http://blogs.msdn.com/b/b8/archive/2011/09/08/delivering-fast-boot-times-in-windows-8.aspx
They talk about parallel-processing and compressing/decompressing 
hibernation image, maybe they took some of your inventions ;-)
That said, this idea of yours is more revolutionary though. Even if it 
wouldn't turn out to be faster, I'd find it amazing to see it at work.

On a different issue: No webgit anymore (git.tuxonice.net => Drupal)?

Regards,

Harald

-- 
`Experience is the best teacher.'