Thinking aloud.

Hi all.

For a few years now, TuxOnIce has been in maintenance mode, having 
reached a point where most of us seem to be pretty happy with the 
functionality it provides.

Recently, however, I've started to think about one more way in which it 
could be improved. I'm not saying I'm going to start work on 
implementing this immediately (and maybe not ever), but I thought it 
might be good to share the idea with you and get it publicly documented 
that the idea was around before the implementation, lest fun things like 
patents come into the picture later (may it never happen!).

Anyway, what I've been thinking about is that fact that each time we 
write the image, we write the complete image. Surely there must be a 
portion of the data that is exactly the same as last time, and therefore 
shouldn't need to be written again. If we could avoid writing that 
portion again, we would definitely reduce wear and tear on the drive(s) 
being written to, and potentially make writing the image [much?] faster. 
Hopefully this could be done without greatly increasing TuxOnIce's 
memory usage or the time taken to read the image.

My initial thought is to use SHA1 checksums to determine what pages have 
changed, keeping those checksums (and the pfns to which they refer) in 
memory or on disk between hibernation cycles. This would also of course 
require us to not free storage between cycles, like with the Keep Image 
mode. (Is checksumming a full page and then compressing the data and 
writing it if necessary significantly faster (or slower?) than 
compressing a full page and then checksumming the compressed data?)

A simple initial implementation could be to add the changed pages in 
each pageset to the end of the image, modifying the header so it knows 
of the existence of the extra data and loads it after loading the 
original image data (etc). The whole image could be rewritten when we 
run out of room for the incremental modifications, or after a fixed 
number of cycles, or triggered manually.

We could also look at overwriting data in the image as contents change. 
It would probably reduce fragmentation if pages were sorted according to 
the number of times they'd changed (put pages that have never changed 
together since they're probably less likely to change in the future and 
so on). This would of course only make sense after you've already done 
some cycles.

What do you think?

Regards,

Nigel