Multi-threaded archive test mode

Hi Denis


If you ever find yourself bored, here's a potentially helpful new feature idea:
multi-threaded archive test mode support.

Why?

dar's test mode can be really, really, really slow. All other modes are
I/O-bound, so wouldn't benefit from something like that, but test mode is different.

Usually, the test mode is also more or less I/O-bound, either due to compression
being used (and decompression being reasonably fast) or CRC calculations going
on, which are also fast.

Unfortunately, there's an edge case to all of this: large sparse files.

When handling sparse files, dar naturally knows that holes are all-zero, so it
doesn't need to actually read any data from the archive. Instead, it will just
happily calculate CRC values in a tight CPU-bound loop.

With just one thread checking the data, a lot of wall clock time is wasted,
while the machine almost idle I/O-wise and only hogging up one logical CPU core.

To counter this, dar could check archive files in multiple threads, one file per
thread. This would make sure that either CPU or I/O resources are always fully used.


I'm also aware of drawbacks to this approach. I was able to come up with these:
  - Doesn't work in sequential read mode, unless all files to be processed are
    fully buffered in memory first, which is a TERRIBLE idea for obvious
    reasons.
  - Can reduce read performance drastically with conventional hard disk drives
    due to constant seeks within the archive.

Due to that, I don't think that this feature should be enabled by default (and
it shouldn't be possible to enable it for sequential read mode at all).

The -G/--multi-thread option is also not a good for it, it probably should be a
new option such as --test-threads.


I'm looking forward to your analysis.



Mihai