Fixed at last!

Hi all.

I thought you might like to hear some good news.

I finally found some time to put a concerted effort into finding and 
fixing the cause of TuxOnIce not working with the BFS scheduler. The 
issue turned out, I think, to be the same one that's been causing those 
pesky occasional failures to read compressed data (the -22 error).

The problem turned out to be nice and simple: when we create the extra 
threads for doing multithreaded I/O, we bind each one to its own CPU. 
That's find and dandy until we get to the atomic copy, when we do CPU 
hotplugging. The affinities get broken and threads end up stomping over 
each others' memory. It seems that BFS does a better job of breaking the 
affinities than the normal scheduler, which made the problem easier to find.

So, after thinking all this time that it was a locking problem in the 
code that was already there, it turned out to be a few lines of code 
that weren't there instead: the solution is to simply get each thread to 
remember its affinity and reset it before it starts to do any work.

And the outcome:

TuxOnIce debugging info:
- TuxOnIce core  : 3.2.1
- Kernel Version : 3.2.13+
- Compiler vers. : 4.7
- Attempt number : 1
- Parameters     : 0 667661 0 1 -2 0
- Overall expected compression percentage: 0.
- Compressor is 'lzo'.
   Compressed 1095974912 bytes into 261283523 (76 percent compression).
- Block I/O active.
- Max outstanding reads 1866. Max writes 3449.
   Memory_needed: 1024 x (4096 + 352 + 112) = 4669440 bytes.
   Free mem throttle point reached 0.
- Swap Allocator enabled.
   Swap available for image: 2097151 pages.
- File Allocator active.
   Storage available for image: 0 pages.
- I/O speed: Write 782 MB/s, Read 1061 MB/s.
- Extra pages    : 74 used/2000.
- Result         : Succeeded.
nigel@nigel:~$

Those astoundingly big speeds are due to this being an Intel 520 SSD on 
a quad-core laptop. I don't usually get a GB/s. More like about 900MB/s. 
This is having just booted - no real apps loaded.

Patches being applied to git trees now, and will be pushed to github 
shortly. Hopefully I'll also get some patches up on tuxonice.net soon.

Nigel