Re: [Patch] chunkd: add self-checking

On 03/02/2010 01:31 PM, Pete Zaitcev wrote:
> This patch adds the self-check to Chunk. With it, the daemon can rescan
> all of its keys and drop those that fail to match their own checksums
> or throw an I/O error. Objects that are found found faulty are made
> invisible to applications (back-end files are renamed, so that bad
> blocks are not reused). This is intended to work in concert with
> applications that store redundant copies of their data objects.
>
> This patch includes the part that tracks the active I/O, since the
> self-check is the only user of it, so it made little sense to separate
> the two. We have to track the I/O so that self-check does not mistakenly
> assume a partially stored object to be faulty and kills it.
>
> Running the self-check can adversely affect performance. As a crude
> way to limit the problem, we limit the load to one check thread only.
> Still, as anyone who had mlocate or Beagle running in their desktop
> knows, the biggest issue is not additional I/O as such, but the blowing
> away the page cache and dentries in kernel. Also, our scheduling is
> between rudimentary and non-existing. We only provide a looping check
> with a randomized delay and an external control to start and verify the
> running of the checking.
>
> Therefore, to avoid surprises with sudden loss of objects and with
> performance anomalies, periodic self-check defaults to off.
>
> Still, self-check is an intergral part of the daemon, so we include
> unit tests for both the I/O tracking and self-check itself.
>
> Signed-off-by: Pete Zaitcev<[email protected]>
>
> ---
>   doc/setup.txt           |   10 +
>   include/Makefile.am     |    2
>   include/chunk-private.h |    4
>   include/chunk_msg.h     |   21 ++
>   include/chunkc.h        |    4
>   include/objcache.h      |   75 ++++++++
>   lib/chunkdc.c           |   82 +++++++++
>   server/Makefile.am      |    3
>   server/be-fs.c          |  130 ++++++++++++++
>   server/chunkd.h         |   28 +++
>   server/cldu.c           |    1
>   server/config.c         |   18 ++
>   server/objcache.c       |  138 +++++++++++++++
>   server/object.c         |    9 +
>   server/selfcheck.c      |  293 +++++++++++++++++++++++++++++++++
>   server/server.c         |  135 +++++++++++++++
>   test/.gitignore         |    2
>   test/Makefile.am        |    7
>   test/objcache-unit.c    |   64 +++++++
>   test/selfcheck-unit.c   |  334 ++++++++++++++++++++++++++++++++++++++
>   test/test.h             |    2
>   tools/chcli.c           |  116 +++++++++++--
>   22 files changed, 1454 insertions(+), 24 deletions(-)

applied...  looks mostly OK.  Things like self-check period still need 
to be removed, and a few other minor quibbles.  But let's go ahead and 
get this in, then tackle those things.