Extreme CPU usage by enbd-client?

Hi,

I'm sorry for spamming the list so badly. Currently I'm using enbd very
intensive, and I've learned a lot, but I also keep getting new questions.

While browsing through the enbd list archives, I found the following post:
http://lists.community.tummy.com/pipermail/enbd/2005/004104.html

Quote (Denis Bonnenfant):
> Perfs are pretty good. Without any tweaking,  36 MB/s during
> reconstruction over GbE link, and bonnie between 12-20 MB/s Writes / 20-50 MB/s
>  reads. It's a little bit less for writes thant with direct access to disk, and
> slightly more for reads.

My eyes dropped out after reading this: 35MB/s during reconstruction?!
Currently, I'm running four enbd-servers (somewhat older hardware like
800MHz, but I don't think that should cause problems?) with the
following options:
> enbd-server 1100 /dev/hda5 -i iss01-hda5 -b 4096

And the clients:
> enbd-client iss01 1100 -n 2 -e -b 4096 /dev/nda

On a P4 2.8GHz, I get about 5MB/s (!!) during initial reconstruction of
my raid5 array (without any fastraid patches), but that machine was
instable (some hardware problems). I've replaced the client machine with
a P4 2.6GHz and now I get about 3.5MB/s during initial reconstruction.
The initial reconstruction hasn't finished yet, so I can't do other
benchmarks.

I've tried multiple block sizes, multiple combinations of servers (to
test if one of the servers is the bottleneck), but the difference
between a P4 2.8GHz client and a P4 2.6GHz client makes me think the
problem lies within the client.

One thing which attracts my attention, is the CPU usage on the client.
All client processes together use about 100% of my CPU time, which could
explain the speed drop after replacing the P4 2.8GHz. However, I can't
think of a reason why the client needs such an extreme CPU usage, maybe
someone can explain this to me?

FYI, the top output during the reconstruction:
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  3072 root      15   0  1932  988 1500 S 10.0  0.2   1:07.01 enbd-client
>  3076 root      15   0  1932  988 1500 S  8.7  0.2   1:07.55 enbd-client
>  3067 root      15   0  1932  988 1500 S  8.3  0.2   1:00.62 enbd-client
>  3071 root      15   0  1932  988 1500 S  7.5  0.2   1:00.07 enbd-client
>  3080 root      15   0  1932  988 1500 S  7.5  0.2   0:57.85 enbd-client
>  3077 root      15   0  1932  988 1500 S  5.8  0.2   0:58.34 enbd-client
>  3081 root      15   0  1932  992 1500 S  5.8  0.2   0:50.01 enbd-client
>  3082 root      15   0  1932  992 1500 S  5.8  0.2   0:48.92 enbd-client
(CPU usage is normal when I/O is idle)

uname -a:
> Linux infinity 2.6.8-2-686-smp #1 SMP Tue Aug 16 12:08:30 UTC 2005 i686 GNU/Linux

By the way: this concerns both 2.4.32 and 2.4.33pre (downloaded
2.4.33pre this evening, around 18:00). I didn't try earlier versions.

If you need some debugging info: just let me know.


Regards,

-- Bas