performance boundaries

Hello everyone,

 

we have encountered a problem, I hope someone might be able to help.

The facts are the following: there is a raid consisting of 40 x 500gb disks,
located on 

two 3ware raid controllers and joint by lvm. Its performance under heavy
load reaches on the network 

the maximum of the ethernet cards, which is 2Gbit/sec (2x1Gb ethernet cards
on the linux side bonding, on the

cisco side etherchannel)

If we encrypt with loop-aes (128bit) the only lvm ensured device
(/dev(vg0/lv0 -> /dev/loop0), the performance of the server

falls down to 800Mbit/s and the loop0 device uses only once cpu on 100%. We
configured the system according to the loop-aes README file's

performance part, which looks something like this:

 

3ware / 20x500gbyte disks (raid50) = /dev/sdb

3ware / 20x500gbyte disks (raid50) = /dev/sdc

 

lvm - /dev/vg0/lv0 = /dev/sdb,/dev/sdc

 

if the loop devices are ordered to sdb->loop0 and sdc->loop1 and so the lvm
connects the /dev/loop0 and /dev/loop1 devices, 

it looks as follows:

 

3ware / 20x500gbyte disks (raid50) = /dev/sdb -> /dev/loop0

3ware / 20x500gbyte disks (raid50) = /dev/sdc -> /dev/loop1

 

lvm - /dev/vg0/lv0 = /dev/loop0,/dev/loop1

 

So we have two loop devices now and the read processes split into 50%-50% on
the two loop devices and 1.6Gbit/sec traffic is reached.

All above shows, that on one loop device maximum 800Mbit/sec traffic can go
through in the present configuration. The question is that

is there any patch, trick, translation option which can brake the
800Mbit/sec borderline?

 

Also, what is the optimal setting for the lo_prealloc value when the loop
module is loaded in? Should we add a higher value or is there

a limit for it? Also, is it better to set the read ahead value on the
physical array (/dev/sdX) and/or to the device connected to it (/dev/loopX)?


Or both should have the same value?

 

Thank you in advance, and hope if we resolve this problem it helps other
people in the future,

 

Viktor