Re: Re: Load balancing across multiple network paths

On Fri, 2004-12-17 at 10:41, Peter T. Breuer wrote:
> Very interesting. What I see is:
> 
>    one disk is worse than four disks, so something is happening in
>    parallel!

Yes, it's just software raid-0. A sequential read off a single disk
(locally) gives around 22MB/s and 27 MB/s off the raid-0.

<snip>
>    Performance peaks at 48KB request size limits. You probably want to
>    change make f card to see if it is a limit of the card.

Will do.

<snip>
>    Performance declines with increasing request size beyond 48KB even 
>    with ONE disk, down to about 6MB/s at 128KB. You cannot attribute
>    that to anything else except the behaviour of the network and the
>    NIC, I think, since preumablyteh disks do not care whether you take
>    data off them in small or large lumps! Well, maybe one could
>    attribute it to the server having to read the data into buffers and
>    then send it. If so, try running the server with the -M switch,
>    will will tell it to send data to the net directly from a mapped area
>    of the disk.
> 
> I'm a little concerned about that last point. I should check to see how
> the server send data without -M. I suspect it needs to read a bit write
> a bit read a bit write a bit ... rather than doing whatever it is doing
> now. 

I cannot find such an option in enbd-server. I guess you are talking
about 2.4.32, but unfortunately I'm using 2.4.31 because of its
stability.

> > X is max number of blocks per requests (which I believe is
> > merge_requests+1, no?) and Y is total throughput. All points correspond
> > to avg of two 1Gb sustained reads with variance less than 1%. The first
> 
> How much RAM do you have?

128MB on both client and server. No swap on server.

<snip>
> > On an unrelated issue: are there any suggestions, comments, warnings etc
> > with respect to the following scenario?
> > 
> > * N computers with 4 ide disks each, in a raid0 array, running enbd-server. 
> > * one computer running N enbd-clients, and a raid5 device on top of them.
> 
> Err, striped raid exported to clients running raid5 over the exported
> devices. Yes - raid5 over the net will be a nightmare unless you use
> FR5 instead of raid5 (fr5.sf.net). 

That is the idea. Apart from its features related to resyncing, is there
anything else that fr5 offers beyond the standard raid5?

> And in general, putting all your
> eggs in one basket (the net) is not a robust solution!

Yes, I should have added (what I'm already doing on the test system):
* two separate, private networks, two independent switches (ultimately
on two UPSs) and two NICs in each machine.

and maybe:
* secondary host ready to take over the raid5 if the machine running the
clients dies.

> A more standard configuration would be something like: arrange clients
> in groups of 3. Let each machine run raid5 over one local disk and two
> remote disks imported via ENBD, and let it export two local disks via
> ENBD to the other raids.  That lets each machine export one raided
> "disk" to the world, and so you get three exported (raided) disks of
> twice the size of a physical disk out of nine physical disks.  And if
> any one of the three machine dies, the other two both will survive its
> loss in their arrays, and can recreate its array from the data they have
> between them.

Interesting scheme. However, from my point of view, there are some
disadvantages to this: 
* three logical volumes instead of one, therefore three file servers
(maybe one can re-export them to a single host but it starts getting
nasty in that the advantages of the separation are lost).
* if one host dies, one of the three arrays is taken out (of course, one
can setup a failover system that would serve the (degraded) array from a
different machine - again quite complex).
* is not the network still the main basket (compared to the above
scheme)? :)

> > Are there any guidelines or rules of thumb for module or filesystem
> > parameters etc?
> 
> If you are running raid5, you are interested in achieving stability and 
> rapid recovery, not speed!

Um, yes, obviously. I wasn't referring to speed necessarily in my
question.

Nikos




This email has been independently scanned for viruses and any virus detected has been removed using McAfee anti-virus software