Re: Load balancing across multiple network paths

"Also sprach Nikos Gkorogiannis:"
> On Wed, 2004-12-08 at 12:32, Peter T. Breuer wrote: 
> > As aliases? You mean two addresses on the command line, one per (slave)
> > daemon? Can't say that I've tried that for a while ...
> 
> Yes, I start the client with
> 
> enbd-client 192.168.0.1:12345 192.168.0.1 192.168.1.1 -b 4096 /dev/nda

I'd forgotten the syntax! I suppose it is correct. The first must be
where the master server that launches the slaves is to be found, and the
other two must be the IPs on which the slaves are.

And these NICs are on different subnets. The really have different
cables and different routes to the target?

> > > notice that both network interfaces are being used (confirmed also in
> > > /proc/nbdinfo stats)
> > 
> > How does it show that, exactly? Can you post the stats maybe?
> 
> Doing a 
> dd if=/dev/nda of=/dev/null bs=64k count=4k
> just after starting the client gives the following /proc/nbdinfo
> 
> Device a:       Open
> [a] State:      verify, rw, merge requests, enabled, show_errs, last
> error 0, lives 0, bp 0
> [a] Queued:     +0R/0W curr (check 0R/0W) +1R/1W max
> [a] Buffersize: 262144  (sectors=512, blocks=64)
> [a] Blocksize:  4096    (log=12)
> [a] Size:       16811008KB
> [a] Blocks:     4202752
> [a] Sockets:    2       (*)     (+)
> [a] Requested:  64.056K (35.4K) (28.6K) 64.05KR/0W      max 32
> [a] Despatched: 64.056K (35.4K) (28.6K) 64.05KR/0W      md5 0W (0 eq, 0
> ne, 0 dn)
> [a] Errored:    0       (0)     (0)     0+0

Looks more or less equally distributed, but not exactly so - so I don't
think they can be said to be "alternating".

> [a] Pending:    0       (0)     (0)     0R/0W+0R/0W
> [a] B/s now:    0       (0R+0W)
> [a] B/s ave:    1.54M   (1.54MR+0W)
> [a] B/s max:    8.14M   (8.14MR+0W)
> [a] Spectrum:   99%32
> [a] Kthreads:   0       (0 waiting/0 running/1 max)
> [a] Cthreads:   2       (+)     (+)
> [a] Cpids:      2       (811)   (812)
> Device b-p:     Closed
> 
> Firstly, the B/s max is close to the theoretical max of a 100Mbit/s NIC
> (isn't it?).

Yes, but it's measuring more how fast the kernel can send it data, if I
recall  correctly. It can't really distinguish. I could look up the
calculation it does ... (later).

> Second, the Requested and Dispatched are not the same, but
> not that far from each other, which would mean that both daemons are
> doing work, no?

Yes, they are doing work, but not alternating, according to the stats I
see. 

> > The daemons are independent. They do not alternate, because they do not
> > (in effect) know the other exists. So there is some other explanation.
> 
> So, from what you saying I should expect almost double throughput? Hm,
> curious...

Yes, are you able to measure the traffic through the NICs? Can you show
the ifconfig lines? I want to look at the packet counts. Better if you
can zero them first by removing and relaoding drivers, then running a test.

(Btw, you can use ethernet binding).

Peter