Re: An opes services usage question

Alex, sorry for the delay in my latest response. I have been doing some 
investigation and also some travel and other things ...with my day job 
   :-)  

To get back to the discussion. It seems that many of the assumptions in 
the background of our discussions about load balancers are based on web 
load balancers that provide virtual addresses and hide the server 
cluster from the user. But, I have been thinking a little differently in 
considering usage of an opes framework in the mobile wireless market 
segment. This segment may be unique in that load balancers do not 
rebalance for every client connection. Instead they typically rebalance 
on a "per client" basis. This is done because the devices that are being 
balanced have client information associated with their traffic such as 
$, access rights, browser form factors, XML style sheets...etc. Affinity 
to a particular server is strongly maintained for each client. Once 
affinity is in place the load balancer could operate, for example, by 
just replacing the destination MAC address as the traffic arrives based 
on the source IP address (you could view the load balancer as just a 
"bump on the wire").  I believe this additional information should help 
clarify what I am thinking, a bit. I guess I am looking for the simplest 
solution for this environment and I keep thinking the most elegant 
solution would be lower layer one.... just some of my thoughts...

It seems that of the two limitations you suggest, the best way to 
proceed (IMO) would be to assume proxies with exposed/routable adresses. 
I am thinking this assumption is the best one because it probably 
provides a faster solution, would be more flexible, and satisfies small 
deployment needs that do not have load balancers but still need the 
metadata returned to a specific server. I think we are also in agreement 
that with either limitation we still need something to support the 
metadata information exchange. I was also wondering if anyone else has 
any suggestions or additional information to consider?  

Regards  John

Alex Rousskov wrote:

>On Thu, 18 Mar 2004, John G. Waclawsky wrote:
>
>  
>
>>I believe the client server affinity problem for metadata is really
>>the simpler case of just needing client affinity for the metadata to
>>be sent back to a previous proxy stage.
>>    
>>
>
>I agree that we need affinity for the metadata to be sent from a
>processing node down the application protocol flow (next proxy stage)
>back to a processing node up the application protocol flow (previous
>proxy stage). I am not sure why you say "simpler case".  Simpler than
>what?
>
>  
>
>>I believe this affinity problem should and can be solved at layer 3
>>if the source (client) IP address is available in whatever protocol
>>is decided to be used in the transfer of the metadata. I did some
>>checking and it seems that most load balancers maintain this IP
>>address affinity for you at layer 3.
>>    
>>
>
>I am not sure what you mean. The load balancer (L4-7) usually
>maintains some kind of connection/address map to match clients and
>servers. The load balancer can use that map to support affinity
>features for existing connections and for returning clients (in some
>cases). However, the L3 mapping is usually not exposed to clients and
>may change. That is, a client, in general, does not know the IP
>address of a proxy that served its request if that proxy is behind a
>load balancer. Note that availability of a proxy IP address in
>application protocol headers does not mean that the load balancer will
>use that proxy for the next request from the same client and does not
>mean that the IP address is reachable. In most cases, visible IP
>addresses of balanced proxies are simply an architectural bug (a load
>balancer should have stripped them as unreliable/incorrect/private
>info, but it did not).
>
>I am sure there are installations where IP address information in the
>application protocol headers (e.g. Via header in HTTP) leaked from
>behind a load balancer is more or less reliable, at least short term.
>Do you want to limit your solution to these environments?
>
>  
>
>>So a layer 3 solution can be application protocol
>>agnostic if we send the metadata in-band back through the load
>>balancer.
>>    
>>
>
>Yes, _provided_ that load balancer balancing algorithm matches your
>affinity needs. For example, if a load balancer uses a simple round
>robin on request basis, then you cannot get to the same proxy. Or, of
>an HTTP load balancer uses URL-based switching, you may not be able to
>get to the same proxy unless you request the same URL.
>
>  
>
>>My current thinking is that the same layer 3 load
>>balancing / affinity information could be leveraged to transfer
>>proxy IP address information and resolve the flow participant
>>discovery solution for sending metadata out-of-band.
>>    
>>
>
>My understanding is that layer 3 load balancing / affinity information
>is not generally available. Only load balancer knows it. Can you give
>a couple of specific examples that satisfy your needs?
>
>  
>
>>Driving the solution down to layer 3 allows metadata to be sent
>>either in-band (through the load balancer) or out-of-band in those
>>situations where proxies have IP reachable addresses. I did some
>>more checking and also found that IP reachable addresses are often
>>the case because many of the these load balanced environments have
>>management LANs associated with them.
>>    
>>
>
>I am not sure how existence of a [properly secured and isolated]
>management LAN indicated IP-routable addresses. I am sure there are
>cases where proxies are globally addressable, of course (which either
>indicated poor network setup OR some external requirements that
>exposed proxy IPs).
>
>  
>
>>I agree we need a new protocol to support the metadata exchange (and
>>possibly for acknowledgments of metadata). But, if we want the most
>>general solution that also allows the sending of metadata
>>out-of-band then maybe the new protocol (or some existing protocol
>>used or adapted for this situation) could solve the flow participant
>>discovery problem too.
>>    
>>
>
>IMO, the three requirements (that we agree on) imply that no general
>solution is possible without requiring proxies with exposed/routable
>IPs (something you seem to require in the above discussion about L3
>information) or requiring load balancers that are aware of the new
>protocol. Do you agree? Which of the two limitations do you want to
>assume?
>
>Alex.
>
>  
>
>>>Are the following three requirements correct?
>>>
>>> 1) Agents need to communicate metainformation to each other
>>> 2) metainformation is related to some application protocol, but
>>>    agent communication protocol should be application agnostic
>>>    (i.e., should work with many application protocols)
>>> 3) some agents are being load balanced with respect to the
>>>    application protocol in question
>>>
>>>If yes, I conclude:
>>>
>>> - some agents will not have IP addresses reachable from other
>>>   agents
>>>
>>>And, hence,
>>>
>>> - agent communication without assistance of load balancers
>>>   would be impossible in general
>>>
>>>To resolve the conflict, you have two options, I think:
>>>
>>> (a) require load balancers to handle part of the communication
>>>     (this is how HTTP, SSL, and ICAP are load balanced today)
>>>
>>> (b) limit the environment to agents that are IP-reachable
>>>     despite being load-balanced
>>>     (this would exclude some (many?) load balancing environments)
>>>
>>>Do you see what I am getting at? In short, if something is behind a
>>>load balancer, that something may not have a routable IP address
>>>and, hence, cannot communicate directly with the world.
>>>
>>>Whether you chose (a) or (b), you still need a new protocol or new
>>>protocol profile to support the metainformation exchange.
>>>      
>>>
>
>
>  
>