Re: An opes services usage question

On Thu, 18 Mar 2004, John G. Waclawsky wrote:

> I believe the client server affinity problem for metadata is really
> the simpler case of just needing client affinity for the metadata to
> be sent back to a previous proxy stage.

I agree that we need affinity for the metadata to be sent from a
processing node down the application protocol flow (next proxy stage)
back to a processing node up the application protocol flow (previous
proxy stage). I am not sure why you say "simpler case".  Simpler than
what?

> I believe this affinity problem should and can be solved at layer 3
> if the source (client) IP address is available in whatever protocol
> is decided to be used in the transfer of the metadata. I did some
> checking and it seems that most load balancers maintain this IP
> address affinity for you at layer 3.

I am not sure what you mean. The load balancer (L4-7) usually
maintains some kind of connection/address map to match clients and
servers. The load balancer can use that map to support affinity
features for existing connections and for returning clients (in some
cases). However, the L3 mapping is usually not exposed to clients and
may change. That is, a client, in general, does not know the IP
address of a proxy that served its request if that proxy is behind a
load balancer. Note that availability of a proxy IP address in
application protocol headers does not mean that the load balancer will
use that proxy for the next request from the same client and does not
mean that the IP address is reachable. In most cases, visible IP
addresses of balanced proxies are simply an architectural bug (a load
balancer should have stripped them as unreliable/incorrect/private
info, but it did not).

I am sure there are installations where IP address information in the
application protocol headers (e.g. Via header in HTTP) leaked from
behind a load balancer is more or less reliable, at least short term.
Do you want to limit your solution to these environments?

> So a layer 3 solution can be application protocol
> agnostic if we send the metadata in-band back through the load
> balancer.

Yes, _provided_ that load balancer balancing algorithm matches your
affinity needs. For example, if a load balancer uses a simple round
robin on request basis, then you cannot get to the same proxy. Or, of
an HTTP load balancer uses URL-based switching, you may not be able to
get to the same proxy unless you request the same URL.

> My current thinking is that the same layer 3 load
> balancing / affinity information could be leveraged to transfer
> proxy IP address information and resolve the flow participant
> discovery solution for sending metadata out-of-band.

My understanding is that layer 3 load balancing / affinity information
is not generally available. Only load balancer knows it. Can you give
a couple of specific examples that satisfy your needs?

> Driving the solution down to layer 3 allows metadata to be sent
> either in-band (through the load balancer) or out-of-band in those
> situations where proxies have IP reachable addresses. I did some
> more checking and also found that IP reachable addresses are often
> the case because many of the these load balanced environments have
> management LANs associated with them.

I am not sure how existence of a [properly secured and isolated]
management LAN indicated IP-routable addresses. I am sure there are
cases where proxies are globally addressable, of course (which either
indicated poor network setup OR some external requirements that
exposed proxy IPs).

> I agree we need a new protocol to support the metadata exchange (and
> possibly for acknowledgments of metadata). But, if we want the most
> general solution that also allows the sending of metadata
> out-of-band then maybe the new protocol (or some existing protocol
> used or adapted for this situation) could solve the flow participant
> discovery problem too.

IMO, the three requirements (that we agree on) imply that no general
solution is possible without requiring proxies with exposed/routable
IPs (something you seem to require in the above discussion about L3
information) or requiring load balancers that are aware of the new
protocol. Do you agree? Which of the two limitations do you want to
assume?

Alex.

> >Are the following three requirements correct?
> >
> >  1) Agents need to communicate metainformation to each other
> >  2) metainformation is related to some application protocol, but
> >     agent communication protocol should be application agnostic
> >     (i.e., should work with many application protocols)
> >  3) some agents are being load balanced with respect to the
> >     application protocol in question
> >
> >If yes, I conclude:
> >
> >  - some agents will not have IP addresses reachable from other
> >    agents
> >
> >And, hence,
> >
> >  - agent communication without assistance of load balancers
> >    would be impossible in general
> >
> >To resolve the conflict, you have two options, I think:
> >
> >  (a) require load balancers to handle part of the communication
> >      (this is how HTTP, SSL, and ICAP are load balanced today)
> >
> >  (b) limit the environment to agents that are IP-reachable
> >      despite being load-balanced
> >      (this would exclude some (many?) load balancing environments)
> >
> >Do you see what I am getting at? In short, if something is behind a
> >load balancer, that something may not have a routable IP address
> >and, hence, cannot communicate directly with the world.
> >
> >Whether you chose (a) or (b), you still need a new protocol or new
> >protocol profile to support the metainformation exchange.