Re: Action Item - Part I: WSRX and MEP signaling on the wire

David,

Please see my comments below.

Cheers,

Christopher Ferris
STSM, Emerging e-business Industry Architecture
email: [email protected]
blog: http://www.ibm.com/developerworks/blogs/dw_blog.jspa?blog=440
phone: +1 508 377 9295

David Hull <[email protected]> wrote on 01/12/2006 12:40:52 AM:

> Chris,
> 
> Thanks for the response.  You are of course right that WS-RM defines
> acknowledgment at the reliability layer:  A message is acknowledged and 
then
> delivered.  Interestingly, WS-Reliability at least appears to take 
> the other approach: A message is delivered to the application level 
> and then acknowledged.  At least, the WS-R picture with four boxes 
> in it shows arrows both ways between the application and reliability
> layers, while WS-RM shows arrows only one way.
> 
> I'm shocked, shocked I say, to find a source of potential confusion 
> among reliability standards.

Well, I have been making this point to the WS-R crowd for years. 
Initially,
I believe that what you describe was the case. That WS-R required that 
before
acknowledgeing the message, that the message had to be "delivered" to the
application layer... Something that I found to be problematic. However, 
over time,
there were some subtle, and I would argue too subtle, changes made that 
implied
that "delivery" could mean simply to a stable storage medium, not 
necessarily
to the application itself (e.g. queued delivery)

> 
> In any case, leaving aside the possibility that delivery could 
> involve arbitrary amounts of application-level processing (e.g., I 
> want to know when the receiver taken some action based on my 
> message, not just that the message arrived), I believe that the 
> issues I'm describing will occur whenever there is more than one 
> transport-level hop involved, and I believe that both flavors of 
> reliability standard allow for this.  If not, then I'd think there 
> would be a need for some sort of extension to cover such cases.  I 
> think the analogy to return-receipt is fairly tight.  The return 
> receipt is meant to ack not only that the message arrived in a 
> mailbox, but that someone actually saw it.

I have long argued that if you need an application-level ack, then have
the application send one. The purpose, as I see it, of the reliable
messaging layer is to ensure a successful hand-off, not to ensure that the
application does, or even can, process the message.

> 
> As far as I can tell, there shouldn't be any significant difference 
> from the reliability layer's point of view between forwarding a 
> message along another hop and doing some other sort of processing 
> before declaring it delivered.  In either case, I don't send back an
> ack until something arbitrarily long-running and outside my control 
> finishes happening.

Respectfully, I disagree. 

> 
> The real feature I'm really trying to highlight here is that the 
> flow of messages and acks may be decoupled in various ways from the 
> transport-level flow of requests and responses.  From your comments,
> may I take it that you accept that premise?  As has been pointed 

yes

> out, this is essentially building a separate protocol of messages 
> and acks on top of a protocol of requests and responses.  Mark has 
> already done a good deal more than cringe at this.

As is his wont :-)

> 
> As to HTTP request-response and SOAPiness thereof, we seem to be 
> converging on a "standard model" matching what you describe.  I'm 
> certainly comfortable with "HTTP is always request-response, but the
> either the request or the response or both may not be SOAPy".  I'm 
> not sure if DaveO would put it quite that way, but I'll let him 
> speak for himself on that.
> 
> Generalizing a bit, there are transports that act in a request-
> response manner, as Noah describes in [1].  HTTP is one and, from 
> what I can make out, XMPP's <iq/> mechanism can be regarded as 
> another.  There are also transports that act in a fire and forget 
> manner, for example XMPP's <message/> mechanism.  For these, the MEP
> will always be one-way, fire and forget or whatever we call it. 

Correct, but that does not preclude that some form of correlation, outside
the scope of the underlying transport/transfer protocol, cannot be 
employed
(e.g. ws-addressing) to premit higher-level MEPs such as request/response.

> 
> The transport MEP is (unsurprisingly) a static property of the 
transport.
> 
> Christopher B Ferris wrote: 
> 
> Apologies for the belated response... too much travel. Please see my
> comments below. 
> 
> Cheers, 
> 
> Christopher Ferris
> STSM, Emerging e-business Industry Architecture
> email: [email protected]
> blog: http://www.ibm.com/developerworks/blogs/dw_blog.jspa?blog=440
> phone: +1 508 377 9295 
> 
> [email protected] wrote on 01/04/2006 05:24:37 PM:
> 
> > This is a bit more free-form than I'd originally intended, but bear 
> > with me.  I think we get somewhere.
> > 
> > First, a motivating use case involving reliability.  I believe there
> > are WSRX members on this list or at least lurking nearby.  If so, 
> > please let me know if I've missed something fundamental.
> > 
> > The reliability standards I've seen distinguish between 
> > submit/deliver and transmit/receive.  Transmitting and receiving are
> > transport-level concepts and carry no particular guarantee of 
> > reliability.  An acknowledgment message acknowledges not that a 
message was 
> > received at the transport level, but that it was delivered to its 
> 
> Not so. The WS-RM SequenceAcknowledgement merely acknowledges that the 
> message was successfully received. It makes no claims whatsoever that 
the 
> message has, or even will, be delivered to the ultimate recipient. 
> It is effectively the equivalent of the HTTP 202 Accepted response in 
its 
> meaning. The distinction is that an HTTP 202 Accepted is returned on the 

> same connection as the HTTP request message, whereas the WS-RM SeqAck 
> can be returned on a separate connection and that it covers the complete 

> range(s) of received messages received, such that it need not be 
transmitted 
> (or transferred as the case may be) reliably. 
> 
> > ultimate recipient.  Just exactly what this means is left out of 
> > scope.  Essentially a message is considered received if the sending 
> > side learns of an acknowledgment for it according to whatever 
> > acknowledgment scheme is in effect.
> > 
> > The upshot here is that, even if I use a TCP-based transport, and 
> > even if that transport allows for status codes or some other 
> > indication of a successful exchange, successful transmission of a 
> > message does not imply successful delivery.  Delivery might depend 
> > on some further application-level processing, forwarding to another 
> > destination, or whatever. 
> 
> See above. 
> 
> > 
> > To be concrete, suppose I am using WSN or similar to manage sending 
> > notifications to a given node.  I have provided an HTTP endpoint 
> > reference for the consumer of the subscriptions, and I have 
> > specified (using WSN's not-yet-specified policy assertions :-) that 
> > these notifications should be delivered reliably. 
> 
> Actually, you would specify the use of WS-RM using the WS-RM Policy 
> Assertion:-) 
> 
> > 
> > Notifications are inherently one-way.  Given HTTP as a transport, we
> > would want to use the request-optional-response MEP to send 
> > notifications.  I'm not sure if WSRX does (or doesn't) support what 
> > I'm about to describe, but I would expect that the producer (or 
> > something acting on its behalf, but that's a separate issue) send 
> > notifications to the consumer by POSTing SOAP messages to the HTTP 
> > address given in the subscription request.
> > 
> > What comes back?
> > 
> > In the absence of other information, I would expect that each 
> > notification POST should get as its response either a 202 or a SOAP 
> > message containing acknowledgment information for some set of 
> > previous messages.  The 202 would happen if there were no 
reliability-level
> > acknowledgments available.  Alternatively, the server could send 
> > always back an acknowledgment message.  If there were nothing new to
> > acknowledge, it would be identical to the previous one.  I don't 
> > have a strong opinion as to which of these is better; my intuition 
> > is to allow for either. 
> 
> Agreed, especially given that there is no requirement in WS-RM that 
> a SequenceAcknowledgement be sent for each and every received message. 
> In fact, technically, there is only one SequenceAcknowledgement 
required; 
> the one that covers the entire range of messages in the Sequence. 
> 
> In practice, however, it is likely that implementations will send a 
> "naked" SequenceAcknowledgement message in the HTTP response entity 
> body for oneway notifications/messages on each response. It should be 
noted 
> that the SequenceAcknowledgement may not necessarily include the message 

> number of the message transferred on the HTTP request message. 
> 
> > 
> > An interesting variant occurs when the producer is sending multiple 
> > streams of notifications to the consumer, in which case a 
> > notification in one stream might elicit an acknowledgment for a 
> > notification from a different stream.  Why not?  The HTTP response 
> > channel is available and might not be available again for arbitrarily 
long.
> 
> This is supported by WS-RM. 
> 
> > 
> > Notable points: 
> > The transport-level MEP is always request response. 
> 
> In the context of the HTTP binding, yes (although I am 
> SURE that MB will cringe at the use of the term "transport-level" :-) 
> 
> > The SOAP MEP is always request-optional-response 
> > The application-level MEP is always one-way 
> > The potentially complex part here is in the reliability layer.  It 
> > sees two streams of messages: notifications coming out and 
> > acknowledgments coming back.  From this point of view 
> > The flow of acknowledgments is independent of the transport-level 
> > flow (a given HTTP response acknowledges zero or more notifications) 
> > The flow of acknowledgments is independent of the application-level 
> > flow (for the same reason) 
> 
> yes 
> 
> > The flow of notifications ought to be independent of the transport 
> > level flow.  If nothing else, WSN allows for boxcarring of 
> > notifications: a given "notify" SOAP message may contain several 
> > notifications (but does the reliability layer know this?). 
> 
> No, WS-RM would only be concerned with the message, not its contents. 
> 
> > Further, since the reliability layer may retransmit, so a given 
> > application-level notification may trigger more than one transport-
> > level message. 
> > The recurring theme I'm seeing here is that each level has its own 
> > view of message flow.  Often, those views will converge, but they 
> 
> Bingo. 
> 
> > can diverge for any number of perfectly ordinary reasons, even 
> > leaving aside cases involving cell phones and tunnels: 
> > A request-response operation may be realized as more than one 
> > transport-level exchange (the WSA "async request-response" case) 
> > A one-way operation may be realized as a request-response, either as
> > a push operation or through polling. 
> 
> Even a oneway message over the HTTP binding involves a request/response. 

> The distinction, as I have seen in other emails, in this and related 
> threads, is the SOAPiness of the response. 
> 
> > Retransmission means that one application-level operation may 
> > manifest as several transport-level operations, even where the 
> > application-level operation would otherwise match the transport level. 

> > Boxcarring, where applicable, means that several application-level 
> > operations may manifest as a single transport-level operation. 
> > Reliability-level acknowledgments boxcar by design. 
> 
> Don't know if I would necessarily use that term, but yes... as discussed 

> above, the SeqAck covers the entire range of messages in the Sequence, 
not 
> just the last one received as would be the case of an HTTP 2xx response. 

> 
> > This may seem complex, but there is a bright side.  The various 
concerns 
> > have to be separated, so we might as well gain from this separation. 
> 
> You go, girl! This has been my point all along. 
> 
> > In particular, SOAP can focus narrowly on abstracting the 
> > characteristics of the transport and not worry about anything else. 
> > I'm more and more convinced that we're about done as far as HTTP 
> is concerned: 
> > The SOAP request-(optional-)response MEP describes a SOAP message 
> > going out and something SOAPy optionally coming back. 
> 
> As Noah points out, something optionally SOAPy coming back. 
> 
> > The SOAP response MEP describes a non-SOAP request and something 
> > SOAPy coming back. 
> > Um, how do we describe a SOAP request and something non-SOAPy coming 
back? 
> > We shouldn't have to worry about non-SOAP out and non-SOAP back, 
right? 
> 
> Correct.