Asynchrony and BEEP

Hi All,

Since the IETF WG wrapped up, this list seems to be fairly quiet, but I
have a technical matter I'd like to discuss.

I have an asynchronous application that needs a protocol.  Similar to
the experience of RFC 3117, I did a little research on protocols in the
hopes of finding one that fits (more or less).

Until recently, I was confident that BEEP provided all that I needed.
It turns out a fundamental assumption I had made was quite wrong
(there's a lesson in that).  I had assumed that the presence of a
message number that must be unique would mean that responses to messages
were correlated based on this number.  Someone recently pointed out
Section 2.6 of RFC 3080 that disabused me of this notion - channels are
entirely serial in each direction.

Having read more material on the subject, I'd like to describe the
problem that this presents.  I'd also like to discuss a possible
solution.

The application in question (location services) requires that any
protocol it uses scales well.  Our estimates predict a wide range of
usage rates, but at the upper end the rate could be something in the
order of 100s of requests per second.  However, throughput is not the
only concern.  Requests take varying amounts of time to be processed;
current estimates are anything from milliseconds to somewhere between 10
and 20 seconds.

Case study: an unnamed protocol [1] from the same field that uses an
HTTP binding.  Similar to the application I describe, this protocol
requests can take anywhere from milliseconds to minutes to process.
Since HTTP suffers from head-of-queue blocking, in order to get
reasonable throughput, more connections are opened.  There is a very
simple formula that determines how many connections you need:

  number of connections = average response time x rate of transactions

I probably don't need to go into the details, so suffice it to say that
this is somewhat inefficient.  Grossly inefficient.  Providing a system
that serves this many connections is tricky, even though there might be
more than sufficient processing capacity to service those requests (most
of the time is spent waiting, so this isn't as hard as it sounds).

Back in the land of BEEP, channels also exhibit head-of-queue blocking.
This implies that the same solution is necessary if high throughput is
required.  It's interesting to note that some APIs provide channel
pooling features [2].  The problem is much more manageable because
channels are much cheaper than connections, however they don't cost
nothing.  And I'm not talking about system resources, but also about
development effort and maintenance.  To add to the grief, a BEEP peer
need only support 256 channels, so asking for 10000 channels is probably
pushing it.

All of this would work much more smoothly if responses could be provided
out of order.  It turns out that there is no technical or protocol
reason that forces channels to be serial.  After reading through all the
relevant documentation (all the RFCs, the BEEP book), I haven't seen any
evidence that suggests a strong technical reason for making channels
this way.  The book draws an analogy between a channel and a thread,
which is convenient for explanation but doesn't provide any
justification.

I'd like to put message numbers to use for request-response correlation.
Message number as specified only provides a means of detecting protocol
errors.  This provides a second layer of defence against a buggy peer,
after the sequence number.  Message number is not, contrary to what is
implied, used to correlate response with request; correlation can be
done by relying on response ordering alone.

I toyed with the thought of a tuning profile that enabled asynchrony,
but that has the unfortunate drawback of affecting all channels.  In
some cases the serial nature of channels is desirable.

The current solution I favour is putting the unused 'features' attribute
from the BEEP greeting to work.  If both peers provide a feature list
that includes ``async'', then they may choose to create asynchronous
channels.  A channel that is created using ``<start number="3"
async="true">...'' [3] would not be subject to the reply ordering
constraint from Section 2.6.1 of RFC 3080.  Responses sent on that
channel would be matched to their requests by the message number.

I like BEEP.  It satisfies most of the requirements.  This one small
thing is all that it lacks.  I am willing to take this to the IETF, but
I want to get a feeling for what the BEEP community thinks of the
proposal.  Would this functionality be useful where you use BEEP?  Can
you see any technical shortcomings in this proposal?

Regards,
Martin 


~~
[1]
<http://www.openmobilealliance.org/Technical/release_program/mls_v1_1.as
px>

[2]
<http://www.aspl.es/fact/files/af-arch/vortex/html/group__vortex__channe
l__pool_ga7130833a4500c9647fd84ad21dc2093.html>

[3] This wouldn't be valid XML, by the strict terms of the
specification.  Since BEEP doesn't reference the DTD, BEEP control
messages aren't valid either, so the point is probably moot.  Since BEEP
doesn't use namespaces in XML, I can't see a better way to extend the
XML syntax. 

~~
------------------------------------------------------------------------------------------------
This message is for the designated recipient only and may
contain privileged, proprietary, or otherwise private information.  
If you have received it in error, please notify the sender
immediately and delete the original.  Any unauthorized use of
this email is prohibited.
------------------------------------------------------------------------------------------------
[mf2]