Re: ANN: Etiquette: a protocol construction tool

Hi Eugene, thanks for the details.

> Luke>  Multitasking. One thread should be able to run many state
> Luke>  machines, so that when one blocks on I/O another is allowed to
> Luke>  execute.
> 
> I don't quite understand how can execution proceed in the same context
> when blocking input operation is performed.

[This turned out to be a long post..]

There are two contexts: the Lisp thread and the state machine being
executed. I want to run many state machines with a single Lisp
thread. One state machines runs at a time, but when it needs input it
is "scheduled out" and another one is executed instead.

Here's the pitch:

  Programs are simpler to write using blocking I/O than non-blocking.

  Non-blocking I/O is more scalable than blocking because it only
  needs a fixed number of threads (e.g. one).

  Frameworks like Etiquette can combine the best of both worlds by
  providing a blocking programming interface that is implemented with
  non-blocking I/O under the hood. They do this by writing a custom
  state-machine scheduler based on a "select() loop".

So you could add "blocking" read and write primitives to your
interpreter. If a state machine tries to do I/O that would block, you
capture its state and save it, then use the SERVE-EVENT select()-loop
to find out when the input is available and the machine can
continue. So the state machines are written as if they have their own
thread, but are all multiplexed onto one actual Lisp thread.

That might seems abstract, but it solves a very real problem. To get
adequate performance and robustness, a server for most protocols
_must_ handle many clients at once. Consider the problems of an SMTP
server that only serves one client at a time:

  If you do 10ms worth of processing for each message, and each
  request takes on average one second due to network latency (there
  are several round-trips), then you only spend 1% of your time doing
  actual work.

  If a client doesn't respond, e.g. because he fell off the network,
  then you are paralysed until you detect a timeout. That can take
  e.g. 30 seconds.

One solution to this is to use a select()-loop and non-blocking
I/O. This is tricky to code, but can give you great scalability. You
can talk to a thousand clients at once, and the state record for each
is just a small data structure.

Another is to use multithreading. With one thread per client, you can
use the convenient blocking-I/O interface and avoid stalling. But
threads are typically expensive, and you cannot just create a thousand
of them and still get good performance. (Not necessarily so, but I
don't think current Lisps implement very light-weight threading -- if
they do please tell me!)

A variation is to have a thread pool of a fixed maximum size, but this
is not very robust. It assumes you will never have to block on more
than N connections at a time, which breaks down with slow/unreliable
clients and with DoS attackers.

The _right_ solution is to have a system that lets you write simple
code and have it executed in a select()-loop. Then you get the nice
programming interface plus the scalability and robustness. In my
opinion implementing such an interface is _the_ most important thing
for Etiquette-like frameworks.

My perspective may be warped of course. :-)

For more information about the scalability issues, check out the "C10K
problem" page:

  http://www.kegel.com/c10k.html

Note: it's now clear that the world will never notice that Erlang has
solved all this stuff to a tee. We've been merrily using this fact
against our competitors for years :-)

Cheers,
Luke