Re: Sending files with E?

[email protected] (Kevin Reid) on Tuesday, October 7, 2008 wrote:

>All my work so far addresses basic stream objects and OS-level IO  
>(files, pipes and sockets) -- I haven't started on the problem of  
>efficiently streaming data over a CapTP connection.

It seems to me that any streaming of data over CapTP connections
will run into the flow-control problem. Flow control is currently
left as an application issue.

We have the E-on-Java example where a vat receives all messages
promptly and queues them on the vat queue. This technique fails
when too many messages are sent and the vat queue runs out of
storage.

The Waterken protocol offers another approach, where the messages
are queued in the sender. Flow-control can be applied by stopping
the sending vat, but a "stuck" vat can still cascade into many
stuck vats.

I have always considered the problem of a program that wants to
invoke a remote object, once for each member of some collection
class, to be the canonical problem. The naive way of implementing
this algorithm is to use an interator on the class, and for each
member send an eventual message to the remote object. For a large
collection class, this algorithm will overflow the memory for the
receiving vat.

A naive approach to application-level flow-control is to hold off
sending the second message until the remote vat has responded to
the first message. This approach, of course, looses most of the
performance advantage of message pipe-lining.

There are some intermediate approaches where a number of messages
are send, and then when the remote vat responds, more messages are
sent. These approaches could be optimized to send more than one
message per TCP packet. Programming any of these approaches sounds
messy.

Cheers - Bill

-----------------------------------------------------------------------
Bill Frantz        | There are also no libertar-  | Periwinkle
(408)356-8506      | ians in financial crises.    | 16345 Englewood Ave
www.pwpconsult.com |               - Jeff Frankel | Los Gatos, CA 95032