Re: Ref.whenResolved

On 5 December 2010 20:53, Mark S. Miller <[email protected]> wrote:
> On Sun, Dec 5, 2010 at 8:55 AM, Thomas Leonard <[email protected]> wrote:
>>
>> Ref.whenResolved makes this guarantee:
>>
>>     * Should the reactor be invoked with a
>>     * non-broken value (and therefore a fulfilled value), all earlier
>> messages
>>     * sent on ref before the whenResolved are guaranteed to have been
>>     * successfully delivered.
>>
>> This seems excessive. For example, if you do this (to ensure that
>> farRef is resolved):
>>
>>  when (farRef) -> { ... }
>>
>> it forces a network round-trip before the when block executes.
>
> Yes, that's intentional for the "when" expression. I agree that it conflates
> two separate issues together, and makes code that only cares about the
> resolution pay for a round trip delay it might not care about. As a higher
> level convenience, this bundling seems safer, as it better supports the
> intuition "use promise pipelining to do optimistic data flow chaining, but
> use the 'when' expression to turn data flow back into control flow, where
> the success block only happens 'after' previous stuff on this reference."

I started by only doing the optimisation in the (common) case where
the when is the first thing that happens (i.e. you get two
__whenMoreResolved messages in a row), but I couldn't think of a
convincing case where it would matter if a when block executed
earlier.

But it probably depends what the programmer was expecting. I've been thinking of

  when (x) -> { ... }

as meaning "when x is resolved", rather than "when all messages
already sent to x's vat have been delivered".

> If we did unbundle this, then IMO we'd still need a convenient way to say
> "after this reference is fulfilled and all previously sent messages have
> been delivered". My general heuristic regarding ordering is to provide the
> programmer by default with the strongest ordering that they
> might inadvertently expect and that we can cheaply-enough enforce,
> especially if implementations will often accidentally obey this stronger
> ordering anyway. To do less is to invite more bugs which happen rarely. The
> limits of this are a bit more than E-order but still well short of causal
> order.
> Given the current primitives you can code your cheaper-but-weaker operation
> as
> def fastWhenResolved(ref, reactor) {
>     if (Ref.isResolved(ref)) {
>         return reactor <- (ref) # skips round trip
>     } else {
>         return Ref.whenResolved(ref, reactor)
>     }
> }
>
>>
>> And if you do:
>>
>>  when (def results := rcvr<-getData()) -> { ... }
>>
>> it transmits the result data over the network twice!
>
> I don't follow you. How/why does the result get transmitted twice?

  def results := rcvr<-getData()

sends a deliverOp, including a resolver for 'results'. The remote
system will do this to send the data back:

  results__resolver.resolve("the returned data goes here")

Then, since results is still an OldRemotePromise in the original vat
at this point,

  when (results) -> { println(results) }

does something like:

  results<-__whenMoreResolved(fn theDataAgain {
    println(results)
  })

This sends a deliverOnly message following the original call on the
wire. The remote data will then copy itself back again in its
__whenMoreResolved method.

>> While trying to simplify the causality traces, I added a
>> Ref.whenResolved(reactor) instance method.
>
> It can't be exposed in E as an instance method, since references don't have
> their own methods. So if you provide this directly in Java, even in Java you
> should expose as public only another static method on Ref, and then the
> static method would internally do the polymorphic dispatch based on a
> non-public instance method. That way, all the Java clients outside Ref would
> use the static method just as the non-Java clients would. This helps us
> prepare to switch from the current Java-based captp to Kevin's E-based captp
> implementation.

Might need to rename it then (whenResolvedOnly?), as there's already a
static whenResolved method that does extra stuff.

>> This does E.send(reactor,
>> "run", resolution) as soon as the resolution is known. e.g. a NearRef
>> does:
>>
>>    public Throwable whenResolved(Object reactor) {
>>        return E.sendAllOnly(reactor, "run", new Object[] {myTarget});
>>    }
>>
>> SwitchableRef and OldRemotePromise queue up reactors locally and call
>> resolutionRef.whenResolved() once committed.
>
> Yes, this does motivate why you want a new primitive rather than using
> the fastWhenResolved pattern above -- your primitive will fire earlier in
> these cases.
>
>> In my previous example (invoking a simple increment() method in a
>> remote vat), this reduces the number of turns from 11 to 6, makes the
>> causality graph a simple chain (no fork), and only sends the result
>> value over the network only once.
>>
>> Before, a deliverOp used four turns:
>>
>> 1. Receive CapTP deliverOp message:
>>    Ref answer := E.send(target, verb, args)
>>    E.send(answer, "__whenMoreResolved", rdr) # (tell the remote
>> caller the result)
>> 2. The actual method is invoked. answer resolves, enqueuing the
>> __whenMoreResolved on the vat (now = false)
>> 3. PendingDelivery does E.sendAll(result, "__whenMoreResolved") (now =
>> true)
>> 4. __whenMoreResolved does E.send(rdr, "run", self)
>>
>> Now, it uses two:
>>
>> 1. Receive CapTP deliverOp message:
>>    Ref answer := E.send(target, verb, args)
>>    answer.whenResolved(rdr) # (tell the remote caller the result)
>> 2.  The actual method is invoked. answer resolves, enqueuing the return
>> message
>
> By "enqueuing the return message", do you mean the same "rdr <- (answer)"
> message as sent in the first scenario?

Yes.

> I suppose the main (only?) thing I find jarring in your technique is the
> extra bookkeeping within SwitchableRef needed to remember these reactors.
> Since promises are presumed to be used much more often locally than
> remotely, this extra bookkeeping for the local case seems unfortunate.
> Perhaps my reluctance here is misplaced? Perhaps in the local case, the turn
> reduction more than pays for the extra bookkeeping?

The local (SwitchableRef) case is less important than the OldRemoteRef
one, I think. Though the overhead is only a null pointer until you use
a when block on it (at which point it allocates a FlexList). But I
suspect that is less overhead than an extra turn.

>> I'm wondering whether it makes sense to do this always, not just when
>> debugging?
>
> I don't think it makes sense to do this just when debugging. We should
> either do this always or never. Regarding the extra noise in the causeway
> trace logs, the Causeway filtering logic *should* simplify this into the
> equivalent display -- if the appropriate infrastructure files are unchecked
> in the filter dialog. I say "*should*" because I don't know if Causeway
> currently does. We should test on some actual logs. If Causeway doesn't,
> this may simply call for some adjustment to Causeway's filtering logic.

There are some filters in the Tools menu, but they're always shaded out.

>> What was the rationale for requiring previous messages to be delivered
>> before running the when block?
>
> Above ordering heuristic.
>
>>
>> What is the "now" flag for for messages enqueued on a vat, and why
>> does Vat.qSendMsg set it to false?
>
> The equivalent question just came up as Tyler, Terry, and I were looking at
> Waterken which has no equivalent flag or extra turn. I know I added it to
> maintain some ordering guarantee, but I don't remember the specifics. I will
> investigate.

Thanks.


-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

_______________________________________________
e-lang mailing list
[email protected]
http://www.eros-os.org/mailman/listinfo/e-lang