Re: how to extend a capability on a data source?

I apologize for not being grounded in many of the ideas you all explore here.  Frankly, I’m a hack and an imperitivist. ;)   This is a multi-faceted conceptual domain.  I do not truly understand, to the degree of differentiation needed, data flow, semi-data-flow and control-flow, especially the various underlying mechanisms data flow is implemented upon (lazy-lists, message-passing, generators), though I tried reading up on them to a degree.

From reading on elib, particularly the section on turning control-flow to semi-data-flow: http://erights.org/elib/concurrency/msg-passing.html, the reason it is termed semi-data-flow is evidently that just the receiver needs to resolve before the send is evaluated.  I tried to enforce argument resolution into my MessageSend, but that creates issues, primarily that computation blocks until all promises are resolved, instead of allowing for internal sending, thus continuing the dispersal of eventual sending.  A desirable eventuality, but is it bounded?  Through my sunglasses, we need an implicit and biased mechanism of apoptosis, to keep the tumors small, combined with replication and high-availability, the way MapReduce workers are managed in Hadoop.

On the other hand, how can one distinguish that an arg is immediate or eventual, since only eventual sends are allowed against an eventual ref, no immediate calls.  So what would happen if full data flow is not implemented and one of the args is eventual, meanwhile the code attempts to make an immediate call to that arg?  Coerce it into an eventual send?

It has always been so confusing to me, that I am confused about my confusion.  Aleph 3.

Perhaps there is a better term than “tracking”.  What I think of when I introduce this term is precisely what you emphasize as the benefit of your full data flow: maintaining a connection between the source and the receiver.  This begs the question as to what exactly is a source and a receiver.  I was thinking of “tracking” the send, itself, in some respects, and being able to manage it.  Introspect the send graph at runtime.  On reflection, it seems we are talking about “tracking” refs, and being able to revoke and audit them.

Could you mention the deficiencies of message passing, that is problematic to implementing data-flow and perhaps touch on lazy-lists and generators, as well your choice of implementation as a spreadsheet?  The E semi-data-flow, more specifically my attempt at understanding it and implementing it, is focussed around references to objects, promises and resolvers (continuations).  In augmenting the call-stack to a send-queue, as well as involving a minimum of 3 refs when stretching this send-queue frame between event-loops (receiver, result-value-promise, resolver-continuation), connection seems to be maintained between source and receiver, using straight message passing.  Am I confused? 

Adding the ability to register the actual send in the connection’s scope, would give the ability to manage the dispersed send-queue.  Supporting the feature of registering event listeners on the MessageSend, before sending to the receiver, and when it is sent, the event registrations would stretch as well, gives us monitoring.  The ability to monitor and manage remote sends (execution send-queue frames), in addition to the refs, is the way I think about how to bring about coherent replication, failover and high availability.

I read about attenuation, with a filtering proxy capability, and that sounds like just the ticket for offering a restricted capability on a data source.  It was even mentioned that attenuation can be achieved with membranes.

thank you,
rob

On Nov 2, 2013, at 11:43 AM, David Barbour <[email protected]> wrote:

> If an abstraction is implemented correctly, then only the properties of the abstraction are relevant. (Otherwise, you have a leak.) Consequently, it is irrelevant whether dataflow is implemented above message passing vs. lazy lists vs. generators or anything else. What matters is that dataflow has different properties than messaging.
> 
> In this case, the relevant property is that messages are "fire and forget", and lack any means to update after sending.
> 
> Dataflow, meanwhile, is like spreadsheets: changes propagate, and thus a connection remains between information source and receiver. If capabilities are passed through a dataflow model, that connection provides a means for arbitrary update of the capability, including revocation. This property can be systematically enforced.
> 
> I don't know what you envision regarding tracking of eventual send.
> On Nov 2, 2013 11:57 AM, "Rob Withers" <[email protected]> wrote:
> Thanks for reminding me of the wiki.  Isn’t data-flow built using message passing, typically?  How is streaming different than message passing?  
> 
> What is quite interesting to me, is the general concept of being able to “track” and interact with distributed eventual computations.  This includes many of the features you allude too, like revocability, membranes, audibility, as well as just promise sending itself.  Having a continuation that reports the result back to the sender could also be considered in this category, perhaps.
> 
> I have been working with kafka and hadoop, lately.  Both are highly-available systems, with replication.  How could “tracking” an eventual send be made highly-available?
> 
> thank you,
> rob
> 
> 
> On Nov 2, 2013, at 6:11 AM, David Barbour <[email protected]> wrote:
> 
>> Start with a simpler concept: revocable capability pattern. 
>> 
>> http://wiki.erights.org/wiki/Walnut/Secure_Distributed_Computing/Capability_Patterns#Revocable_Capabilities
>> 
>> We can call this 'explicit' because we must explicitly call 'makeCapabilityRevokerPair', and we must explicitly track the revoker. If we didn't have the foresight to do this, we're out of luck (unless we stop our whole machine). We can call this 'stateful' because of how revocation is modeled: changing mutable state on a reference. 
>> 
>> The revocable capability pattern is easy to extend. For example, instead of just 'revoke', we could introduce methods that allow us to install auditors or switch the capability at a later time. Another extension to the revocable capability pattern is the membrane pattern, which is essentially a deeply recursive revocation pattern, which also touches any capabilities one shares through the revoker:
>> 
>> http://wiki.erights.org/wiki/Walnut/Secure_Distributed_Computing/Capability_Patterns#Membranes
>> 
>> It is worth noting that, by nature of being deeply recursive, membrane patterns are a lot more difficult to extend with auditing or other features in a sensible way. Anyhow, that's an explicit, stateful membrane.
>> 
>> I do something very different here, which removes the need for explicit revokers and membranes, and is also much more expressive for precise auditing and so on. However, this is accomplished in part because I abandoned "message passing" as a basis for communication, and switched to a more streaming/dataflow model that implicitly maintains connectivity, like a spreadsheet for the real world.   
>> 
>> 
>> 
>> On Sat, Nov 2, 2013 at 2:08 AM, Rob Withers <[email protected]> wrote:
>> What is a stateful, explicit membrane and what does one look & feel like?
>> 
>> thank you kindly,
>> rob
>> 
>> On Nov 1, 2013, at 9:33 AM, Mark S. Miller <[email protected]> wrote:
>> 
>>> Well, when you do. I've got a zillion questions, but they're not worth formulating until I can stare at an example ;).
>>> 
>>> 
>>> On Fri, Nov 1, 2013 at 7:12 AM, David Barbour <[email protected]> wrote:
>>> The implicit membrane doesn't look like a membrane. I suppose some Kerberos ticket-granting strategies might be a close analogy. Tickets expire, but in some cases can be renewed and replaced, with the expectation that users switch to the new ticket before the old one expires. In the same sense, distributed capabilities in an RDP system are periodically expired and replaced - with a little bit of overlap for smooth, latency-tolerant transitions.
>>> 
>>> Anyhow, I don't have time at the moment to build a proper example. Sorry.
>>> 
>>> 
>>> 
>>> On Fri, Nov 1, 2013 at 7:55 AM, Mark S. Miller <[email protected]> wrote:
>>> Hi David, what does this implicit membrane look like in your model? Could you post an example?
>>> 
>>> 
>>> On Fri, Nov 1, 2013 at 5:43 AM, David Barbour <[email protected]> wrote:
>>> I am developing a system where all capabilities grants are uniformly visible, revocable, auditable, attenuable. It is based on a reactive programming model: a capability must be continuously, declaratively granted. Conceptually, the old capability is continuously expired, and a new one continuously replaces it. Of course, I can't actually implement "continuous" revocation, but the implementation can approximate continuous revocation by replacing and expiring capabilities periodically or randomly or whenever changes are needed. 
>>> 
>>> A consequence is that capabilities in this model are not used as persistent values, and the risk of a temporary capability leak is mitigated. Explicit, stateful membranes don't seem to be necessary in this design, since the implicit membrane based on dataflow seems to cover all use-cases. However, user-defined security policies - i.e. the code that controls how the capabilities are distributed - can be persistent. 
>>> 
>>> Best,
>>> 
>>> Dave
>>> 
>>> On Fri, Nov 1, 2013 at 2:31 AM, William ML Leslie <[email protected]> wrote:
>>> On 1 November 2013 17:43, Rob Withers <[email protected]> wrote:
>>> > the widget isn’t as nearly important as the relationships one builds.
>>> 
>>> Well, that's an interesting point.  Has anyone got a service where
>>> users are creating their own membranes?  Have any of you attempted to
>>> store user-defined membranes in a relational database?  How is it
>>> done?
>>> 
>>> --
>>> William Leslie
>>> 
>>> Notice:
>>> Likely much of this email is, by the nature of copyright, covered
>>> under copyright law.  You absolutely may reproduce any part of it in
>>> accordance with the copyright law of the nation you are reading this
>>> in.  Any attempt to deny you those rights would be illegal without
>>> prior contractual agreement.
>>> 
>>> _______________________________________________
>>> cap-talk mailing list
>>> [email protected]
>>> http://www.eros-os.org/mailman/listinfo/cap-talk
>>> 
>>> 
>>> _______________________________________________
>>> cap-talk mailing list
>>> [email protected]
>>> http://www.eros-os.org/mailman/listinfo/cap-talk
>>> 
>>> 
>>> 
>>> 
>>> -- 
>>>     Cheers,
>>>     --MarkM
>>> 
>>> _______________________________________________
>>> cap-talk mailing list
>>> [email protected]
>>> http://www.eros-os.org/mailman/listinfo/cap-talk
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> cap-talk mailing list
>>> [email protected]
>>> http://www.eros-os.org/mailman/listinfo/cap-talk
>>> 
>>> 
>>> 
>>> 
>>> -- 
>>>     Cheers,
>>>     --MarkM
>>> _______________________________________________
>>> cap-talk mailing list
>>> [email protected]
>>> http://www.eros-os.org/mailman/listinfo/cap-talk
>> 
>> 
>> _______________________________________________
>> cap-talk mailing list
>> [email protected]
>> http://www.eros-os.org/mailman/listinfo/cap-talk
>> 
>> 
>> _______________________________________________
>> cap-talk mailing list
>> [email protected]
>> http://www.eros-os.org/mailman/listinfo/cap-talk
> 
> 
> _______________________________________________
> cap-talk mailing list
> [email protected]
> http://www.eros-os.org/mailman/listinfo/cap-talk
> 
> _______________________________________________
> cap-talk mailing list
> [email protected]
> http://www.eros-os.org/mailman/listinfo/cap-talk

_______________________________________________
cap-talk mailing list
[email protected]
http://www.eros-os.org/mailman/listinfo/cap-talk