Re: E patches for review: ELoader, seedVat

On Fri, 2010-05-21 at 07:25 -0400, Kevin Reid wrote:
> On May 21, 2010, at 4:59, Thomas Leonard wrote:
> 
> >> 1. Envs also have the distinction which prohibits binding a noun  
> >> twice
> >> within a single scope, and is controlled by nestOuter(). This is data
> >> a ConstMap does not have.
> >
> > Is this useful when calling eval() from E?
> 
> The intent is that it should be possible to write an evaluator using  
> Env (Scope) as the environment type. This is not possible if Env does  
> not preserve this information.
[...]
> "Scope" is an incorrect and deprecated name. It is to be renamed  
> to "Env". There is no distinction between them.

Internally, I imagine an interpreter would do something like:

def interpret(code, env :Map) {
  def [transformedE, layout, nLocals] := transform(code, env)
  def scope := makeScope(env, nLocals)
  return subEval(transformedE, scope)
}

The scope includes things like evalContext.locals[], which are needed by
the interpreter.

On the other hand, another evaluator might work by generating JVM
byte-code. In that case, no Scope object is created and there is no
locals[] array.

Currently, EExpr.eval does something like:

 public Object eval(Scope scope) {
   ScopeLayout oldLayout = scope.getScopeLayout();
   Object[] triple = transform(this, oldLayout);
   ...
   Scope newScope = scope.update(nLocals);
   ...
 }

Here, the Scope type fills both roles ("scope" and "newScope"). Some
safej rules prevent E programs from accessing "internal" details (e.g.
locals[]). This means you can't write the interpreter in E (using
Scope), because E can't access these bits even when it needs them.

If they were separate types ("Map env/Scope scope" rather than "Scope
scope/Scope newScope"), the safej restrictions could go away and normal
capability rules would apply (if you have a Scope, you can call all its
methods, whether you're E or Java).

> >> Thomas, would you be interested in working on implementing guard- 
> >> based
> >> auditing in E-on-Java? It would then be straightforward to add the
> >> DeepFrozen auditor for E code to E-on-Java, which enables a variety  
> >> of
> >> useful possibilities.
> >
> > I'm a bit confused about DeepFrozen. For example, could this be
> > DeepFrozen?

> The correct syntax is
> 
> def requireOne implements DeepFrozen {
>     to run(x) {
>         require(x == 1)
>     }
> }
> 
> > It seems not, because it mutates &require:
> >
> > ? (&require).isFinal()
> > # value: false
> > ? requireOne(1)
> > ? (&require).isFinal()
> > # value: true
> >
> > Can any useful code be DeepFrozen?
> 
> This is a mistake in the implementation of safeEnv (safeScope), not in  
> DeepFrozen. It should not be possible to observe whether a lazy eval  
> slot has been forced yet, because otherwise it leaks information among  
> users of a given safeEnv when it should not (regardless of the  
> existence of DeepFrozen).

Good point. Still, it *is* mutating the slot, even if you can't see it.
Is that a problem? BootRefHandler does this:

 if (Ref.isDeepPassByCopy(arg) /* implicit "and is thread-safe" */) {
   return arg;
 }

An E object which forces a LazyEvalSlot (even a DeepFrozen one like
require) still isn't thread-safe.

> > I'm also confused about bindings. ... Is this so that auditors don't  
> > get free access to all the values?
> 
> Yes. Auditors should be able to analyze the behavior of your code, but  
> not use the authority it bears.
> 
> > If so, why not just subclass FinalSlot, e.g.
> >
> >  class AuditableFinalSlot extends FinalSlot implements AuditableSlot
> >
> > Then only give auditors auditable slots to look at (which would  
> > include
> > everything in safeScope)?
> 
> Because this would prevent auditing from working at all on custom  
> slots unless they all carefully implemented AuditableSlot and were  
> audited to do so correctly -- and what does 'correctly' mean for a  
> sufficiently wacky slot? The binding system leaves it up to the  
> auditor to determine what an acceptable slot is.
> 
> (On the other hand, the binding system does make some annoying  
> complications, like &&foo, which are the part I like least about GBA  
> as it stands now (which is entirely my own design); so if you could  
> expand on this proposal and show it works better in practice, I'd be  
> interested to hear it.)

OK, so the problem is:

- we want to know the guard (reliably)
- we want to allow custom slots
- we don't trust custom slots to enforce their claimed guard

Perhaps we could wrap wacky slots in a known wrapper? e.g.

def &myVar := makeGuardedSlot(myWackySlot)

Where makeGuardedSlot would call myWackySlot.getGuard and store a copy
of the guard. GuardedSlot would check the value against the guard every
time you called get().

An auditor could see that the slot was a GuardedSlot, and thus trust it
to ensure the values it returned always conformed to the guard. But for
the common case (FinalSlot) it would work fine anyway without wrapping.

> >> Perhaps, instead of using the persistence sealer, use the unsafe
> >> loader, which presumably can (ought to be able to, anyway) uncall
> >> makeTraceln?
> >
> > It can, but I don't want to let people get makeTraceln from <this>,  
> > e.g.
> > with:
> >
> > def loader {
> >    to __optUncall() {
> >       return [makeLoader, "run", [sourceDir, envExtras, fqnPrefix]]
> >   }
> > }
> >
> > You could recover makeLoader with:
> >
> >  def makeLoader := <this>.__optUncall()[0]
> >
> > You can then use makeLoader to create loaders that log with any  
> > fqname,
> > or uncall it to get makeTraceln. So, I think this needs to be sealed
> > somehow.
> 
> Hold on. I'm not sure how what I proposed makes a difference here.
> 
> Could you explain the entire object graph leading up to makeTraceln?

<this> is an ELoader, created by makeELoader. It holds a reference to
makeTraceln and creates traceln instances by combining a prefix with the
filename. Like this:

? def makeELoader := <elang:interp.ELoaderAuthor>(makeTraceln)
? def <libfoo> := makeELoader(dir, [].asMap(), "libfoo$")

Any emaker loaded by libfoo gets a traceln that logs things from "libfoo
$...".

Having <libfoo> should not allow unrestricted access to makeTraceln.

> I suspect a violation of the principle that any object with private  
> state must use an amplified uncall rather than __optUncall/0.

Exactly. The question is whether to amplify using the persistence sealer
or something else.


-- 
Dr Thomas Leonard
IT Innovation Centre
2 Venture Road
Southampton
Hampshire SO16 7NP

Tel: +44 0 23 8076 0834
Fax: +44 0 23 8076 0833
mailto:tal-v5nx5w6akNyLE8xUarVfuPLx9OUvmyODWmv/[email protected]
http://www.it-innovation.soton.ac.uk