Re: Jython Question

Jython has a standard compiler design:

1. Source code is parsed into an abstract syntax tree (AST) using Antlr 3.
The AST represents the source code in a more convenient form, and it can be
accessed and manipulated from Python as well via the standard ast module (
https://docs.python.org/2/library/ast.html)

2. Scope analysis is performed by visiting the AST (recursively walking
this tree). All names are identified with respect to their scope, whether
local or global, including variables that are lexically scoped (closures).

You can see the results of this scoping process in code like the following:

def f():
    x = 6
    def g():
        print "locals in g", locals()
        return x * 7
    print "locals in f", locals()
    return g() * 2

print "globals", globals()
f()

f is a global variable; x and g are locals in the context of f; and x is a
local lexically closed over in g.

3. The AST is recursively walked once more, producing Java bytecode for
every Python construct. If you decompile the produced Java bytecode to
Java, you will notice it is not at all what you have would written if you
were idiomatically translating to Java. This is because the Python language
is inherently dynamic. So this allows you to conditionally define functions
or classes, rebind a function to a given name, and so forth. We don't know
how you would use it, although we have good ideas on how to make this in
some cases much more efficient using the invokedynamic work that came about
in Java 7. It remains future work however. See this video from PyCon 2012
on the ideas we explored: https://www.youtube.com/watch?v=y_cXzaymXm0

The below bytecode corresponds to the slightly simpler

def f():
    x = 6
    def g():
        return x * 7
    return g() * 2

f()


import org.python.compiler.*;
import org.python.core.*;

@APIVersion(36)
@MTime(1441250497000L)
@Filename("scope2.py")
public class scope2$py extends PyFunctionTable implements PyRunnable
{
    static scope2$py self;
    static final PyCode f$0;
    static final PyCode f$1;
    static final PyCode g$2;

    public PyObject f$0(final PyFrame pyFrame, final ThreadState
threadState) {
        pyFrame.setline(1);
        pyFrame.setlocal("f", (PyObject)new PyFunction(pyFrame.f_globals,
Py.EmptyObjects, scope2$py.f$1, (PyObject)null));
        pyFrame.setline(7);
        pyFrame.getname("f").__call__(threadState);
        pyFrame.f_lasti = -1;
        return Py.None;
    }

    public PyObject f$1(final PyFrame pyFrame, final ThreadState
threadState) {
        pyFrame.setline(2);
        pyFrame.setderef(0, (PyObject)Py.newInteger(6));
        pyFrame.setline(3);
        pyFrame.setlocal(0, (PyObject)new PyFunction(pyFrame.f_globals,
Py.EmptyObjects, scope2$py.g$2, (PyObject)null, new PyObject[] {
pyFrame.getclosure(0) }));
        pyFrame.setline(5);
        final PyObject mul =
pyFrame.getlocal(0).__call__(threadState)._mul((PyObject)Py.newInteger(2));
        pyFrame.f_lasti = -1;
        return mul;
    }

    public PyObject g$2(final PyFrame pyFrame, final ThreadState
threadState) {
        pyFrame.setline(4);
        final PyObject mul =
pyFrame.getderef(0)._mul((PyObject)Py.newInteger(7));
        pyFrame.f_lasti = -1;
        return mul;
    }

    public scope2$py(final String s) {
        super();
        scope2$py.self = this;
        f$0 = Py.newCode(0, new String[0], s, "<module>", 0, false, false,
(PyFunctionTable)scope2$py.self, 0, (String[])null, (String[])null, 0,
4096);
        f$1 = Py.newCode(0, new String[] { "g", "x" }, s, "f", 1, false,
false, (PyFunctionTable)scope2$py.self, 1, new String[] { "x" },
(String[])null, 1, 4097);
        g$2 = Py.newCode(0, new String[0], s, "g", 3, false, false,
(PyFunctionTable)scope2$py.self, 2, (String[])null, new String[] { "x" },
0, 4097);
    }

    public PyCode getMain() {
        return scope2$py.f$0;
    }

    public static void main(final String[] array) {
        Py.runMain(CodeLoader.createSimpleBootstrap(new
scope2$py("scope2$py").getMain()), array);
    }

    public static CodeBootstrap getCodeBootstrap() {
        return
PyRunnableBootstrap.getFilenameConstructorReflectionBootstrap((Class)scope2$py.class);
    }

    public PyObject call_function(final int n, final PyFrame pyFrame, final
ThreadState threadState) {
        switch (n) {
            case 0: {
                return this.f$0(pyFrame, threadState);
            }
            case 1: {
                return this.f$1(pyFrame, threadState);
            }
            case 2: {
                return this.g$2(pyFrame, threadState);
            }
            default: {
                return null;
            }
        }
    }
}

The decompiler I'm using is the Procyon decompiler. Also IntelliJ - and
perhaps other IDEs - now include decompilation support.

For more, I recommend reviewing the source code in the compiler package,
starting with
https://github.com/jythontools/jython/blob/master/src/org/python/compiler/CodeCompiler.java

- Jim


On Wed, Sep 2, 2015 at 7:45 PM, sam patterson <[email protected]>
wrote:

> Hello,
>
> I stumbled upon the capabilities of Jython and I've become very interested
> in how exactly it works. I understand thus far that the Python source code
> is compiled into Java bytecode, but in laymans terms how does a compiler
> accomplish such things and are there any books or articles that you would
> recommend reading to learn more on the topic?
>
> Many Thanks,
>
> Sam
>
>
> ------------------------------------------------------------------------------
> Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
> Get real-time metrics from all of your servers, apps and tools
> in one place.
> SourceForge users - Click here to start your Free Trial of Datadog now!
> http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140
> _______________________________________________
> Jython-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/jython-users
>
>

------------------------------------------------------------------------------
Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140

_______________________________________________
Jython-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/jython-users