Re: Re: Re: Re: Re: Re: [Findbugs-core] toward ASM 4.0?

USER GUIDELINES TO ENSURE BACKWARD COMPATIBILITY OF YOUR CODE

SUMMARY

GUIDELINE 1: to write a ClassVisitor subclass for ASM version X, call the ClassVisitor constructor with this exact version as argument, and *never override or call methods that are deprecated* in this version of the ClassVisitor class (or that are introduced in later versions).

GUIDELINE 2: do not use inheritance of visitors, use delegation instead (i.e. visitor chains). A good practice is to make your visitor classes final by default to ensure this.

EXCEPTIONS TO GUIDELINE 2: 
- you can use inheritance of visitors if you fully control the inheritance chain yourself, and release all the classes of the hierarchy at the same time. You must then ensure that all the classes in the hierarchy are written for the same ASM version. Still, make the leaf classes of your hierarchy final.
- you can use inheritance of "visitors" if no class except the leaf ones override any visit method (for instance, if you use intermediate classes between ClassVisitor and the concrete visitor classes only to introduce convenience methods). Still, make the leaf classes of your hierarchy final (unless they do not override any visit method either; in this case provide a constructor taking an ASM version as argument so that subclasses can specify for which version they are written).

GUIDELINE 3: to write a class analyzer or adapter with the tree API of ASM version X, create your ClassNode by using the constructor with this exact version as argument (as opposed to the default constructor, without parameters).

GUIDELINE 4: to write a class analyzer or adapter with the tree API of ASM version X, using a ClassNode created by someone else, call its check() method with this exact version as argument before using the ClassNode in any way.

Guidelines 1 and 2 also apply for subclasses of ClassNode, MethodNode, etc, of Interpreter and its subclasses in asm.tree.analysis, of the ASMifierVisitor, TraceVisitor, or CheckXxxAdapter classes in asm.util, and of any class in the asm.commons package.

FULL DOCUMENT

1 Introduction

New elements have been introduced in the past in the class file format, and new elements will continue to be added in the future (e.g. for modularity, annotations on Java types, etc). Up to ASM 3.x, each such change led to backward incompatible changes in the ASM API. Binary compatibility was broken several times, which led to the coexistence of several incompatible ASM versions. This, in turn, led many users to repackage ASM directly in their product, to avoid conflicts with other parts of the software that required a different ASM version. As a result, some products now contains 3 or 4 versions of ASM repackaged in their distribution, which is rather inefficient in terms of package size (even if ASM is small). Although the modularity system planned for Java 8 might help solve this problem, it will probably force users to switch to Java 8 to be able to use this feature. This is not an option for ASM, whose users, or at least some of them, rely on the fact that it can still run on Java 1.3.

All this motivated the introduction of a new mechanism in ASM 4.0. Its goal is to ensure that all future ASM versions will remain backward compatible with any previous version, down to ASM 4.0, even when new features will be introduced to the class file format. This means that a class generator, a class analyzer or a class adapter written for one ASM version, starting from 4.0, will still be usable with any future ASM version. *However*, this property can not be ensured by ASM alone(*). It requires users to follow a few simple guidelines when writing their code. The goal of this document is to present these guidelines, and to give an idea of the internal mechanism used in ASM to ensure backward compatibility.

Note for ASM 3.x users: the backward compatibility mechanism introduced in ASM 4.0 required to change the ClassVisitor, FieldVisitor, MethodVisitor, etc interfaces to abstract classes, with a constructor taking an ASM version as argument. To convert your code to ASM 4.0, you must therefore replace "implements" with "extends" in your code analyzers and adapters, and specify an ASM version in their constructors. In addition, ClassAdapter and MethodAdapter have been merged into ClassVisitor and MethodVisitor. To convert your code, you simply need to replace ClassAdapter with ClassVisitor, and MethodAdapter with MethodVisitor. If you defined custom FieldAdapter or AnnotationAdapter classes, you can now replace them with FieldVisitor and AnnotationVisitor.

(*) in fact this can be done, but at a potentially high runtime cost. We can provide a debugging version of ASM including these potentially costly checks.

1.1 Backward compatibility contract

The user guidelines are presented in the next sections. Before this, we define here more precisely what we mean by "backward compatibility". 

First of all, it is important to study how new class file features impact code generators, analyzers and adapters. That is, independently of any implementation and binary compatibility issues, does a class generator, analyzer or adapter designed before the introduction of these new features remains valid after these modifications? Said otherwise, if we suppose that the new features are simply ignored and passed untouched through a transformation chain designed before their introduction, does this chain remains valid? In fact the impact differs for class generators, analyzers and adapters:
- class generators are not impacted: they generate code with some fixed class version, and these generated classes will remain valid with future JVM versions, because the JVM ensures backward binary compatibility.
- class analyzers may or may not be impacted. For instance, a code that analyzes the bytecode instructions, written for Java 4, will probably still work with Java 5 classes, despite the introduction of annotations. But this same code will probably no longer work with Java 7 classes, because it can not ignore the new invokedynamic instruction.
- class adapters may or may not be impacted. A dead code removal tool is not impacted by the introduction of annotations, or even by the new invokedynamic instruction. On the other hand, a class renaming tool is impacted by both.

This shows that new class file features can have an unpredictable impact on existing class analyzers or adapters. If the new features are simply ignored and passed unchanged through an analysis or transformation chain, sometimes this chain will run without errors and produce a valid result, sometimes it will run without errors but will produce an invalid result, and sometimes it will fail during execution. The second case is particularly problematic, since it breaks the analysis or transformation chain semantics without the user being aware of this. This can lead to hard to find bugs. To solve this, instead of ignoring the new features, we think it is preferable to raise an error as soon as an unknown feature is encountered in an analysis or transformation chain. The error signals that this chain may or may not work with the new class format, and that its author must analyze the situation to update it if necessary.

All this leads to the definition of the following backward compatibility contract:
- ASM version X is written for Java classes whose version is less than or equal to x. It cannot generate classes with a version y > x, and it must *fail* if given as input, in ClassReader.accept, a class whose version is > x.
- code written for ASM X and following the guidelines presented below must continue to work, unmodified, with input classes up to version x, with any future version Y > X of ASM.
- code written for ASM X and following the guidelines presented below must continue to work, unmodified, with input classes whose declared version is y but that only use features defined in versions <= x, with ASM Y or any future version.
- code written for ASM X and following the guidelines presented below must *fail* if given as input a class that uses features introduced in class versions y > x, with ASM X or any other future version.

Note that the last three points do not concern class generators, which do not have class inputs. 

1.2 An example

In order to illustrate the user guidelines and the internal ASM mechanism ensuring backward compatibility, we suppose throughout this document that two new imaginary attributes will be added to Java 8 classes, one to store the class author(s), and one to store its license. We also suppose that these new attributes will be exposed via two new methods in ClassVisitor, in ASM 5.0:

void visitLicense(String license);

to visit the license, and a new version of visitSource to visit the author at the same time as the source file name and debug information(*):

void visitSource(String author, String source, String debug);

The old visitSource method remains valid, but is declared deprecated in ASM 5.0:

@Deprecated
void visitSource(String source, String debug);

The author and license attributes are optional, i.e., calling visitLicense is not mandatory, and 'author' can be null in a visitSource call.

(*) in reality we would probably add a single visitLicense(String author, String license) method, since modifying a method signature is more complex than adding a method, as will be shown below. We do this here only for illustration purposes.

2 Core API

This section presents the guidelines that you must follow when using the core ASM API, in order to ensure that your code will remain valid with any future ASM versions (in the sense of the above contract).

First of all, if you write a class generator, you don't have any guideline to follow. For example, if you write a class generator for ASM 4.0, it will probably contain a call like "visitSource(mySource, myDebug)", and of course no call to visitLicense. If you run it unchanged with ASM 5.0, this will call the deprecated visitSource method, but the ASM 5.0 ClassWriter will internally redirect this to "visitSource(null, mySource, myDebug)", yielding the expected result (but a bit less efficiently than if you upgrade your code to call the new method directly). Likewise, the absence of a call to visitLicense will not be a problem (the generated class version will not have changed either, and classes of this version are not expected to have a license attribute).

If, on the other hand, you write a class analyzer or a class adapter, i.e. if you override the ClassVisitor class (or any other similar class like FieldVisitor or MethodVisitor), you must follow a few guidelines, presented below.

2.1 Basic rule

We consider here the simple case of a class extending directly ClassVisitor (the discussion and guidelines are the same for the other visitor classes; the case of indirect subclasses is discussed in the next section). In this case there is ony one guideline(*):

(*) this guideline can be partially enforced via reflection in the ClassVisitor constructor (the "override" part can be checked, not the "call" part).

GUIDELINE 1: to write a ClassVisitor subclass for ASM version X, call the ClassVisitor constructor with this exact version as argument, and *never override or call methods that are deprecated* in this version of the ClassVisitor class (or that are introduced in later versions).

And that's it. In our example scenario (see Section 1.2), a class adapter written for ASM 4.0 must therefore look like this:

class MyClassAdapter extends ClassVisitor {
    public MyClassAdapter(ClassVisitor cv) {
        super(ASM4, cv);
    }
    ...
    public void visitSource(String source, String debug) { // optional
        ...
        super.visitSource(source, debug); // optional
    }
}

Once updated for ASM 5.0, visitSource(String, String) must be removed, and the class must thus look like this:

class MyClassAdapter extends ClassVisitor {
    public MyClassAdapter(ClassVisitor cv) {
        super(ASM5, cv);
    }
    ...
    public void visitSource(String author, String source, String debug) { // optional
        ...
        super.visitSource(author, source, debug); // optional
    }
    public void visitLicense(String license) { // optional
        ...
        super.visitLicense(license); // optional
    }
}

How does this work? Internally, ClassVisitor is implemented as follows in ASM 4.0:

abstract class ClassVisitor {
    int api;
    ClassVisitor cv;
    public ClassVisitor(int api, ClassVisitor cv) {
        this.api = api;
        this.cv = cv;
    }
    ...
    public void visitSource(String source, String debug) {
        if (cv != null) {
            cv.visitSource(source, debug);
        }
    }
}

In ASM 5.0, this code becomes:

abstract class ClassVisitor {
    ...
    public void visitSource(String source, String debug) {
        if (api < ASM5) {
            if (cv != null) {
                cv.visitSource(source, debug);
            }
        } else {
            visitSource(null, source, debug);
        }
    }
    public void visitSource(Sring author, String source, String debug) {
        if (api < ASM5) {
            if (author == null) {
                visitSource(source, debug);
            } else {
                throw new RuntimeException();
            }
        } else {
            if (cv != null) {
                cv.visitSource(author, source, debug);
            }
        }
    }
    public void visitLicense(String license) {
        if (api < ASM5) {
            throw new RuntimeException();
        } else {
            if (cv != null) {
                cv.visitSource(source, debug);
            }
        }
    }
}

If MyClassAdapter 4.0 extends ClassVisitor 4.0, everything works as expected. If we upgrade to ASM 5.0 without changing our code, MyClassAdapter 4.0 will now extend ClassVisitor 5.0. But the "api" field will still be ASM4 < ASM5, and it is easy to see that in this case ClassVisitor 5.0 behaves like ClassVisitor 4.0 when calling visitSource(String, String). In addition, if the new visitSource method is called with a null author, the call will be redirected to the old version. Finally, if a non null author or license is found in the input class, the execution will fail, as defined in our contract (either in the new visitSource method or in visitLicense).

If we upgrade to ASM 5.0, and update our code at the same time, we now have MyClassAdapter 5.0 extending ClassVisitor 5.0. The "api" field is now ASM5, and visitLicense and the new visitSource methods behave then by simply delegating calls to the next visitor cv. In addition, the old visitSource method now redirect calls to the new visitSource method, which ensures that if an old class adapter is used before our own in a transformation chain, MyClassAdapter 5.0 will not miss this visit event.

ClassReader will always call the latest version of each visit method. Thus, no indirection will occur if we use MyClassAdapter 4.0 with ASM 4.0, or MyClassAdapter 5.0 with ASM 5.0. It is only if we use MyClassAdapter 4.0 with ASM 5.0 that an indirection occurs in ClassVisitor (at the 3rd line of new visitSource method). Thus, although old code will still work with new ASM versions, it will run a little slower. Upgrading it to use the new API will restore its performance.

2.2 Inheritance rules

The above guideline is sufficient for a direct subclass of ClassVisitor or any other similar class. For indirect subclasses, i.e. if you define a subclass A1 extending ClassVisitor, itself extended by A2, ... itself extended by An, then *all these subclasses must be written for the same ASM version* (mixing different versions in an inheritance chain could lead to several versions of the same method - like visitSource(String,String) and visitSource(String,String,String) - overriden at the same time, with potentially different behaviors, resulting in wrong or unpredictable results). If these classes come from different sources, each updated independently and released separately, this property is almost impossible to ensure(*). This leads to a second guideline:

(*) it can be partially enforced via reflection in the ClassVisitor constructor (the "override" part can be checked, not the "call" part).

GUIDELINE 2: do not use inheritance of visitors, use delegation instead (i.e. visitor chains). A good practice is to make your visitor classes final by default to ensure this.

EXCEPTIONS TO GUIDELINE 2: 
- you can use inheritance of visitors if you fully control the inheritance chain yourself, and release all the classes of the hierarchy at the same time. You must then ensure that all the classes in the hierarchy are written for the same ASM version. Still, make the leaf classes of your hierarchy final.
- you can use inheritance of "visitors" if no class except the leaf ones override any visit method (for instance, if you use intermediate classes between ClassVisitor and the concrete visitor classes only to introduce convenience methods). Still, make the leaf classes of your hierarchy final (unless they do not override any visit method either; in this case provide a constructor taking an ASM version as argument so that subclasses can specify for which version they are written).

3 Tree API

This section presents the guidelines that you must follow when using the ASM tree API. As you will see they are almost the same as the one for the core API.

Note for ASM 3.x users: the ClassNode and other similar classes now have a new constructor, in addition to the old no argument constructor. This new constructor has an ASM version as parameter. In addition, ClassNode and the other similar classes have a new method 'check' that takes an ASM version as argument. The role of these new constructors and methods is explained below.

First of all, if you write a class generator using the tree API, there is no guideline to follow (as with the core API). You can create the ClassNode and other elements with any constructor version, and use any method of these classes.

If, on the other hand, you write a class analyzer or a class adapter with the tree API, i.e. if you use a ClassNode or other similar classes populated directly or indirectly via a ClassReader.accept(), or if you override one of these classes, then you must follow a few guidelines, presented below.

3.1 Creating class nodes

We consider here the case where you create a ClassNode, populate it via a ClassReader, and then analyze or transform it before optionally writing the result with a ClassWriter (the discussion and guidelines are the same for the other node classes; analyzing or transforming a ClassNode created by someone else is discussed in the next section). In this case there is only one guideline:

GUIDELINE 3: to write a class analyzer or adapter with the tree API of ASM version X, create your ClassNode by using the constructor with this exact version as argument (as opposed to the default constructor, without parameters).

The goal of this guideline is to throw an error as soon as an unknown feature is encountered when populating the ClassNode via a ClassReader (as defined in the backward compatibility contract). If you do not follow it, your analysis or transformation code may fail later when encountering an unknown element, or it may succeed but produce a wrong result because it should not have ignored these unknown elements. In other words, the last clause of the contract may not be ensured if this guideline is not followed.

How does this work? Internally, ClassNode is implemented as follows in ASM 4.0

class ClassNode extends ClassVisitor {
    public ClassNode() {
        super(ASM4, null);
    }
    public ClassNode(int api) {
        super(api, null);
    }
    ...
    public void visitSource(String source, String debug) {
        // store source and debug in local fields ...
    }
}

In ASM 5.0, this code becomes:

class ClassNode extends ClassVisitor {
    ...
    public void visitSource(String source, String debug) {
        if (api < ASM5) {
            // store source and debug in local fields ...
        } else {
            visitSource(null, source, debug);
        }
    }
    public void visitSource(Sring author, String source, String debug) {
        if (api < ASM5) {
            if (author == null) {
                visitSource(source, debug);
            } else {
                throw new RuntimeException();
            }
        } else {
            // store author, source and debug in local fields ...
        }
    }
    public void visitLicense(String license) {
        if (api < ASM5) {
            throw new RuntimeException();
        } else {
            // store license in local fields ...
        }
    }
}

If you use ASM 4.0, creating a ClassNode(ASM4) does nothing special. But if you upgrade to ASM 5.0, without changing your code, you will get a ClassNode 5.0 whose "api" field will be ASM4 < ASM5. It is then easy to see that if the input class contains a non null author or license attribute, populating the ClassNode via a ClassReader will fail, as defined in our contract. If you also upgrade your code, changing the "api" field to ASM5 and also updating the rest of the code to take these new attributes into account, then no errors will be thrown when populating the node.

Note that the ClassNode 5.0 code is very similar to the ClassVisitor 5.0 code. This is to ensure a proper semantics if you define subclasses of ClassNode (similarly to subclasses of ClassVisitor - see Section 3.3).

3.2 Using existing class nodes

If your class analyzer or adapter receives a ClassNode created by someone else, then you cannot be sure of the ASM version that was passed to its constructor when it was created, if any. You could check the "api" field yourself, but if you find that this version is higher than the version you support, simply rejecting the class would be too conservative. Indeed, it may happen that this class does not contain any unknown feature. On the other hand, you cannot test if unknown features are present or not (in our example scenario, how could you test, when writing code for ASM 4.0, that the unknown "license" field is not present in your ClassNode, since you do not know at this stage that such a field will be added in the future?). The 'check' method is designed to solve this issue. This leads to the following guideline:

GUIDELINE 4: to write a class analyzer or adapter with the tree API of ASM version X, using a ClassNode created by someone else, call its check() method with this exact version as argument before using the ClassNode in any way.

The goal is the same as for guideline 3. The last clause of the contract may not be ensured if this guideline is not followed.

How does this work? Internally, the check method is implemented as follows in ASM 4.0

class ClassNode {
    ...
    public void check(int api) {
        // nothing to do
    }
}

In ASM 5.0 this code becomes:

class ClassNode {
    ...
    public void check(int api) {
        if (api < ASM5 && (author != null || license != null)) {
            throw new RuntimeException();
        }
    }
}

If your code is written for ASM 4.0, and if you get a ClassNode 4.0, whose "api" field will be ASM4, there will be no problem and "check" does nothing. But if you get a ClassNode 5.0, the check(ASM4) method will fail if this node actually contains a non null author or license, i.e. if it contains new features that were unknown in ASM 4.0.

Note: this guideline can also be used if you create the ClassNode yourself. Then you don't need to follow guideline 3, i.e. you don't need to specify an ASM version in the ClassNode constructor. The checks will occur instead in the "check" method (but this may be less efficient that doing the checks earlier, when populating the ClassNode).

3.3 Writing ClassNode subclasses

If you want to provide subclasses of ClassNode or other similar node classes, then guidelines 1 and 2 apply. 

Note that, in the special case, often used, of a MethodNode anonymous subclass whose visitEnd() method is overriden:

class MyClassVisitor extends ClassVisitor {
    ...
    public MethodVisitor visitMethod(...) {
        final MethodVisitor mv = super.visitMethod(...);
        if (mv != null) {
            return new MethodNode(ASM4) {
                public void visitEnd() {
                    // perform a transformation
                    accept(mv);
                }
            }
        }
        return mv;
    }
}

then guideline 2 is automatically enforced (the anonymous class cannot be overriden although it is not explicitely declared final). You simply need to follow guideline 3, i.e. specify an ASM version in the MethodNode constructor (or follow guideline 4, i.e. call check(ASM4) before performing the transformation).

4 Other packages

The classes in asm.util and asm.commons have two variants of each constructor: one with and one without an ASM version parameter.

If you simply want to instantiate and use as is the ASMifierVisitor, TraceVisitor, or CheckXxxAdapter classes in asm.util, or any class in the asm.commons package, then you can instantiate them with a constructor without an ASM version parameter. You could also use a constructor with an ASM version parameter, but this would unnecessarily restrict these components to the specified ASM version (while using the no-arg constructor is equivalent to say "use the latest ASM version"). This is why the constructors using an ASM version parameter are "protected".

If, on the other hand, you want to override the ASMifierVisitor, TraceVisitor, or CheckXxxAdapter classes in asm.util, or any class in the asm.commons package, then the guidelines 1 and 2 apply. In particular, your constructor *must* call super(...) with the ASM version you want to use as parameter.

Finally, the same distinction must be made if you want to use vs. override the Interpreter class or its subclasses in asm.tree.analysis. Note also that before using the analysis package you must create a MethodNode or get one from someone else, and that guidelines 3 and 4 must be used here before passing this node to an Analyzer.

-- 
You receive this message as a subscriber of the [email protected] mailing list.
To unsubscribe: mailto:[email protected]
For general help: mailto:[email protected]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws