Parsing generics painfully

Mark Raynsford <[email protected]> Wed, 1 Nov 2017 18:29:35 +0000
Newsgroups	gmane.comp.java.objectweb.asm
Organization	io7m.com
Message-ID	<[email protected]>
Hello.

Does anyone have any example code regarding parsing generics into an
AST-like structure? I'm having extreme difficulty correctly building
ASTs from the event-based SignatureVisitor, and the lack of
documentation is really not helpful at all. For example, let's assume
a class like this:

  class T<A extends List<int[][][]>> { }

The documentation for visitArrayType() says: "Visits a signature
corresponding to an array type.". This says nothing about how many
times that method will be called, or when it will be called, or what it
actually means when it is called! Through painful trial and error,
here's the order of calls I get when parsing the above:

TRACE com.io7m.cantoria.api.CGenericsParsing: parseClassGenericParameters: <A::Ljava/util/List<[[[I>;>Ljava/lang/Object;
TRACE com.io7m.cantoria.api.CGenericsParsing$ClassGenericVisitor: visitFormalTypeParameter: A
TRACE class com.io7m.cantoria.api.CGenericsParsing$ClassBoundVisitor[0]: created
TRACE com.io7m.cantoria.api.CGenericsParsing$ClassGenericVisitor: visitInterfaceBound
TRACE class com.io7m.cantoria.api.CGenericsParsing$ClassBoundVisitor[0]: visitClassType: java/util/List
TRACE com.io7m.cantoria.api.CClassRegistry: findClass: java.util:List
TRACE com.io7m.cantoria.api.CClassRegistry: findClassInModule: java.base/java.util:List
TRACE class com.io7m.cantoria.api.CGenericsParsing$ClassBoundVisitor[0]: visitTypeArgument: =
TRACE com.io7m.cantoria.api.CGenericsParsing$TypeArgumentVisitor: created (=)
TRACE com.io7m.cantoria.api.CGenericsParsing$ArrayVisitor: created
TRACE com.io7m.cantoria.api.CGenericsParsing$TypeArgumentVisitor: visitArrayType
TRACE com.io7m.cantoria.api.CGenericsParsing$ArrayVisitor: visitArrayType: 1
TRACE com.io7m.cantoria.api.CGenericsParsing$ArrayVisitor: visitArrayType: 2
TRACE com.io7m.cantoria.api.CGenericsParsing$ArrayVisitor: visitBaseType: I
TRACE class com.io7m.cantoria.api.CGenericsParsing$ClassBoundVisitor[0]: visitEnd

So essentially, my code encounters the type parameter "A" and then
creates a ClassBoundVisitor to visit the actual bounds (returned from visitClassBound()
and visitInterfaceBound() methods). The ClassBoundVisitor then encounters the
List<...> bound, and instantiates a TypeArgumentVisitor to visit the
arguments to List. The TypeArgumentVisitor then instantiates an ArrayVisitor
to visit the int[][][] type, and it appears visitArrayType is called
once per [], except for the first [].

I'm having immense difficulty coming up with ways to pass data between the 
instantiated visitors, and most of the time there's no way to know that
a particular visitor is "finished". For example, you can see in the
call trace above that the original ClassBoundVisitor has its visitEnd
method called, but the ArrayVisitor instance never receives any kind of
"end" notification. This makes it very difficult to correctly nest values
and to construct AST nodes.

Am I simply not using the API correctly?

Here's a set of algebraic data types matching the definitions of TypeParameter,
TypeArgument, etc, from JLS 9 section 4.*:

  https://raw.githubusercontent.com/io7m/cantoria/develop/notes/Generics.hs

Here's the equivalent Java types that I'm using as my AST (described
as Immutables.org types):

  https://github.com/io7m/cantoria/blob/develop/com.io7m.cantoria.api/src/main/java/com/io7m/cantoria/api/CGGenericsType.java

Here's 700 lines of extremely painful code that tries to transform the
SignatureVisitor method calls into the Java AST:

  https://raw.githubusercontent.com/io7m/cantoria/develop/com.io7m.cantoria.api/src/main/java/com/io7m/cantoria/api/CGenericsParsing.java

I have absolutely zero confidence in the correctness of that code
(in the case of arrays, it's downright incomplete and incorrect), and
zero confidence that I'll be able to maintain it six months from now
after I've forgotten the semantics of the SignatureVisitor. You can
see that I've gone as far as to create an abstract UnreachableVisitor
base class so that I simply get exceptions when a method on the visitor
is unexpectedly called.

Am I suffering needlessly? Is there some better way to do this?

For the rest of the project, I'm using the plain tree API, and 
that's a joy to work with. I'm kind of surprised that there doesn't
appear to be an an equivalent tree API for generics...

-- 
Mark Raynsford | http://www.io7m.com
signature.asc (application/pgp-signature, 833 B)