Re: symbol lookup and overload resolution

Richard Kelly wrote:
> Stefan Seefeld wrote:
> 
>>
>> Now I'm cleaning up, refactoring, and enhancing the C++ parser with
>> the goal to make the lower level APIs (parse tree, symbol lookup, 
>> etc.) publicly usable, too.
> 
> 
> Here's an example I ran into just today.  At a certain point, my 
> prototype tool is analyzing class C and it needs to see if function f() 
> is visible.  f() might be a member function in C, or it might be a free 
> function in the scope containing C, or it might be a free function in 
> the scope containing that scope, and so on.  (I know there are other 
> possibilities, too, what with namespace using directives, Koenig lookup, 
> and all.  However, I ignored those possibilities in my first attempt.)

In the long run there should be a tool that translates the parse /syntax tree
into a high level form such as XMI (i.e. UML), so you get a high level modeling
view on things. But for now let me just assume you'v got code like

C::foobar()
{
   f();
}

and you want to know whether that is valid.
The tests/OpenCxx/src/SymbolLookup.cc applet will show you how to do that.
A 'SymbolLookup::Visitor' class (which I should rename 'Walker' once the old
Walker is removed) traverses the parse tree, adjusting the 'current scope'
accordingly. Then, when you are looking up 'f' from within C::foobar(),
you'll get back a 'SymbolSet' containing all overloaded functions matching 'f'.
The Symbol instance has a 'scope()' method that lets you get back to the scope
the symbol was declared in, which may of type SymbolLookup::Class or
SymbolLookup::Namespace.

> There didn't seem to be a ready made symbol lookup function for me to 
> use -- unless I overlooked it? -- so I wrote my own function to search 
> all the scopes from class C outwards to the global namespace.  Once your 

It is there, see above. The Parser::parse() call will return a SymbolLookup::Table
which provides a facade to symbol lookup. Once you know the current scope you
can invoke 'lookup()' on it, using the encoded name of the symbol you want to
find, and it will do the right thing (i.e. use qualified or unqualified lookup,
respect 'using' directives, etc.). It's not *quite* complete, but close.
The tests/OpenCxx/SymbolLookup/input/*.cc files provide unit tests to see
what works and what not. I'v run them manually only so far as I still need
to fix the expected output before I integrate them into the automatic tester.


> symbol lookup API is publicly usable, will I just be able to call it and 
> get back the function f() that's visible at that point, if any?

Yes.

> And if 
> there are multiple functions named f(), will I get back the set of 
> functions that would be candidates for overload resolution?  Either 
> would suffice, I think, because the analysis I'm doing should make it 
> easy to choose the right function from a set of functions.

Ok, then what you need is already there. I started to work on overload
resolution, i.e. for example I have a 'TypeEvaluator' that takes an
expression and figures out its type.
Given a set of types for all arguments, one needs to find the 'closest'
match in the available overloaded functions, using the metrics from the
C++ specifications for type conversions.

>> The first step into that direction was the creation of a new C++ 
>> library that programs can link to directly, i.e. without having to use 
>> a python frontend.
> 
> 
> That sounds sweet.  My prototype tool is written in Python, and there's 
> no particular reason now for me to worry about rewriting it in C++. 
> However, I have noticed that almost all of its time is spent in the call 
> to Cxx.Parser.process(), even for relatively small input files.  Once 
> your C++ library is stable, should I expect a speedup in parsing?

Hopefully, though your observation doesn't indicate that. Most of the
work that's currently done by Cxx.Parser.process() still needs to be done.
I have not measured how much of the time is spent in the C++-to-python
translation of the syntax tree.

>> As I said, the first benefit will be that the parser can work 
>> correctly in areas where it wasn't.
> 
> 
> Definitely.  Which reminds me, with your latest updates to Synopsis, 
> will it correctly parse all of the C++ language?  Templates, 
> specializations, Koenig lookup, virtual inheritance, etc.?  We make 
> pretty good use of the language here, and if possible I'd like to be 
> able to tell my co-workers that the tool I'm building on top of Synopsis 
> will handle any source code that they throw at it.

It will. :-)
I didn't modify the parser yet, i.e. while it is already generating
the symbol lookup table, it doesn't *use* it yet to disambiguate.

The reference code I usually use to see whether it works correctly
(<hint> in lack of good unit tests </hint>) is boost.

> Also, are your latest updates ready for use now, or should I wait a 
> while before downloading them?  Your 2005/01/29 snapshot is working 
> great for me so far, but once you feel confident in your most recent 
> work, I'd like to start using it.

Ok. It definitely requires more work, and as always I encourage everybody
to help me to get things ready.
The parser, as it is used from the python frontend, won't even create
the symbol lookup table, i.e. all the operations in SymbolLookup::Table
are basically no-ops, just so my development work won't destabilize the
parser when it is used as a document extractor.
As soon as we can be confident that symbol lookup works correctly, we
can

1. switch on symbol lookup by default
2. make the parser itself use it
3. make the symbol lookup API(s) public, in particular expose it
    to python

Regards,
		Stefan