Re: bison for nlp

Hi!

> Le 19 nov. 2018 à 16:09, r0ller <[email protected]> a écrit :
> 
> Hi Akim,
> 
> I managed to take the first step and get it running but it wasn’t as easy as I thought.

Sorry about that :(

> First, I wanted to take the approach that the 'Simple C++ Example' demonstrates in the bison manual. However, I could not figure out what my yylex() should return when defining api.value.type variant.

What exactly was not clear on this?  I don’t recommend not using api.token.constructor, but if you don’t see this bit of the documentation:

https://www.gnu.org/software/bison/manual/bison.html#Split-Symbols

The example is:

[0-9]+   {
           yylval->emplace (text_to_int (yytext));
           return yy::parser::token::INTEGER;
         }
[a-z]+   {
           yylval->emplace (yytext);
           return yy::parser::token::IDENTIFIER;
         }


> The question is, in case of having many tokens like I do, how do I decide which shall be returned, as bison now generates a symbol_type make_TOKEN() for each token which I shall be able to return in yylex().

I’m not sure I understand the question.  The signature of yylex is completely different with and without api.token.constructor.  So it’s all or nothing, you can’t expect to be incremental on this aspect.

I certainly agree this is a painful migration, but I do believe you’ll find it is worth it.

> Though, I'd rather not put a huge switch() in yylex(). Is there any other solution like defining a "dummy" token like
> 
> %token <int> INT;
> 
> whose constructor make_INT(const& int) would simply return the int passed to it? Or shall I simply try to cast the integers of my tokens to symbol_type?

Once you decided to move to api.token.constructor, use make_FOO and only make_FOO.



> The other problem I ran into was related to the non-terminals: wherever I wanted to read the value of a symbol in an action via e.g. $1, I got an error about type conversion as it could not be converted any more to an integer as in the C parser.

Sorry, I don’t understand what you mean.  If you declared the nterm to be an int, then $1 is an int.


> For this I have only one guess namely, that each non-terminal needs a %type declaration like
> 
> %type <int> ENG_Con;
> 
> and even the = operator needs to be defined for it, right? So here I got stuck at least with regards to api.value.type variant.

I would need more details (read: code) to understand your problem.



> Then I decided to take a step back and not to use complete symbols but split symbols for a first try. This I managed to figure out and make it work but with a small hack as I declared yylex as:
> 
> int yylex(int* yylval);
> 
> If I did it like:
> 
> int yylex(semantic_type* yylval);
> 
> the compiler kept complaining about not knowing semantic_type (nor parser::semantic_type, nor yy::parser::semantic_type). So I took clang’s hint when it said semantic_type* is aka int* and it worked.

Woot?  I doubt that you are only using ints as semantic value???

If the compiler did not know about yy::parser::semantic_type, maybe it’s because you used %{…%} instead of %code requires {…}.  Please, be careful when imitating simple.yy and the others.


> In the end, to make my hack a bit more nicer, I added %define api.value.type {int}. 

So you really just have ints?  Then, yes, variants are overkill.  Unless you already know that at some point you will have more than ints.


_______________________________________________
[email protected] https://lists.gnu.org/mailman/listinfo/help-bison