Re: Small grammar, long compilation time

On 2017-10-30 14:15, Larry Evans wrote:
> On 10/27/2017 10:42 AM, Søren Enevoldsen wrote:
>> On 27-10-2017 16:24, Larry Evans wrote:
>>> On 10/27/2017 09:03 AM, Søren Enevoldsen wrote:
>>>> On 2017-10-27 15:35, Larry Evans wrote:
>>> [snip]
>>>
>>>>>> Thanks for the explanation and pointers Larry. But I am not sure 
>>>>>> what is happening for me.
>>>>>>
>>>>>> I am using GCC 7.2.0 and compile using:
>>>>>> g++ wccs_parser.cpp -Wall -pedantic -O2 -march=x86-64 -std=c++17 
>>>>>> -m64 -I. -flto=4 -static-libgcc -static-libstdc++ -ljemalloc -o 
>>>>>> wccs_parser
>>>>>>
>>>>>> A. Compiling without the equivalent of "USE_PROCESS_IN_PAREN" 
>>>>>> takes ~14s and ~930MB (maximum resident).
>>>>>>
>>>>>> 1. I clone your repo into ~/spirit
>>>>>> 2. I add ~/spirit to very front of my CPLUS_INCLUDE_PATH . The 
>>>>>> full boost package (1.65.1) is very last. In fact these are the 
>>>>>> only two paths there.
>>>>>> 3. I compile again and it takes ~15s and ~925MB.
>>>>>>
>>>>>> I am quite sure it now uses your ~/spirit because now I get a 
>>>>>> pragma warning from the original boost (which I did not get before):
>>>>>>
>>>>>> /home/soren/spirit/include/boost/spirit/home/x3/nonterminal/rule.hpp:23:19: 
>>>>>> note: #pragma message: not(BOOST_SPIRIT_ATTR_XFORM_IN_RULE)
>>>>>>     #pragma message "not(BOOST_SPIRIT_ATTR_XFORM_IN_RULE)"
>>>>>> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>> /home/soren/spirit/include/boost/spirit/home/x3/nonterminal/rule.hpp:28:19: 
>>>>>> note: #pragma message: not(BOOST_SPIRIT_GET_RHS_CRTP)
>>>>>>     #pragma message "not(BOOST_SPIRIT_GET_RHS_CRTP)"
>>>>>
>>>>> Ah!  I forgot to add that you need to #define those 2 macros to
>>>>> avoid the long compile times.  Otherwise, the branch behaves
>>>>> just like the existing spirit.
>>>>>
>>>>>>
>>>>>> And I can still not compile with "USE_PROCESS_IN_PAREN".
>>>>>>
>>>>>> - Søren
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------ 
>>>>>>
>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Spirit-general mailing list
>>>>>> [email protected]
>>>>>> https://lists.sourceforge.net/lists/listinfo/spirit-general
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------ 
>>>>>
>>>>> Check out the vibrant tech community on one of the world's most
>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>> _______________________________________________
>>>>> Spirit-general mailing list
>>>>> [email protected]
>>>>> https://lists.sourceforge.net/lists/listinfo/spirit-general
>>>>
>>>> That certainly sped things up. But now it fails with template 
>>>> static assertion. This is the very end, but it occurs at least 
>>>> thrice in the output.
>>>>
>>>> wccs_parser.cpp:226:59:   required from here
>>>> /home/soren/spirit/include/boost/spirit/home/x3/operator/detail/sequence.hpp:143:9: 
>>>> error: static assertion failed: Attribute does not have the 
>>>> expected size.
>>>>
>>>> Line 226:59 is essentially root, in the parse call (last line in 
>>>> below):
>>>>
>>>>          wccs_parser::AstAnyProcess root;
>>>>          auto iter = str.begin();
>>>>          auto end = str.end();
>>>>          bool r = parse(iter, end, wccs_parser::entry, root);
>>>>
>>>> This also occurs a bit up:
>>>>
>>>> .....
>>>>
>>>> ..... , 
>>>> boost::spirit::x3::forward_ast<wccs_parser::AstConstantProcess> >&)’
>>>>               rat_v.post(ok_parse,attr,attr_);
>>>>
>>>>
>>> Yes.  I was able to reproduce this too when:
>>>
>>> -DBOOST_SPIRIT_X3_EXPERIMENTAL_ATTR_XFORM_IN_RULE=1
>>>
>>> However, if you change that to:
>>>
>>> -DBOOST_SPIRIT_X3_EXPERIMENTAL_ATTR_XFORM_IN_RULE=0
>>>
>>> then it compiles without problem:
>>>
>>> --{--cut here--
>>> Compilation started at Fri Oct 27 09:16:26
>>>
>>> make compile
>>> time timeout 10.0s 
>>> /home/evansl/dwnlds/llvm/5.0/prebuilt/clang+llvm-5.0.0-linux-x86_64-ubuntu16.04/bin/clang++ 
>>> -c -std=c++14 -fdiagnostics-show-template-tree -ftemplate-depth=200 
>>> -ftemplate-backtrace-limit=0 
>>> -I/home/evansl/prog_dev/boost/releases/ro/boost_1_65_1/sandbox/pull_requests/context_debug_fix.fork/spirit/include 
>>> -I/home/evansl/prog_dev/boost/releases/ro/boost_1_65_1 
>>> -I/home/evansl/prog_dev/boost/sandbox/lje/non_variadic_templates 
>>> -DBOOST_SPIRIT_X3_EXPERIMENTAL_GET_RHS_NS=1 
>>> -DBOOST_SPIRIT_X3_EXPERIMENTAL_GET_RHS_CRTP=0 
>>> -DBOOST_SPIRIT_X3_EXPERIMENTAL_ATTR_XFORM_IN_RULE=0 wccs_parser.cpp
>>>
>>> real    0m4.026s
>>> user    0m3.884s
>>> sys    0m0.140s
>>>
>>> Compilation finished at Fri Oct 27 09:16:30
>>>
>>> --}--cut here--
>>>
>>> Thanks for this test case because I've been trying to figure out
>>> if -DBOOST_SPIRIT_X3_EXPERIMENTAL_ATTR_XFORM_IN_RULE=1 is always
>>> correct.  This shows it's not always correct.  The only reason
>>> for this macro was to solve another problem:
>>>
>>> https://stackoverflow.com/questions/43791079/x3-linker-error-with-separate-tu 
>>>
>>>
>>> I still don't understand how attributes work; so, I'll have to
>>> continue trying to decipher the attribute handling code.
>>>
>>> Sorry, that's the best I can do at the moment :(
>>>
>>> -regards,
>>> Larry
>>>
>>>
>>> ------------------------------------------------------------------------------ 
>>>
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> Spirit-general mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/spirit-general
>>
>> It works now with those define arguments. Thanks for the help.
>>
>> As someone new to Spirit, problem like these are not very user 
>> friendly. While Spirit do seem like a high quality system that an 
>> enormous amount of effort went into, errors like these suggests that 
>> it is quite "fragile" - if a minor, sensible (in my view)  rule 
>> addition can have such drastic effect. Hopefully, when your fixes are 
>> complete they will get integrated into the main branch.
>>
>> - Søren
>>
> Soren, I've just uploaded changes that, hopefully, will fix
> your problem ans well as the x3-linker-error-with-separate-tu
> problem mentioned before.  I've also uploaded your test case modified
> to take input from a std::string strs[]={}; instead of std::cin.
> Could you provide some test cases for this strs?  I might
> decide to use your code as a good test case for a patch; hence,
> it would also help if you could simplify the test case to highlight
> where the problem is.  I'm confident that would make these
> changes more palatable to the maintainers.
>
> -regards,
> Larry
>
>
>
>
>
> ------------------------------------------------------------------------------ 
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Spirit-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/spirit-general

I submitted a pull request with some strings that should parse ok (they 
do here). I had to turn off BOOST_SPIRIT_X3_DEBUG in order to compile 
the code in the link you sent.

But just to be clear, it is compilation time that is the problem, not 
any particular input string. I have been trying to simplify the case. 
Whenever I eliminate a rule in the "tower"/hierarchy of rules   Process 
::= choice and choice -> composition -> renaming -> restriction -> 
primitive, where we have the rule Process ::= choice again in 
parenthesis, the compilation time becomes practical again. I would 
assume such a hierarchy is not unusual for expressions. A naive guess is 
that it is somehow connected to the use of AstAnyProcess and the 
x3::forward_ast. But I have been unable to simplify the grammar without 
unexploding the compile time.

- Søren


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/spirit-general