Re: Small grammar, long compilation time

On 2017-10-27 15:35, Larry Evans wrote:
> On 10/27/2017 08:24 AM, Søren Enevoldsen wrote:
>> On 2017-10-27 14:41, Larry Evans wrote:
>>> On 10/26/2017 04:33 PM, Larry Evans wrote:
>>>> On 10/26/2017 09:03 AM, Søren Enevoldsen wrote:
>>>>> The following program takes about 10s to compile. However, the 
>>>>> rule currently used in parenProcess, 'nullProcess' should actually 
>>>>> be 'process'. However, when I make the change, compilation does 
>>>>> not finish. Instead CPU keeps spinning hot, and memory usage 
>>>>> gradually increases. Since it is a rather small grammar, I am 
>>>>> guessing I am doing something wrong. How do I make the grammar 
>>>>> work then I correct it by changing parenProcess to '(' > process > 
>>>>> ')' ? (Note I cannot reduce the number of rules between process 
>>>>> and parenProcess for a smaller example, since then the compile 
>>>>> time immediately decreases).
>>>>>
>>>>> - Søren
>>>>>
>>>>> ====================================================================
>>>>>
>>>>> #include <boost/config/warning_disable.hpp>
>>>>> #include <boost/spirit/home/x3.hpp>
>>>>> #include <boost/spirit/home/x3/support/ast/variant.hpp>
>>>>> #include <boost/spirit/include/qi_char_class.hpp>
>>>>> #include <boost/fusion/include/adapt_struct.hpp>
>>>> [snip]
>>>>> x3::rule<struct processRule, AstAnyProcess> process{"process"};
>>>>>
>>>>> ///  Here be compile time dragons.
>>>>> auto const parenProcess = '(' > nullProcess > ')';
>>>>>
>>>>> auto const primitive = rule<AstAnyProcess>("primitive")
>>>>>      = parenProcess
>>>>>      | nullProcess
>>>>>      | constant;
>>>> [snip]
>>>> With this "convenience" change:
>>>>
>>>> #define USE_PROCESS_IN_PAREN
>>>> auto const parenProcess
>>>>    = '('
>>>> #ifdef USE_PROCESS_IN_PAREN
>>>>    ///  Here be compile time dragons.
>>>>    > process
>>>> #else
>>>>    > nullProcess
>>>> #endif
>>>>    > ')'
>>>>    ;
>>>>
>>>> and when compiled, it did take more than 10s (way more)
>>>> and I did see my memory use go up and 100% of some of
>>>> my cpu's.  However, when the get_rhs branch of a spirit
>>>> fork here:
>>>>
>>>> https://github.com/cppljevans/spirit/tree/get_rhs
>>>>
>>>> was -I included in front of the standard boost,
>>>> it compiled in a flash.
>>>>
>>>> This problem has been reported before; however, I've
>>>> been advised that it needs more documentation before
>>>> submitting it as a pull request.  I'm working slowly
>>>> on that.
>>>>
>>>> HTH.
>>>>
>>>> -regards,
>>>> Larry
>>>
>>> If you limit the instantiation depth to 200, you can
>>> get some idea of why it fails as shown in the attached.
>>> In brief, the context is being repeatedly augmented
>>> by the make_unique_context leading to the excessive
>>> instantiations.  The get_rhs branch avoids that
>>> repeated call to make_unique_context.
>>>
>>> -regards,
>>> Larry
>>>
>>>
>>> ------------------------------------------------------------------------------ 
>>>
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>
>>>
>>> _______________________________________________
>>> Spirit-general mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/spirit-general
>>
>> Thanks for the explanation and pointers Larry. But I am not sure what 
>> is happening for me.
>>
>> I am using GCC 7.2.0 and compile using:
>> g++ wccs_parser.cpp -Wall -pedantic -O2 -march=x86-64 -std=c++17 -m64 
>> -I. -flto=4 -static-libgcc -static-libstdc++ -ljemalloc -o wccs_parser
>>
>> A. Compiling without the equivalent of "USE_PROCESS_IN_PAREN" takes 
>> ~14s and ~930MB (maximum resident).
>>
>> 1. I clone your repo into ~/spirit
>> 2. I add ~/spirit to very front of my CPLUS_INCLUDE_PATH . The full 
>> boost package (1.65.1) is very last. In fact these are the only two 
>> paths there.
>> 3. I compile again and it takes ~15s and ~925MB.
>>
>> I am quite sure it now uses your ~/spirit because now I get a pragma 
>> warning from the original boost (which I did not get before):
>>
>> /home/soren/spirit/include/boost/spirit/home/x3/nonterminal/rule.hpp:23:19: 
>> note: #pragma message: not(BOOST_SPIRIT_ATTR_XFORM_IN_RULE)
>>     #pragma message "not(BOOST_SPIRIT_ATTR_XFORM_IN_RULE)"
>>                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> /home/soren/spirit/include/boost/spirit/home/x3/nonterminal/rule.hpp:28:19: 
>> note: #pragma message: not(BOOST_SPIRIT_GET_RHS_CRTP)
>>     #pragma message "not(BOOST_SPIRIT_GET_RHS_CRTP)"
>
> Ah!  I forgot to add that you need to #define those 2 macros to
> avoid the long compile times.  Otherwise, the branch behaves
> just like the existing spirit.
>
>>
>> And I can still not compile with "USE_PROCESS_IN_PAREN".
>>
>> - Søren
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------ 
>>
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>
>>
>>
>> _______________________________________________
>> Spirit-general mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/spirit-general
>>
>
>
>
> ------------------------------------------------------------------------------ 
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Spirit-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/spirit-general

That certainly sped things up. But now it fails with template static 
assertion. This is the very end, but it occurs at least thrice in the 
output.

wccs_parser.cpp:226:59:   required from here
/home/soren/spirit/include/boost/spirit/home/x3/operator/detail/sequence.hpp:143:9: 
error: static assertion failed: Attribute does not have the expected size.

Line 226:59 is essentially root, in the parse call (last line in below):

         wccs_parser::AstAnyProcess root;
         auto iter = str.begin();
         auto end = str.end();
         bool r = parse(iter, end, wccs_parser::entry, root);

This also occurs a bit up:

.....

..... , boost::spirit::x3::forward_ast<wccs_parser::AstConstantProcess> >&)’
              rat_v.post(ok_parse,attr,attr_);



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/spirit-general