cpp-netlib-devel

[cpp-netlib-devel] URI: Spirit Compile Times

[Dean M. B. <mik...@gm...>]

Hi Everyone,

I recently remembered general advice about how to keep the compile
times down with Spirit.Qi and one of the general tips was to limit the
number of rules you have. The rationale was if you were writing a
parser and had a lot of rule assignments, you were essentially
invoking Proto's evaluation when assigning to a rule.

I see that the URI code has a lot of rules (>5) which can still be
reduced into a single rule. Jeroen/Glyn can you try and look at
reducing the number of rules and consolidate to just one rule just to
see whether it would help reduce the compile times?

Thanks in advance and I hope to hear what you guys think about trying
that out. I might do that on my own but getting time lately to work on
cpp-netlib ironically just got really hard.

-- 
Dean Michael Berris
deanberris.com

Re: [cpp-netlib-devel] URI: Spirit Compile Times

[Jeroen H. <vex...@gm...>]

Hi,

On 30 November 2010 17:38, Dean Michael Berris <mik...@gm...> wrote:
> Hi Everyone,
>
> I recently remembered general advice about how to keep the compile
> times down with Spirit.Qi and one of the general tips was to limit the
> number of rules you have. The rationale was if you were writing a
> parser and had a lot of rule assignments, you were essentially
> invoking Proto's evaluation when assigning to a rule.
>
> I see that the URI code has a lot of rules (>5) which can still be
> reduced into a single rule. Jeroen/Glyn can you try and look at
> reducing the number of rules and consolidate to just one rule just to
> see whether it would help reduce the compile times?
>
> Thanks in advance and I hope to hear what you guys think about trying
> that out. I might do that on my own but getting time lately to work on
> cpp-netlib ironically just got really hard.
>
> --
> Dean Michael Berris
> deanberris.com
>

There is a best practices documentation addendum here:
http://boost-spirit.com/home/articles/doc-addendum/best-practices/,
which states:

"Avoid complex rules. Rules with complex definitions hurt the compiler
badly. We’ve seen rules that are more than a hundred lines long and
take a couple of minutes to compile. On some compilers, experience
shows that the compile time is exponential in relation to the RHS
expression length. C++ compilers were not designed to handle such big
expressions and some just couldn’t cope (crashes). It is always best
to break complex rules into more manageable, easier to digest parts.
Doing so also makes the rules more readable."

and I've always understood is that multiple simple rules are
preferable to a single complex rule. However, if I find time I'll look
into combining some rules and testing whether it helps the
compile-time, it shouldn't be to hard.

Jeroen

Re: [cpp-netlib-devel] URI: Spirit Compile Times

[Dean M. B. <mik...@gm...>]

On Wed, Dec 1, 2010 at 3:51 AM, Jeroen Habraken <vex...@gm...> wrote:
>
> On 30 November 2010 17:38, Dean Michael Berris <mik...@gm...> wrote:
>>
>> I see that the URI code has a lot of rules (>5) which can still be
>> reduced into a single rule. Jeroen/Glyn can you try and look at
>> reducing the number of rules and consolidate to just one rule just to
>> see whether it would help reduce the compile times?
>>
>> Thanks in advance and I hope to hear what you guys think about trying
>> that out. I might do that on my own but getting time lately to work on
>> cpp-netlib ironically just got really hard.
>>
>
> There is a best practices documentation addendum here:
> http://boost-spirit.com/home/articles/doc-addendum/best-practices/,
> which states:
>
> "Avoid complex rules. Rules with complex definitions hurt the compiler
> badly. We’ve seen rules that are more than a hundred lines long and
> take a couple of minutes to compile. On some compilers, experience
> shows that the compile time is exponential in relation to the RHS
> expression length. C++ compilers were not designed to handle such big
> expressions and some just couldn’t cope (crashes). It is always best
> to break complex rules into more manageable, easier to digest parts.
> Doing so also makes the rules more readable."
>

Yes, that is true, but there was a mailing list discussion about the
cost of doing assignments to rules which basically invokes the Proto
evaluator. This kills the compiler because the Proto evaluation is a
full-on compiler in itself. Invoking it multiple times is a bigger
problem than having it deal with a sufficiently complex rule (in our
case it's really just a sequencing of optionals, which shouldn't
require too much mumbo-jumbo on the Proto side I imagine).

> and I've always understood is that multiple simple rules are
> preferable to a single complex rule. However, if I find time I'll look
> into combining some rules and testing whether it helps the
> compile-time, it shouldn't be to hard.
>

Yep they were -- especially in the Classic days IIRC. But with Qi and
the Proto stuff, the rule assignments get more expensive than the more
complex rules if I get my Proto correctly.

I'll look forward to your experiments and hopefully we can address
some of the compile time concerns as I go head-on into 0.9.

Thanks for looking into it Jeroen!

-- 
Dean Michael Berris
deanberris.com