[Flex-help] Generated scanner consumes too much time - help request

[Simon S. <sim...@gn...>]

Hi flex-users and developers,
 
I've just did a profiling (using callgrind) on the GnuCOBOL compiler
(parsing only) with 2,500,000 LOC and found out that the recent
scanner[1] consumes most of the time:

* 44% spent in yylex (16% of these are spent in GnuCOBOL internal
functions leaving 28% of the complete parser process in this single flex
generated file)

* 13% in yy_get_previous_state

* 5.6% spent in yy_get_next_buffer with 1.3% in fgets
 
Removing all the "noise" there is still over 25% spent in the scanner
and "get_previous_state" doesn't sound like it should be called that often.
 
I guess there are some bad practices used in scanner.l, but I don't know
what to look out for.
Can you please have a look at this and post suggestions or point to bad
practice found in there?
 
Is the amount of calls to get_previous_state normal?
 
scanner.c was generated with flex 2.5.37 - Do you expect a speedup with
a more recent release?
 
Thank you for taking the time to answer and maybe even for checking the
source,
Simon

[1] https://sourceforge.net/p/open-cobol/code/HEAD/tree/trunk/cobc/scanner.l