Re: [Cscope-devel] cscope performance: flex vs lex

[Darryl O. <da...@so...>]

"Mike Hopkirk(hops)" <ho...@sc...> wrote:

> > 				User+Sys (seconds)
> > lex (AT&T)			 40.25
> > flex -l (default)		247.68	(5.17X slower)
> > flex (no -l)			208.03  (6.15X slower)
> 
> interesting - conventional wisdom has it that flex is much faster than lex.
> perhaps its due the scanner description being still somewhat 
> lex optimised/oriented rather than flex directed

     It's probably not so much that it's lex-optimized, but that it's
using features that are horribly slow in flex.  I investigated the
slowness, yesterday, and here's what I found:

* The flex-generated code is spending at least 40% of the CPU time in
  yy_get_previous_state().

* To make the flex code run faster (using special flex options like
  -Cf), the lex code needs to get rid of REJECT (simple) and variable
  trailing context patterns (hard).  An example of a variable trailing
  context pattern is:

	\#[ \t]*endif/.*\n[ \t\n]*#[ \t]*if

  I'm not sure, but I suspect that we'd have to add a yacc layer on top
  of the lex parser to get around this.

> > [ Or, perhaps, SCO should also open-source lex? ;-) ]
> 
> perhaps there'll be fewer hurdles to doing so from here on in ....

     I'm hoping.  ;-)

--
	Darryl Okahata
	da...@so...

DISCLAIMER: this message is the author's personal opinion and does not
constitute the support, opinion, or policy of Agilent Technologies, or
of the little green men that have been following him all day.