[CEDET-devel] Re[1]: semantic-lex performance patch

[Eric M. L. <er...@si...>]

>>> David PONCE <dav...@wa...> seems to think that:
>Hi Eric,
>
>I currently work on some performance improvement of the LR parser and
>noticed that the lexer was slower than the parser itself!  After I
>carefully reviewed semantic-lex code I found that the
>`semantic-lex-push-token' macro produced code that evaluated the
>TOKEN expression twice :-(
>
>I let you imagine the performance impact on a such expression (from
>`semantic-lex-paren-or-list'):
>
>    (semantic-lex-push-token
>     (semantic-lex-token
>      'semantic-list (match-beginning 0)
>      (save-excursion
>	(semantic-lex-unterminated-syntax-protection 'semantic-list
>	  (forward-list 1)
>	  (point))
>	)))
>
>Following is a patch that fixes that.  I did some measurements with
>ELP, using the wisent-java lexers and got between 26 to 34% of lex
>time speedup :-)
>
>More interesting, is that with that fix, the Java full parser became
>much faster than the tag parser, as semantic-lex is only called one
>time to lex the entire buffer.
>
>For example, to parse a source file of 718 lines with some syntax
>errors, and a large source file form the Sun's JDK (JTable.java - 6099
>lines), without syntax errors, `semantic-parse-region' globally took
>30% less elapsed time!!!
>
>I guess you will agree to commit this patch for the upcoming beta2b
>;-)
>
>David
  [ ... ]

Indeed, I do agree with you.  Please check it in.

Are there any other such oversights?

What is it about calling into the lexer that takes so much extra
time?  ie, is calling `semantic-lex' on an empty buffer really that
time consuming?

Eric

-- 
          Eric Ludlam:                 za...@gn..., er...@si...
   Home: http://www.ludlam.net            Siege: www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: www.gnu.org