Re[2]: [CEDET-devel] semantic lexer for python

[Eric M. L. <er...@si...>]

>>> pon...@ne... (David Ponce) seems to think that:
>
>---------708bd7da1659c033708bd7da1659c033
>Content-Type: text/plain; charset=iso-8859-1
>Content-Transfer-Encoding: 8bit
>Content-Disposition: inline
>
>Hi Eric & Richard,
>
>Attached you will find the latest version of `semantic-flex'
>including Richard's newline enhancement plus the following:
>
>- newline are also correctly handled when `semantic-ignore-comments'
>  is disabled.
>
>- I moved BOL detection before processing `semantic-flex-extensions'
>  because BOL tokens must be always inserted before any other tokens
>  found on the same line.  
>  
>  Maybe a more general approach to keep the consistency of tokens
>  ordering would be to replace the last
>  
>  (nreverse ts)
>  
>  by
> 
>  (sort ts #'(lambda (t1 t2) (<= (cadr t1) (cadr t2))))
>
>  Or even better, to also merge here whitespace and comment tokens
>  too?

nreverse is going to be the fastest method of re-ordering the tokens.
If I remember correctly, Emacs uses quicksort, and quicksort is least
efficient on a fully-ordered list ( O(n^2) ).  In addition, Emacs is
forced to use nthcdr, which adds O(log(n)) to the mix (An extra scan
every time it divides.)  Thus, the grand total (If I did my analysis
correctly) is O(n^2 log(n)) for our case.  To our detriment, lexical
token lists are very long.

If this final reverse/sort is a small portion of the overall time
spent analyzing (certainly possible), then I don't have a problem
using something else though. ;)

>Anyway I think this new version is more consistent in the way it
>handle the various `semantic-flex-enable-...' options.
  [ ... ]

I think your resolution to his problem is very good.  When you are
comfortable with it, please check it in.

Thanks
Eric

-- 
          Eric Ludlam:                 za...@gn..., er...@si...
   Home: www.ultranet.com/~zappo            Siege: www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: www.gnu.org