Great work! Any chance to get in touch with the CEDET folks
(http://cedet.sourceforge.net/)? It feels kinda weird to have two
different parsers run over the same file.
I didn't spend much time looking at the code, but could both parsers
maybe share stuff? The fontification backend could also be made
modular to access either `semantic' or `synfl' data.
Sharing the same grammar source would already be a great thing to
avoid having to maintain two grammars for each language.
In case you were going to ask -- I am not volunteering ;-)
Stephen J. Turnbull writes:
> That should get your attention!
> Returning to reason and modesty, this is a rather primitive module
> based on an LR parser engine and an SLR parser generator. The parser
> wants tuning and optimization, but is perfectly servicable for
> experimentation. The parser generator probably should be extended to
> a LALR parser generator for general use, but that should be reasonably
> straightforward (though a little time-consuming) for anybody with a
> copy of The Dragon Book to hand. (Aho, Sethi, and Ullman, _Compilers:
> I probably shouldn't play with it much for the next few weeks, so, now
> that it's basically working, here it is:
> cvs -d :pserver:cvs@... \
> checkout -d synfl XEmacs/packages/unsupported/stephen/synfl
> To use it, you should just byte-compile everything and load up.
> The Makefile is untested. It expects to find XEmacs.rules in the
> grandparent directory, as usual. "distribution" in package-info.in is
> set to "unsupported", so "make install" is not a good idea; I don't
> think it has a chance of working at all.
> There's a built-in trace/debug facility, so the package is of
> educational interest. You can watch the LR parser, and the SLR
> parser-generator, as they do their things. I thought it was pretty
> cool, but maybe that's just me.
> The token stream implementation used to catch the results of lexical
> analysis is probably OK. The "interface" used to mesh the token
> stream, the grammar, and the parser output is an abortion; cleaning
> that up is the next task, so my apologies in advance to anybody who
> spontaneously combusts from looking at it. It just didn't want to be
> abstracted yet, but I tried to force it....
> The font-lock capability looks to be about twice as fast as (uncached,
> nonlazy) traditional font-lock already, but all of that is due to not
> using complex regular expressions; the syntax analyzer is currently
> just along for the ride, wasting a small amount of time in the
> interest of making sure it's reasonably efficient. It is. The C
> grammar used so far is trivial, just a stream of tokens.
> Profile data for fontlock+cc-mode, a lock-c grammar, and a lex'n'lock
> implementation applied to src/emacs.c are in doc/profile_emacs.c_*.
> Tick counts are purely notional, as the XEmacs used for profiling is
> fully pessimized. (Sorry, Hrvoje. :-) I will say that since so much
> of the time for font-lock is spent in regexp matching (all in C), and
> so little is for synfl, I'll bet that the advantage for synfl
> _increases_ with optimization of XEmacs.
> It probably wouldn't be too difficult to arrange to call the parser
> recursively on a different grammar (for the same kinds of files that
> mmm-mode is used for). Haven't done that, but I have given a little
> thought to avoiding introducing code that would prevent it.
> The lex'n'lock strategy is of independent interest. What's giving the
> speed for both the synfl locker and the lex'n'locker is to lex the
> tokens and then figure out what the keywords are by looking up the
> tokens in a hash table---this can be done without the parser stuff,
> and it's at least 5 times as fast as traditional font-lock. I am not
> yet willing to conjecture whether either strategy will handle bogus
> syntax or re-font-lock on-the-fly and still maintain efficiency.
> Documentation, outside of the sources, is non-existent, of course.
> They are pretty extensively commented, cross-referenced to the Dragon
> Book, and the docstrings are fairly good, I hope. You know where to
> find me if you have questions. :-)
> Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
> University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
> Ask not how you can "do" free software business;
> ask what your business can "do for" free software.