[Lxr-developer] LXR transition news

[Malcolm B. <ma...@br...>]

Hi all,

I see from the subscription list that several people have moved over
from the old LXR list to this new list and I have to say I'm heartened :-)

The new setup is coming along nicely.  The CVS tree has been sent over
to Sourceforge, and hopefully the admins there will have it up within a
day or two.  I've made an interim release 0.7 of the current lxr code
base so that there is a visible place for people to get this new lxr.

We also have some more project developers/admins, currently Per Kristian
and Peder have joined the new project, bring with them much needed
expertise on the lxr code!

I thought I'd outline some of the things I want to be working on over
the next few months (other than the transition of course).

The first one of these is to improve the interface of lxr to the
exuberant ctags (ectags) program.  This is used to find the indentifiers
in the source code.  Currently there are several problems with the
interface:

1) lxr doesn't work properly with ectags 5.0, the latest release last time
I looked
2) Because C++ & Java are essentially supported by C.pm, the tagging is
wrong sometimes (e.g. C++ reserved words are indexed)
3) lxr can't make use of all the other languages that ectags supports easily

My plan for fixing this goes something like this:

First, create a new tagger class called ectags which is a generic
interface to ectags - ie it will correctly index any file that ectags
can parse.  Specifically, this would implement indexfile().
Create a "meta-language" class that derives from ectags and is driven by
data file for each language we wish to support.   This would implement
referencefile(). Specifically, the SimpleParse spec and reserved words
should be taken from a data file for each language.  Then basic support
for any language can be provided by the metalang class once some simple
configuration data has been provided.
If specific languages have more complex needs then of course new lang
subclasses can still be defined and used as before.
Finally, replace the hardwired pathname <-> lang subclass mapping in
Lang.pm with a data file, so that no code has to be changed either to
decide which files are what type (ie recognise .c++ as C++ or .j as
Java) or to change which class is used to index which type.

So then the adding of basic support for languages supported by ectags
will consist of writing a few regexps in a data file (which if we're
really cunning we can extract from ectags itself - serious bonus points
for this!).

So that's one part.  What other things do I want to see improved?

1) The mysql db backend.  At the moment the indexes and queries on mysql
are not optimal when generating the browsing output.  Expect patches
soon :-)
2) I have heard reports that the postgresql backend is broken - this
needs fixing
3) I've implemented locally the ability to suppress reference display
when doing an indent query - this makes it *much* faster for the common
case of looking for the definition of something, since generating the
references for a common identifier can take 90+seconds on my database.
This may be worth putting in.
4) On a more bluesky note, I'd like to improve the browsing interface
generally by making better use of CSS and HTML 4.0 to support more
configurable fontification and perhaps mouse/sidebar interaction.
Integrating this with mozilla's XUL/RDF support could result in a killer
source browser.
5) And finally, I want to try to understand the demonic perl that
generates the templates for browsing :-)

So I guess these are my initial thoughts on directions to go in and what
I'm likely to be working on in the coming months, and some of the
directions I'd like to take the project.  If anyone else has things
they'd like to see or hack on, then it would be good to hear about it on
this list.

Cheers,

Malcolm