Hi,
I'm a relative newcomer to JFlex, but I did search the archives a bit
first.
Over in Lucene (http://lucene.apache.org/java) we have two JFlex files
that are very closely related named the StandardTokenizer and the
WikipediaTokenizer
(https://svn.apache.org/repos/asf/lucene/java/trunk/src/java/org/apache/lucene/analysis/standard/StandardTokenizerImpl.jflex
and
https://svn.apache.org/repos/asf/lucene/java/trunk/contrib/wikipedia/src/java/org/apache/lucene/wikipedia/analysis/WikipediaTokenizerImpl.jflex
if you care to see the source)
The WikipediaTokenizer was built by copying and pasting the Std. Tok.
into a new file and then extending it with new rules (but keeps in
tact all of Std. Toks rules, I believe). What we would like to be
able to do is somehow import the core definitions and rules from the
Std. Tokenizer into the WikipediaTokenizer to make maintenance
easier. Is there a way to do this in JFlex?
Thanks,
Grant
|
