[jflex-users] Assistance with matching

[<de....@io...>]

Hello.

I'm trying to break up a file into words, "$$" and "\". Specifically, a
"word" is any non-whitespace character. An input such as:

  "a b c$$ $$ \ e \\\"

.. Would yield the tokens:

  1. a
  2. b
  3. c
  4. $$
  5. $$
  6. \
  7. e
  8. \
  9. \
  10. \

I'm having problems coming up with a pattern or set of patterns that
will achieve this, however.

The obvious definition, such as:

Word    = \P{Whitespace}+
Space   = \p{Whitespace}+
Command = \p{Alpha}+
Slash   = \\
Dollars = "$$"

%%

<YYINITIAL> {
  { Space } { /* Ignore */ }

  { Slash } {
    throw new RuntimeException("Slash");
  }

  { Dollars } {
    throw new RuntimeException("Dollars");
  }

  { Word } {
    final TokenText.Builder b = TokenText.builder();
    b.position(this.position());
    b.name(this.yytext());
    return b.build();
  }
}

... Will obviously not work, because although " $$ " and " \ "
will be matched by the Slash and Dollars patterns, an input such as
"f$$" will be matched by the Word pattern, rather than yielding two
tokens "f" and "$$".

What is the simplest way to achieve this with jflex?

M