Re: [jflex-users] ambiguities in JFLex syntax grammar from the manual

[Steve R. <sa...@od...>]

Hi Romildo,

You can see the latest version (for as yet unreleased JFlex version 1.5) 
of the JFlex scanner spec here:

http://jflex.svn.sourceforge.net/viewvc/jflex/trunk/jflex/src/main/jflex/LexScan.flex?revision=586&view=markup

I recall that ViewVC (the Subversion & CVS repository browser used by 
SourceForge) switched to Pygments recently for its syntax highlighting. 
  It would be very nice if the above link resulted in a 
syntax-highlighted display :).

The latest version of the CUP parser for JFlex specs is here:

http://jflex.svn.sourceforge.net/viewvc/jflex/trunk/jflex/src/main/cup/LexParse.cup?revision=585&view=markup

José Romildo Malaquias wrote:
 > 1. The token '|' used as a regex operator for union, and as an action
 >    makes the grammar for the syntax of lexical rules (session 4.3.1 of
 >    the manual) ambiguous.

 From the above-linked grammar spec (LexScan.flex):

   460 <REGEXP> {
   ...
   464   {WSPNL}*"|"{WSP}*$      { if (macroDefinition) {
   ...
   477   {WSPNL}*"|"  { return symbol(BAR); }
   ...

A '|'-action (line #464) must be followed by optional whitespace and 
EOL; other uses are interpreted as unions.

In the manual at the end of section 4.3.2, the interpretation and 
utility of the '|'-action is spelled out more fully.

 > 2. In macro definitions, how does one know where one regular
 >    expression finishes?

Within the macro section of a spec, newlines terminate regular 
expressions - from the above-linked LexScan.flex:

   500 {NL} { if (macroDefinition) { yybegin(MACROS); }
              return symbol(REGEXPEND); }

 > 3. Can a JFlex comment appear anywhere a space is allowed?

Not everywhere; for your specific questions, though:

 > For instance, are the following allowed?
 >
 > %% // end of section 1
 >
 > %{ // the following will be inserted into the generated class

Yes, both of the above are allowed.  After '%%' and '%{' all text is 
ignored until end of line - no comment syntax is required.  So these are 
allowed too:

    %% end of section 1

    %{ the following will be inserted into the generated class

 > macro1 /* this is a macro */ = /* definition */ regex

Yes, both of the above comments (before and after the '=') are allowed. 
  See line 401 (before the '=') and line 502 (after the '=') in 
LexScan.flex.

 > < /*comment1*/ YYINITIAL, /*comment2*/ STR > re { action1 }

Comments in lexical state lists are not allowed.  See the <STATES> 
section in LexScan.flex, starting at line 448.

 > (a /*comment3*/ b | c) d { action2 }
 >
 > regex1 / /*comment3*/ regex2 { action3 }

Yes, these are both allowed.  See line 502 in LexScan.flex.

Steve