Re: [Scriptonite-dev] questions about helper definitions

[Etienne M. G. <eg...@j-...>]

"Raif S. Naffah" wrote:
> 
> it may be a SableCC "finesse" that i'm not aware of, but i'd appreciate it
> if somebody can enlighten me on this:
> 
> 1. why the (Helpers) definitions for unicode_letter and unicode_digit are
> commented out?

Because:
(1) These complex regular expressions slow down the lexer generation time.  As I was
debugging the parser, I commented them out.  Quick and dirty;-)

(2) These definitions were taken from the "Java" language specification.  I am quite
unsure if they are appropriate for ECMAScript.  So, before you uncomment them, you should
doublecheck the specification.

I'd like Scriptonite to be 100% conform to the specification.  The Parser part already is
(but for the literal part).  Some digging remains to be done to gather all information
about lexer elements.

> 2. why do we have
>         (a) a Helper white_space = (sp | ht | ff | line_terminator)*; and
>         (b) a Token white_space =;

The lexer is not done yet.  The current definitions were simply "borrowed" from the Java
grammar.  Don't trust any definition.  What we know is that there should be a
"white_space" token, but we must find its regular expression.

> 3. the Helper white_space_no_lt = ; is left undefined. any specific reason?

See above.

> 4. not all UNICODE 3.0 characters are accounted for. is it by design?

See above.

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableCC:                 http://www.sable.mcgill.ca/sablecc/
----------------------------------------------------------------------