flex: the fast lexical analyser

Curious, it seems to work for me:

% cat t.fl
D           [0-9]
L           [a-zA-Z_]
H           [a-fA-F0-9]
E           [Ee][+-]?{D}+
O           [0-7]

%%

{L}({L}|{D})*              { printf("ID: '%s'\n", yytext); }
{D}+"."{D}*({E})?[fF]? { printf("float: '%s'\n", yytext); }
%%

% flex --version
flex 2.5.35

% flex t.fl

% gcc lex.yy.c -lfl

% echo "8.0f" | ./a.out
float: '8.0f'

%


I'm not sure what to try next..  :/

-Chris




On Wed, May 16, 2012 at 8:33 AM, Rodrigo Dominguez <roddomi@...:

>
> Hello,
>
> I am writing the lexical rules for a C-like language and I am having some
> issues with floating-point constant suffixes:
>
> The input to the lexer is "8.0f". I have two rules in the following order
> in my file:
>
> D           [0-9]
> L           [a-zA-Z_]
> H           [a-fA-F0-9]
> E           [Ee][+-]?{D}+
> O           [0-7]
>
> %%
>
> {L}({L}|{D})*              { return(IDENTIFIER); }
> {D}+"."{D}*({E})?[fF]? { return(FLOATCONSTANT); }
>
> For some reason I would like to understand, the input is being tokenized
> as "8.0" (FLOATCONSTANT) and "f" (IDENTIFIER) instead of "8.0f"
> (FLOATCONSTANT). I have reviewed the precedence rules and it still doesn't
> make sense to me.
>
> Any help would be appreciated.
>
> Rod
>
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help
>

This is weird. I am using flex 2.5.35 (cygwin). I found that if I change my
second rule to:

/* note the extra space between the exponent and the suffix */
{D}+"."{D}*({E})?" "[fF]? { return(FLOATCONSTANT); }

then flex will parse it right. Maybe an issue with cygwin's flex? I will
try native linux flex and will report back.

Thank you,

Rod

PS: Chris, I am sorry if you received this message multiple times. I
recently changed my email address and my previous messages were bouncing
back.

On Wed, May 16, 2012 at 4:12 PM, Chris verBurg <cheetomonster@...:

>
> Curious, it seems to work for me:
>
> % cat t.fl
> D           [0-9]
> L           [a-zA-Z_]
> H           [a-fA-F0-9]
> E           [Ee][+-]?{D}+
> O           [0-7]
>
> %%
>
> {L}({L}|{D})*              { printf("ID: '%s'\n", yytext); }
> {D}+"."{D}*({E})?[fF]? { printf("float: '%s'\n", yytext); }
> %%
>
> % flex --version
> flex 2.5.35
>
> % flex t.fl
>
> % gcc lex.yy.c -lfl
>
> % echo "8.0f" | ./a.out
> float: '8.0f'
>
> %
>
>
> I'm not sure what to try next..  :/
>
> -Chris
>
>
>
>
> On Wed, May 16, 2012 at 8:33 AM, Rodrigo Dominguez <roddomi@...:
>
>>
>> Hello,
>>
>> I am writing the lexical rules for a C-like language and I am having some
>> issues with floating-point constant suffixes:
>>
>> The input to the lexer is "8.0f". I have two rules in the following order
>> in my file:
>>
>> D           [0-9]
>> L           [a-zA-Z_]
>> H           [a-fA-F0-9]
>> E           [Ee][+-]?{D}+
>> O           [0-7]
>>
>> %%
>>
>> {L}({L}|{D})*              { return(IDENTIFIER); }
>> {D}+"."{D}*({E})?[fF]? { return(FLOATCONSTANT); }
>>
>> For some reason I would like to understand, the input is being tokenized
>> as "8.0" (FLOATCONSTANT) and "f" (IDENTIFIER) instead of "8.0f"
>> (FLOATCONSTANT). I have reviewed the precedence rules and it still doesn't
>> make sense to me.
>>
>> Any help would be appreciated.
>>
>> Rod
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Flex-help mailing list
>> Flex-help@...
>> https://lists.sourceforge.net/lists/listinfo/flex-help
>>
>
>

Are you sure there isn't a blank in your input in front of the f?  Try
a monospaced font in your editor.

On 17 May 2012 16:29, Rodrigo Dominguez <rodrigo.d.dominguez@...> wrote:
> This is weird. I am using flex 2.5.35 (cygwin). I found that if I change my
> second rule to:
>
> /* note the extra space between the exponent and the suffix */
> {D}+"."{D}*({E})?" "[fF]? { return(FLOATCONSTANT); }
>
> then flex will parse it right. Maybe an issue with cygwin's flex? I will
> try native linux flex and will report back.
>

I found the problem. It was in my implementation of YY_INPUT. It was
breaking the "8.0f" string into two tokens. Sorry for the noise. Thank you.

On Thu, May 17, 2012 at 11:15 AM, John P. Hartmann <jphartmann@...:

> Are you sure there isn't a blank in your input in front of the f?  Try
> a monospaced font in your editor.
>
> On 17 May 2012 16:29, Rodrigo Dominguez <rodrigo.d.dominguez@...>
> wrote:
> > This is weird. I am using flex 2.5.35 (cygwin). I found that if I change
> my
> > second rule to:
> >
> > /* note the extra space between the exponent and the suffix */
> > {D}+"."{D}*({E})?" "[fF]? { return(FLOATCONSTANT); }
> >
> > then flex will parse it right. Maybe an issue with cygwin's flex? I will
> > try native linux flex and will report back.
> >
>