flex: the fast lexical analyser

On Thu, May 28, 2009 at 03:26:28PM -0400, Martin Cote wrote:
> 
> Is there any other way to make flex to continue the scan
> process even after non-ASCII characters?
>

That's how it works by default (lex does bytes, not chars).
Try a lexer something like this:

	
	SKIPME  	[\x80-\xFF]
	%%
	{SKIPME}	/* ignore */

then feed it your UTF-8 source on stdin.

m

In a grammar where I allow user-defined function names, I tokenize
something that looks like it might be a function name and then use a helper
function to give me a code I can push back to bison.


ident           :?[a-zA-Z_][a-zA-Z_0-9]*

<INITIAL>{ident}  {
int k = lookup_keyword(yytext, yyleng);
if (!k) {
return INVALID_IDENT;
}
return k;
}

The function lookup_keyword connects to a gperf table of known functions
and falls back onto a user-defined function lookup.

I'm not entirely sure how I'd go about doing this for arbitrary characters,
though. It ends up being that your parser does the work of the lexer, or at
least that the two start to get very intimate.

Do you have any more constraints you can tell us about?

Aaron


On Tue, 26 May 2009 23:40:17 -0500, "J. L. Turriff" <jlt1@...>
wrote:
> I'm setting up flex definitions for a scripting language, and I've come 
> across an issue that is not addressed in the flex manual.  The language 
> definition allows the user to define the value of an optional delimiter
> token 
> at execution-time; the token has no default value.  How do I handle this
in
> 
> the flex definitions?
> 
> 	Here's an example of the usage:
> 
> 		command (endchar ?) more specifications...
> 
> 	The value of endchar has no default (its use is optional in /more 
> specifications/) but whatever value is supplied by the user (in this 
> case "?") must be recognized by the scanner.
> 
> Leslie
> 
>
------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT 
> is a gathering of tech-side developers & brand creativity professionals.
> Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, & 
> iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian
> 
> Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com 
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help

Hi,
 So, if I get it straight, your lexer should be something like this:

<INITIAL>{command}     { BEGIN(OPTIONAL_DELIMITER); /* do something for the
command */  }
<OPTIONAL_DELIMITER>{whitespace}  { /* do nothing */ }
<OPTIONAL_DELIMITER>.   {  // I'm using '.', assuming that the delimiter
must be a single char; you can replace it with any other rule
                  BEGIN(MORE_SPECIFICATIONS); // delimiter is optional, so
we enter "MORE_SPECIFICATIONS" state regardless on whether this is indeed a
delimiter char
                  if( !isUserDelimiter(yytext) ) yyless(0);  // if the char
is not delimiter, then don't consume it
                  else return TOK_DELIMITER;              // if it is a
delimiter, return the appropriate token to the parser.
}

<MORE_SPECIFICATIONS>{something}  { /*.....*/; BEGIN(INITIAL) }

   Does this help/did I understand correctly what you are trying to do?
      Virgil.

On Wed, May 27, 2009 at 7:40 AM, J. L. Turriff <jlt1@...> wrote:

>        I'm setting up flex definitions for a scripting language, and I've
> come
> across an issue that is not addressed in the flex manual.  The language
> definition allows the user to define the value of an optional delimiter
> token
> at execution-time; the token has no default value.  How do I handle this in
> the flex definitions?
>
>        Here's an example of the usage:
>
>                command (endchar ?) more specifications...
>
>        The value of endchar has no default (its use is optional in /more
> specifications/) but whatever value is supplied by the user (in this
> case "?") must be recognized by the scanner.
>
> Leslie
>
>
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals.
> Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
> Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help
>

If I understand correctly then before execution of the command you would
know the valid (optional) tokens/endchar like '?'. Based on this assumption,
the rule can be made like:

WORD    [[:alnum:]_-]+

{WORD}  {
            if(isValidOption(yytext))
              return SPECIFIER;
            else
              return SPECIFICATION;
        }

In parser, use non-terminals like:

command:  main optional_specifier specification { ... }
 ;

main: /* handle main function and validations here */
 ;

optional_specification:   /* empty */           { ... }
 |                        SPECIFIER             { ... }
 ;

specification: /* handle specifications here */
 ;

Regards,
Samkit Jain

On Wed, May 27, 2009 at 12:09 PM, Virgil Palanciuc <
virgil.palanciuc@...> wrote:

>   Hi,
>  So, if I get it straight, your lexer should be something like this:
>
> <INITIAL>{command}     { BEGIN(OPTIONAL_DELIMITER); /* do something for the
> command */  }
> <OPTIONAL_DELIMITER>{whitespace}  { /* do nothing */ }
> <OPTIONAL_DELIMITER>.   {  // I'm using '.', assuming that the delimiter
> must be a single char; you can replace it with any other rule
>                  BEGIN(MORE_SPECIFICATIONS); // delimiter is optional, so
> we enter "MORE_SPECIFICATIONS" state regardless on whether this is indeed a
> delimiter char
>                  if( !isUserDelimiter(yytext) ) yyless(0);  // if the char
> is not delimiter, then don't consume it
>                  else return TOK_DELIMITER;              // if it is a
> delimiter, return the appropriate token to the parser.
> }
>
> <MORE_SPECIFICATIONS>{something}  { /*.....*/; BEGIN(INITIAL) }
>
>   Does this help/did I understand correctly what you are trying to do?
>       Virgil.
>
> On Wed, May 27, 2009 at 7:40 AM, J. L. Turriff <jlt1@...>
> wrote:
>
> >        I'm setting up flex definitions for a scripting language, and I've
> > come
> > across an issue that is not addressed in the flex manual.  The language
> > definition allows the user to define the value of an optional delimiter
> > token
> > at execution-time; the token has no default value.  How do I handle this
> in
> > the flex definitions?
> >
> >        Here's an example of the usage:
> >
> >                command (endchar ?) more specifications...
> >
> >        The value of endchar has no default (its use is optional in /more
> > specifications/) but whatever value is supplied by the user (in this
> > case "?") must be recognized by the scanner.
> >
> > Leslie
> >
> >
> >
> ------------------------------------------------------------------------------
> > Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> > is a gathering of tech-side developers & brand creativity professionals.
> > Meet
> > the minds behind Google Creative Lab, Visual Complexity, Processing, &
> > iPhoneDevCamp as they present alongside digital heavyweights like
> Barbarian
> > Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
> > _______________________________________________
> > Flex-help mailing list
> > Flex-help@...
> > https://lists.sourceforge.net/lists/listinfo/flex-help
> >
>
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals.
> Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
> Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help
>

You don't need to link against flex; it includes everything you need in the
generated file. If you run the usage instructions exactly you'll get
something resembling the following:

marcel@... ~/code $ flex sample1.lex 
marcel@... ~/code $ gcc lex.yy.c -lfl
marcel@... ~/code $ ./a.out 
username
marcel

^C


On Mon, 11 May 2009 19:59:58 -0700 (PDT), "L. Milosevic"
<milosevic_86@...> wrote:
> Hi,
> 
> just got the last version of flex and compiled it.
> In order to try it, i downloaded one example from the web.
> 
> ===========sample1.lex================
> /* 
>  * Sample Scanner1: 
>  * Description: Replace the string "username" from standard input 
>  *              with the user's login name (e.g. lgao)
>  * Usage: (1) $ flex sample1.lex
>  *        (2) $ gcc lex.yy.c -lfl
>  *        (3) $ ./a.out
>  *            stdin> username
>  *          stdin> Ctrl-D
>  * Question: What is the purpose of '%{' and '%}'?
>  *           What else could be included in this section?
>  */
> 
> %{
> /* need this for the call to getlogin() below */
> #include <unistd.h>
> %}
> 
> %%
> username    printf("%s\n", getlogin());
> %%
> 
> main()
> {
>   yylex();
> }
> =================end of sample1.lex===============
> 
> I called flex and it generated lex.yy.c. 
> 
> Then i try to generate the binary:
> 
> -bash-3.2$ gcc -lfl -L../installdir/lib/ lex.yy.c 
> /tmp/ccWhvt7x.o: In function `main':
> lex.yy.c:(.text+0x1827): multiple definition of `main'
> ../installdir/lib//libfl.a(libmain.o):libmain.c:(.text+0x0): first
defined
> here
> /tmp/ccWhvt7x.o: In function `yylex':
> lex.yy.c:(.text+0x3b4): undefined reference to `yywrap'
> /tmp/ccWhvt7x.o: In function `input':
> lex.yy.c:(.text+0xda7): undefined reference to `yywrap'
> collect2: ld returned 1 exit status
> 
> 
> As you can see, i'm using the correct library path..
> -bash-3.2$ ls ../installdir/lib/
> libfl.a  libfl_pic.a
> 
> 
> Any idea of what is wrong?
> 
> Cheers, Milo.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>       Veja quais são os assuntos do momento no Yahoo! +Buscados
> http://br.maisbuscados.yahoo.com
> 
>
------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks
to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK
> i700
> Series Scanner you'll get full speed at 300 dpi even with all image 
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help

Now I've seen that If I append to my rule section

\n        { printf("newline\n"); }
.        { printf("strange symbol\n"); }

and types like input code:
"kobe °" [enter]
it loops with "newline" message!!

What's the story?



2009/5/5 Mark Redd <markredd85@...>

> Hello everybody,
> I've a problem with Flex (version 2.5.25)!
>
> I've written my lexical specifications in a foo.l file and it works fine.
>
> Now I want to recognize some extra symbols like '°' or '§' and so on, so i
> added at the end of the rule section this rule:
>
> .        { printf("bad symbol\n"); }
>
> Now If I run my lexical analizer and write standard string it works, but if
> the first input I type is '°', it loops with the printf("bad symbol") and it
> doesn't stop!!
> It is a bug or my mistake?
>
>

Is there something you're not telling us?


marcel@... ~/code $ flex -V
flex 2.5.35
marcel@... ~/code $ cat test.l 
%option main

%%
\n        { printf("newline\n"); }
.        { printf("strange symbol\n"); }
marcel@... ~/code $ flex --main -o test.c test.l && cc test.c
marcel@... ~/code $ ./a.out 
kobe °
strange symbol
strange symbol
strange symbol
strange symbol
strange symbol
strange symbol
strange symbol
newline
^C
marcel@... ~/code $ 


On Tue, 5 May 2009 08:59:15 +0200, Mark Redd <markredd85@...> wrote:
> Now I've seen that If I append to my rule section
> 
> \n        { printf("newline\n"); }
> .        { printf("strange symbol\n"); }
> 
> and types like input code:
> "kobe °" [enter]
> it loops with "newline" message!!
> 
> What's the story?
> 
> 
> 
> 2009/5/5 Mark Redd <markredd85@...>
> 
>> Hello everybody,
>> I've a problem with Flex (version 2.5.25)!
>>
>> I've written my lexical specifications in a foo.l file and it works
fine.
>>
>> Now I want to recognize some extra symbols like '°' or '§' and so on, so
>> i
>> added at the end of the rule section this rule:
>>
>> .        { printf("bad symbol\n"); }
>>
>> Now If I run my lexical analizer and write standard string it works, but
>> if
>> the first input I type is '°', it loops with the printf("bad symbol")
and
>> it
>> doesn't stop!!
>> It is a bug or my mistake?
>>
>>
>
------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks
to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK
> i700
> Series Scanner you'll get full speed at 300 dpi even with all image 
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help

Also worth mentioning is that flex works on BYTES not CHARACTERS. Assuming
UTF-8 here, '°' will be two bytes. If you want to match this in flex you'd
need a rule like:
\xc2\xb0 { ... }

0xc2b0 is the binary representation of '°' in UTF-8. That's why in the
example below there was one more extra "strange symbol" than expected.
While you didn't directly ask this question I assume it'll come up at some
point based on your description.

hth

On Tue, 05 May 2009 06:51:42 -0500, Marcel Laverdet <marcel@...>
wrote:
> Is there something you're not telling us?
> 
> 
> marcel@... ~/code $ flex -V
> flex 2.5.35
> marcel@... ~/code $ cat test.l 
> %option main
> 
> %%
> \n        { printf("newline\n"); }
> .        { printf("strange symbol\n"); }
> marcel@... ~/code $ flex --main -o test.c test.l && cc test.c
> marcel@... ~/code $ ./a.out 
> kobe °
> strange symbol
> strange symbol
> strange symbol
> strange symbol
> strange symbol
> strange symbol
> strange symbol
> newline
> ^C
> marcel@... ~/code $ 
> 
> 
> On Tue, 5 May 2009 08:59:15 +0200, Mark Redd <markredd85@...>
wrote:
>> Now I've seen that If I append to my rule section
>> 
>> \n        { printf("newline\n"); }
>> .        { printf("strange symbol\n"); }
>> 
>> and types like input code:
>> "kobe °" [enter]
>> it loops with "newline" message!!
>> 
>> What's the story?
>> 
>> 
>> 
>> 2009/5/5 Mark Redd <markredd85@...>
>> 
>>> Hello everybody,
>>> I've a problem with Flex (version 2.5.25)!
>>>
>>> I've written my lexical specifications in a foo.l file and it works
> fine.
>>>
>>> Now I want to recognize some extra symbols like '°' or '§' and so on,
so
>>> i
>>> added at the end of the rule section this rule:
>>>
>>> .        { printf("bad symbol\n"); }
>>>
>>> Now If I run my lexical analizer and write standard string it works,
but
>>> if
>>> the first input I type is '°', it loops with the printf("bad symbol")
> and
>>> it
>>> doesn't stop!!
>>> It is a bug or my mistake?
>>>
>>>
>>
>
------------------------------------------------------------------------------
>> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
>> production scanning environment may not be a perfect world - but thanks
> to
>> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK
>> i700
>> Series Scanner you'll get full speed at 300 dpi even with all image 
>> processing features enabled. http://p.sf.net/sfu/kodak-com
>> _______________________________________________
>> Flex-help mailing list
>> Flex-help@...
>> https://lists.sourceforge.net/lists/listinfo/flex-help
> 
>
------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks
to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK
> i700
> Series Scanner you'll get full speed at 300 dpi even with all image 
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help

2009/5/5 Marcel Laverdet <marcel@...>

>
>
> Also worth mentioning is that flex works on BYTES not CHARACTERS. Assuming
> UTF-8 here, '°' will be two bytes. If you want to match this in flex you'd
> need a rule like:
> \xc2\xb0 { ... }
>
> 0xc2b0 is the binary representation of '°' in UTF-8. That's why in the
> example below there was one more extra "strange symbol" than expected.



that's very interesting. there is a way to say "all symbol I type as to be
seen as a single character"? It's very important for me, because I've an
offset counter!

On Tuesday 05 May 2009 06:50:26 pm Mark Redd wrote:
> that's very interesting. there is a way to say "all symbol I type as to be
> seen as a single character"? It's very important for me, because I've an
> offset counter!

You could specify all byte sequences that define "a single character" in the 
encoding you are considering. If that doesn't cut it then you could try some 
other lexer such as re2c, which is able to handle all sorts of character 
encodings such as unicode's utf-8 and even utf-32 (which may make your life 
easier).


Rui Maciel

Yeah the only the way I see to do this while keeping an accurate column
count is by adding something like
--yycolumn; or yycolumn -= 2;
to your action depending on how long the sequence is. Depending on how many
special characters you plan to support, this may be reasonable or not. 

On Tue, 5 May 2009 19:03:52 +0100, Rui Maciel <rui.maciel@...> wrote:
> On Tuesday 05 May 2009 06:50:26 pm Mark Redd wrote:
>> that's very interesting. there is a way to say "all symbol I type as to
>> be
>> seen as a single character"? It's very important for me, because I've an
>> offset counter!
> 
> You could specify all byte sequences that define "a single character" in
> the 
> encoding you are considering. If that doesn't cut it then you could try
> some 
> other lexer such as re2c, which is able to handle all sorts of character 
> encodings such as unicode's utf-8 and even utf-32 (which may make your
life
> 
> easier).
> 
> 
> Rui Maciel
> 
>
------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks
to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK
> i700
> Series Scanner you'll get full speed at 300 dpi even with all image 
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help

That would keep track of byte positions, but you'd also want to track
character positions. Native support for UTF-8 would be the ideal path
forward; I have no feel for what amount of effort would be required,
though.

Aaron


On Tue, 05 May 2009 15:36:11 -0500, Marcel Laverdet <marcel@...>
wrote:
> Yeah the only the way I see to do this while keeping an accurate column
> count is by adding something like
> --yycolumn; or yycolumn -= 2;
> to your action depending on how long the sequence is. Depending on how
many
> special characters you plan to support, this may be reasonable or not. 
> 
> On Tue, 5 May 2009 19:03:52 +0100, Rui Maciel <rui.maciel@...>
wrote:
>> On Tuesday 05 May 2009 06:50:26 pm Mark Redd wrote:
>>> that's very interesting. there is a way to say "all symbol I type as to
>>> be
>>> seen as a single character"? It's very important for me, because I've
an
>>> offset counter!
>> 
>> You could specify all byte sequences that define "a single character" in
>> the 
>> encoding you are considering. If that doesn't cut it then you could try
>> some 
>> other lexer such as re2c, which is able to handle all sorts of character

>> encodings such as unicode's utf-8 and even utf-32 (which may make your
> life
>> 
>> easier).
>> 
>> 
>> Rui Maciel
>> 
>>
>
------------------------------------------------------------------------------
>> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
>> production scanning environment may not be a perfect world - but thanks
> to
>> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK
>> i700
>> Series Scanner you'll get full speed at 300 dpi even with all image 
>> processing features enabled. http://p.sf.net/sfu/kodak-com
>> _______________________________________________
>> Flex-help mailing list
>> Flex-help@...
>> https://lists.sourceforge.net/lists/listinfo/flex-help
> 
>
------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks
to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK
> i700
> Series Scanner you'll get full speed at 300 dpi even with all image 
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help

Hi Guilherme,

you will define a/some pattern/s that matches everything you want to  
ignore...
sounds weird but I hope you get the point.

Regards,
Stephan

Am 03.02.2009 um 16:32 schrieb Guilherme Martins:

> Hi everybody,
>
> This is a very basic question involving the usage of flex.
>
> When a program generated by flex find a text that do not match any  
> specified
> pattern, it puts the text in the standard output. Is it possible to  
> define
> an action to be executed when the scanner finds a pattern which does  
> not
> match?
>
> For example, I wrote a flex file to identify three types of token:  
> integers,
> floats and identifiers. When it finds a parenthesis, an equal sign  
> or other
> symbol which is not recognized by the lexer, the scanner puts it in  
> the
> standard output. My problem is that I need to specify an action to  
> handle
> unrecognized patterns, instead of simply putting them in stdout.
>
> Does someone know what I could do?
>
> Thanks,
> - Guilherme Martins -
>
>
>
> 	
>
> 	
> 		
> _______________________________________________________
> Yahoo! Mail - Sempre a melhor opção para você!
> Experimente já e veja as novidades.
> http://br.yahoo.com/mailbeta/tudonovo/
>
>
> ------------------------------------------------------------------------------
> Register Now & Save for Velocity, the Web Performance & Operations
> Conference from O'Reilly Media. Velocity features a full day of
> expert-led, hands-on workshops and two days of sessions from industry
> leaders in dedicated Performance & Operations tracks. Use code  
> vel09scf
> and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf_______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help

P.S.:
In most (f)lex How-Tos you will find a schema like:
	<your patterns here>
. (literal '.') to ignore anything you didn't define in <your patterns  
here>

Cheers

Am 04.05.2009 um 16:14 schrieb stephan müller:

> Hi Guilherme,
>
> you will define a/some pattern/s that matches everything you want to
> ignore...
> sounds weird but I hope you get the point.
>
> Regards,
> Stephan
>
> Am 03.02.2009 um 16:32 schrieb Guilherme Martins:
>
>> Hi everybody,
>>
>> This is a very basic question involving the usage of flex.
>>
>> When a program generated by flex find a text that do not match any
>> specified
>> pattern, it puts the text in the standard output. Is it possible to
>> define
>> an action to be executed when the scanner finds a pattern which does
>> not
>> match?
>>
>> For example, I wrote a flex file to identify three types of token:
>> integers,
>> floats and identifiers. When it finds a parenthesis, an equal sign
>> or other
>> symbol which is not recognized by the lexer, the scanner puts it in
>> the
>> standard output. My problem is that I need to specify an action to
>> handle
>> unrecognized patterns, instead of simply putting them in stdout.
>>
>> Does someone know what I could do?
>>
>> Thanks,
>> - Guilherme Martins -
>>
>>
>>
>> 	
>>
>> 	
>> 		
>> _______________________________________________________
>> Yahoo! Mail - Sempre a melhor opção para você!
>> Experimente já e veja as novidades.
>> http://br.yahoo.com/mailbeta/tudonovo/
>>
>>
>> ------------------------------------------------------------------------------
>> Register Now & Save for Velocity, the Web Performance & Operations
>> Conference from O'Reilly Media. Velocity features a full day of
>> expert-led, hands-on workshops and two days of sessions from industry
>> leaders in dedicated Performance & Operations tracks. Use code
>> vel09scf
>> and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf_______________________________________________
>> Flex-help mailing list
>> Flex-help@...
>> https://lists.sourceforge.net/lists/listinfo/flex-help
>
> ------------------------------------------------------------------------------
> Register Now & Save for Velocity, the Web Performance & Operations
> Conference from O'Reilly Media. Velocity features a full day of
> expert-led, hands-on workshops and two days of sessions from industry
> leaders in dedicated Performance & Operations tracks. Use code  
> vel09scf
> and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help

You should use

(?s:.)                        { ; }    /* do nothing */

So that even newlines won't be printed.

On Mon, May 4, 2009 at 9:07 PM, stephan müller <
stephanr.mueller@...> wrote:

> P.S.:
> In most (f)lex How-Tos you will find a schema like:
>        <your patterns here>
> . (literal '.') to ignore anything you didn't define in <your patterns
> here>
>
> Cheers
>
> Am 04.05.2009 um 16:14 schrieb stephan müller:
>
> > Hi Guilherme,
> >
> > you will define a/some pattern/s that matches everything you want to
> > ignore...
> > sounds weird but I hope you get the point.
> >
> > Regards,
> > Stephan
> >
> > Am 03.02.2009 um 16:32 schrieb Guilherme Martins:
> >
> >> Hi everybody,
> >>
> >> This is a very basic question involving the usage of flex.
> >>
> >> When a program generated by flex find a text that do not match any
> >> specified
> >> pattern, it puts the text in the standard output. Is it possible to
> >> define
> >> an action to be executed when the scanner finds a pattern which does
> >> not
> >> match?
> >>
> >> For example, I wrote a flex file to identify three types of token:
> >> integers,
> >> floats and identifiers. When it finds a parenthesis, an equal sign
> >> or other
> >> symbol which is not recognized by the lexer, the scanner puts it in
> >> the
> >> standard output. My problem is that I need to specify an action to
> >> handle
> >> unrecognized patterns, instead of simply putting them in stdout.
> >>
> >> Does someone know what I could do?
> >>
> >> Thanks,
> >> - Guilherme Martins -
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________________
> >> Yahoo! Mail - Sempre a melhor opção para você!
> >> Experimente já e veja as novidades.
> >> http://br.yahoo.com/mailbeta/tudonovo/
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Register Now & Save for Velocity, the Web Performance & Operations
> >> Conference from O'Reilly Media. Velocity features a full day of
> >> expert-led, hands-on workshops and two days of sessions from industry
> >> leaders in dedicated Performance & Operations tracks. Use code
> >> vel09scf
> >> and Save an extra 15% before 5/3.
> http://p.sf.net/sfu/velocityconf_______________________________________________
> >> Flex-help mailing list
> >> Flex-help@...
> >> https://lists.sourceforge.net/lists/listinfo/flex-help
> >
> >
> ------------------------------------------------------------------------------
> > Register Now & Save for Velocity, the Web Performance & Operations
> > Conference from O'Reilly Media. Velocity features a full day of
> > expert-led, hands-on workshops and two days of sessions from industry
> > leaders in dedicated Performance & Operations tracks. Use code
> > vel09scf
> > and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
> > _______________________________________________
> > Flex-help mailing list
> > Flex-help@...
> > https://lists.sourceforge.net/lists/listinfo/flex-help
>
>
> ------------------------------------------------------------------------------
> Register Now & Save for Velocity, the Web Performance & Operations
> Conference from O'Reilly Media. Velocity features a full day of
> expert-led, hands-on workshops and two days of sessions from industry
> leaders in dedicated Performance & Operations tracks. Use code vel09scf
> and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help
>

>In any tutorials I red there is the call yylex() in main function, so the
>scanner takes input from keyboard. Instead, I want to scan some strings in
>char[] type. How can I do?

You could always start by reading the man page that comes with flex,
in particular the section MULTIPLE INPUT BUFFERS.

Regards,
John Levine, johnl@..., Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, ex-Mayor
"More Wiener schnitzel, please", said Tom, revealingly.

You need to make another rule that will match a longer token. So like:

"int" ...
[a-z]+ ...

[a-z]+ will match more than "int" and thus the first rule will be
disregarded.

On Sat, 2 May 2009 19:00:24 +0530, Samkit Jain <samkit.feeds@...>
wrote:
> How can one match word boundaries in flex.
> 
> let say I have to match a word "int" and available sentences/words are as
> below. And I have to match only "int" i.e. the first word.
> 
> int
> integer
> integral
> intestine
> intense
> ...
>
------------------------------------------------------------------------------
> Register Now & Save for Velocity, the Web Performance & Operations 
> Conference from O'Reilly Media. Velocity features a full day of 
> expert-led, hands-on workshops and two days of sessions from industry 
> leaders in dedicated Performance & Operations tracks. Use code vel09scf 
> and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
> _______________________________________________
> Flex-help mailing list
> Flex-help@...
> https://lists.sourceforge.net/lists/listinfo/flex-help