flex-devel

[Flex-devel] yytext and similar in -P scanners

[Petr M. <pma...@re...>]

Hi list,

the editor `vile' (aka "vi like emacs") currently fails to build with
new flex, i.e. flex post 2.5.4a.  The problem is that in scanners
generated with -P, symbols yytext, yyleng, yylineno and others are not
available.  Documentation states that they should be:

> Within your scanner itself, you can still refer to the global
> variables and functions using either version of their name; but
> externally, they have the modified name.

I'm sending the patch that should fix this problem.  It does the same
thing old flex does, namely it introduces #defines for yy symbols.

Comments welcome.

Thanks,
PM

[Flex-devel] yytext and similar in -P scanners

[Petr M. <pma...@re...>]

Hi,

a bug has been opened against flex in Red Hat bugzilla:
  http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=239695

Quote:
> Previous flex releases parsed args such that "flex -is" was
> acceptable. This release requires the form "flex -i -s" to
> do the same thing.

Perhaps there were reasons behind this change, but grouping of options
seems to me a reasonable thing to support.  I'm sending the patch that
implements it this way: if short option doesn't allow an argument, but
there is an argument string, error is not reported, but the string is
parsed next time around as another short option.

Comments welcome.

Thanks,
PM

Re: [Flex-devel] yytext and similar in -P scanners

[Will E. <wl...@us...>]

Thanks for the patch and the report. There was no particular reason for
the change to not parsing multiple concurrent options.

On Friday, 11 May 2007, 12:05 pm EDT, Petr Machata <pma...@re...> wrote:

> Hi,
> 
> a bug has been opened against flex in Red Hat bugzilla:
>   http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=239695
> 
> Quote:
> > Previous flex releases parsed args such that "flex -is" was
> > acceptable. This release requires the form "flex -i -s" to
> > do the same thing.
> 
> Perhaps there were reasons behind this change, but grouping of options
> seems to me a reasonable thing to support.  I'm sending the patch that
> implements it this way: if short option doesn't allow an argument, but
> there is an argument string, error is not reported, but the string is
> parsed next time around as another short option.
> 
> Comments welcome.
> 
> Thanks,
> PM
> 

> diff -urp flex-2.5.33/scanopt.c flex-2.5.33-pm/scanopt.c
> --- flex-2.5.33/scanopt.c	2002-08-29 22:30:25.000000000 +0200
> +++ flex-2.5.33-pm/scanopt.c	2007-05-11 17:48:29.000000000 +0200
> @@ -789,12 +789,12 @@ int     scanopt (svoid, arg, optindex)
>  		}
>  
>  		optarg = pstart + 1;
> -		arglen = 0;
> -		while (optarg[arglen])
> -			arglen++;
> -
> -		if (arglen == 0)
> +		if (!*optarg) {
>  			optarg = NULL;
> +			arglen = 0;
> +		}
> +		else
> +			arglen = strlen (optarg);
>  	}
>  
>  	/* At this point, we have a long or short option matched at opt_offset into
> @@ -812,13 +812,16 @@ int     scanopt (svoid, arg, optindex)
>  
>  	/* case: no args allowed */
>  	if (auxp->flags & ARG_NONE) {
> -		if (optarg) {
> +		if (optarg && !is_short) {
>  			scanopt_err (s, opt_offset, is_short, errcode =
>  				     SCANOPT_ERR_ARG_NOT_ALLOWED);
>  			INC_INDEX (s, 1);
>  			return errcode;
>  		}
> -		INC_INDEX (s, 1);
> +		else if (!optarg)
> +			INC_INDEX (s, 1);
> +		else
> +			s->subscript++;
>  		return optp->r_val;
>  	}
>  

> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Flex-devel mailing list
> Fle...@li...
> https://lists.sourceforge.net/lists/listinfo/flex-devel


-- 
Will Estes
Flex Project Maintainer
http://flex.sourceforge.net/

Re: [Flex-devel] yytext and similar in -P scanners

[Will E. <wl...@us...>]

Thanks for the patch. We'll evaluate it for inclusion in the cvs tree.

As a point of clarification, though, there is no such thing as "GNU
flex", nor has there ever been. The flex program has never been a part
of the GNU project, although they did provide some hosting resources for
a long time, back in the day.

It's just "flex".

Thanks,


On Friday, 30 March 2007,15:10 +0200, Petr Machata wrote:

> Hi list,
> 
> the editor `vile' (aka "vi like emacs") currently fails to build with
> new flex, i.e. flex post 2.5.4a.  The problem is that in scanners
> generated with -P, symbols yytext, yyleng, yylineno and others are not
> available.  Documentation states that they should be:
> 
> > Within your scanner itself, you can still refer to the global
> > variables and functions using either version of their name; but
> > externally, they have the modified name.
> 
> I'm sending the patch that should fix this problem.  It does the same
> thing old flex does, namely it introduces #defines for yy symbols.
> 
> Comments welcome.
> 
> Thanks,
> PM
> 

> diff -urp flex-2.5.33/flex.skl flex-2.5.33-pm/flex.skl
> --- flex-2.5.33/flex.skl	2006-02-16 23:20:43.000000000 +0100
> +++ flex-2.5.33-pm/flex.skl	2007-03-30 14:04:42.000000000 +0200
> @@ -54,6 +54,32 @@ m4_changequote([[, ]])
>  %#   the generated scanner as a C-style comment. This is to aid those who
>  %#   edit the skeleton.
>  %#
> +
> +%not-for-header
> +%if-not-reentrant
> +m4_ifelse(M4_YY_PREFIX,yy,,
> +#define yy_create_buffer M4_YY_PREFIX[[_create_buffer]]
> +#define yy_delete_buffer M4_YY_PREFIX[[_delete_buffer]]
> +#define yy_flex_debug M4_YY_PREFIX[[_flex_debug]]
> +#define yy_init_buffer M4_YY_PREFIX[[_init_buffer]]
> +#define yy_flush_buffer M4_YY_PREFIX[[_flush_buffer]]
> +#define yy_load_buffer_state M4_YY_PREFIX[[_load_buffer_state]]
> +#define yy_switch_to_buffer M4_YY_PREFIX[[_switch_to_buffer]]
> +#define yyin M4_YY_PREFIX[[in]]
> +#define yyleng M4_YY_PREFIX[[leng]]
> +#define yylex M4_YY_PREFIX[[lex]]
> +#define yylineno M4_YY_PREFIX[[lineno]]
> +#define yyout M4_YY_PREFIX[[out]]
> +#define yyrestart M4_YY_PREFIX[[restart]]
> +#define yytext M4_YY_PREFIX[[text]]
> +#define yywrap M4_YY_PREFIX[[wrap]]
> +#define yyalloc M4_YY_PREFIX[[alloc]]
> +#define yyrealloc M4_YY_PREFIX[[realloc]]
> +#define yyfree M4_YY_PREFIX[[free]]
> +)
> +%endif
> +%ok-for-header
> +
>  #define FLEX_SCANNER
>  #define YY_FLEX_MAJOR_VERSION FLEX_MAJOR_VERSION
>  #define YY_FLEX_MINOR_VERSION FLEX_MINOR_VERSION
> diff -urp flex-2.5.33/skel.c flex-2.5.33-pm/skel.c
> --- flex-2.5.33/skel.c	2006-02-21 03:45:41.000000000 +0100
> +++ flex-2.5.33-pm/skel.c	2007-03-30 14:04:43.000000000 +0200
> @@ -59,6 +59,32 @@ const char *skel[] = {
>    "%#   the generated scanner as a C-style comment. This is to aid those who",
>    "%#   edit the skeleton.",
>    "%#",
> +  "",
> +  "%not-for-header",
> +  "%if-not-reentrant",
> +  "m4_ifelse(M4_YY_PREFIX,yy,,",
> +  "#define yy_create_buffer M4_YY_PREFIX[[_create_buffer]]",
> +  "#define yy_delete_buffer M4_YY_PREFIX[[_delete_buffer]]",
> +  "#define yy_flex_debug M4_YY_PREFIX[[_flex_debug]]",
> +  "#define yy_init_buffer M4_YY_PREFIX[[_init_buffer]]",
> +  "#define yy_flush_buffer M4_YY_PREFIX[[_flush_buffer]]",
> +  "#define yy_load_buffer_state M4_YY_PREFIX[[_load_buffer_state]]",
> +  "#define yy_switch_to_buffer M4_YY_PREFIX[[_switch_to_buffer]]",
> +  "#define yyin M4_YY_PREFIX[[in]]",
> +  "#define yyleng M4_YY_PREFIX[[leng]]",
> +  "#define yylex M4_YY_PREFIX[[lex]]",
> +  "#define yylineno M4_YY_PREFIX[[lineno]]",
> +  "#define yyout M4_YY_PREFIX[[out]]",
> +  "#define yyrestart M4_YY_PREFIX[[restart]]",
> +  "#define yytext M4_YY_PREFIX[[text]]",
> +  "#define yywrap M4_YY_PREFIX[[wrap]]",
> +  "#define yyalloc M4_YY_PREFIX[[alloc]]",
> +  "#define yyrealloc M4_YY_PREFIX[[realloc]]",
> +  "#define yyfree M4_YY_PREFIX[[free]]",
> +  ")",
> +  "%endif",
> +  "%ok-for-header",
> +  "",
>    "#define FLEX_SCANNER",
>    "#define YY_FLEX_MAJOR_VERSION 2",
>    "#define YY_FLEX_MINOR_VERSION 5",




> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Flex-devel mailing list
> Fle...@li...
> https://lists.sourceforge.net/lists/listinfo/flex-devel

Re: [Flex-devel] yytext and similar in -P scanners

[Petr M. <pma...@re...>]

Will Estes wrote:
> Thanks for the patch. We'll evaluate it for inclusion in the cvs tree.
>=20
> As a point of clarification, though, there is no such thing as "GNU
> flex", nor has there ever been. The flex program has never been a part
> of the GNU project, although they did provide some hosting resources fo=
r
> a long time, back in the day.

Well, I'm glad to know, so that I don't make such mistake in future.
However I didn't relate flex and GNU in any way in my mail.

>=20
> It's just "flex".
>=20
> Thanks,

Thanks,
PM

>=20
> On Friday, 30 March 2007,15:10 +0200, Petr Machata wrote:
>=20
>> Hi list,
>>
>> the editor `vile' (aka "vi like emacs") currently fails to build with
>> new flex, i.e. flex post 2.5.4a.  The problem is that in scanners
>> generated with -P, symbols yytext, yyleng, yylineno and others are not=

>> available.  Documentation states that they should be:
>>
>>> Within your scanner itself, you can still refer to the global
>>> variables and functions using either version of their name; but
>>> externally, they have the modified name.
>> I'm sending the patch that should fix this problem.  It does the same
>> thing old flex does, namely it introduces #defines for yy symbols.
>>
>> Comments welcome.
>>
>> Thanks,
>> PM

Re: [Flex-devel] yytext and similar in -P scanners

[Will E. <wl...@us...>]

On Friday, 30 March 2007,16:12 +0200, Petr Machata wrote:

> Well, I'm glad to know, so that I don't make such mistake in future.
> However I didn't relate flex and GNU in any way in my mail.

Ah, I read "new" and mentally picked up "GNU".

Thanks for your gracious reply.

Re: [Flex-devel] yytext and similar in -P scanners

[Petr M. <pma...@re...>]

Will Estes wrote:
> Thanks for the patch. We'll evaluate it for inclusion in the cvs tree.

There was a bug discovered in my earlier "yy" patch.  When in C++ mode,
it shouldn't emit the yy macros.  I'm sending cvs diff that does this.
(Originally created by Srinivas Aji)

Thanks,
PM

Re: [Flex-devel] yytext and similar in -P scanners

[Will E. <wl...@us...>]

Thanks much for the additional submission.

--Will


On Monday, 25 June 2007,  4:08 pm +0200, Petr Machata <pma...@re...> wrote:

> Will Estes wrote:
> > Thanks for the patch. We'll evaluate it for inclusion in the cvs tree.
> 
> There was a bug discovered in my earlier "yy" patch.  When in C++ mode,
> it shouldn't emit the yy macros.  I'm sending cvs diff that does this.
> (Originally created by Srinivas Aji)
> 
> Thanks,
> PM

> Index: flex.skl
> ===================================================================
> RCS file: /cvsroot/flex/flex/flex.skl,v
> retrieving revision 2.208
> diff -u -r2.208 flex.skl
> --- flex.skl	31 May 2007 06:21:57 -0000	2.208
> +++ flex.skl	25 Jun 2007 13:52:20 -0000
> @@ -56,6 +56,7 @@
>  %#
>  
>  %not-for-header
> +%if-c-only
>  %if-not-reentrant
>  m4_ifelse(M4_YY_PREFIX,yy,,
>  #define yy_create_buffer M4_YY_PREFIX[[_create_buffer]]
> @@ -78,6 +79,7 @@
>  #define yyfree M4_YY_PREFIX[[free]]
>  )
>  %endif
> +%endif
>  %ok-for-header
>  
>  #define FLEX_SCANNER




> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Flex-devel mailing list
> Fle...@li...
> https://lists.sourceforge.net/lists/listinfo/flex-devel


-- 
Will Estes
Flex Project Maintainer
http://flex.sourceforge.net/