Re: [Flex-devel] Suggestions for improvement

[Joe K. <kr...@ni...>]

Aaron Stone wrote:
> 
> On Sep 29, 2008, at 11:31 AM, Joe Krahn wrote:
...
>> When the option is used to generate a public header, the scanner source
>> file should #include that header rather than duplicating the contents in
>> the source file. Otherwise, user code that include that header (directly
>> or indirectly) gets duplicate declarations. At the very least, generated
>> source should pre-define the header's include guard macro.
> 
> That's slightly trickier, but I don't think it would be too hard to get 
> right. I'd actually have _two_ macros, one for the header itself, and 
> another for the generated contents of the header. We cannot guarantee 
> that someone won't add more material to the header in some fashion out 
> of our control, and we want to avoid breaking such things as much as 
> possible.
> 
> Something like:
> 
> #ifndef FLEX_HEADER_H
> #define FLEX_HEADER_H
> 
> #ifndef FLEX_HEADER_GUTS
> #define FLEX_HEADER_GUTS
> 
> ... flex generated stuff...
> 
> #endif
> 
> ... maybe the user puts stuff here?...
> 
> #endif
All that is needed is for the written header to include standard guard 
macros, where the contained HEADER_GUTS part above is a different 
include file output by Flex, for example, a user header can look like 
this, where F;ex's output is named "flex_public.h":
scanner.h:
#ifndef MY_SCANNER_H
#define MY_SCANNER_H
#include "flex_public.h"
...

The flex source code can have the public header content embedded, 
enclosed in the same guard macros written to flex_public.h. The scanner 
source could optionally #include flex_public.h, but having the embedded 
header parts exactly match the written header, including guard macros, 
will solve the problem of re-reading the header content.

> 
>> Flex has functions yyset_lval() and yyset_lloc() so that the scanner
>> globals can be set in reentrant mode, without having to pass them as
>> arguments on every call to yylex(). Unfortunately, those functions are
>> generated ONLY with bison-bridge options, which also forces them to be
>> yylex arguments. My suggestion is to always generate the lval and lloc
>> set/get functions in reentrant mode, unless options noyyset_lval, etc.,
>> are given.
> 
> Sounds reasonable.
> 
>>
>> The main reason Flex has memory allocation wrappers seems to be to avoid
>> errors for older standards that use (char*) for memory pointers. Why not
>> include a flag to directly use malloc, etc.? Or maybe an option to
>> define them as inline, to get the same effect?
> 
> The wrappers also allow the developer to have flex use the memory 
> management system of the application. For example, it is possible to use 
> a pools, slabs, and garbage collection by providing a yyalloc and 
> yyrealloc, and defining yyfree to be a noop. You can achieve a 
> direct-to-malloc conversion with noyyalloc, et al, and #define yyalloc 
> malloc.
> 
The problem is that the reentrant scanner includes yyscanner references 
in the allocation prototypes. You can get around it with macros, but it 
takes more work. You need macros at the top to hide flex's prototypes, 
which will conflict with C lib prototypes:

%top{
#define yyalloc yyalloc__
}
...
%{
#undef yyalloc__
#define yyalloc(size,scanner) malloc(size)
%}

Besides,