Re: [Flex-help] Switching between Buffers in flex

[Daniel R. <dan...@go...>]

I found a solution to my problem here:
http://www.cs.vassar.edu/cgi-bin/info2www?%28flex%29How+do+I+track+the+byte+offset+for+lseek%28%29%3F

This solution makes use of YY_USER_ACTION and yyleng and works as long  
as yymore, yyless, unput and input are not used.

Cheers,
Daniel

On 17 Sep 2010, at 14:10, Daniel Rolls wrote:

> Hi,
>
> Thanks for the reply. I tried this only to discover that checking  
> where the stream points is a stupid idea since new streams can point  
> to the same place in memory as old streams. The original library  
> calls in our application code only specify this pointer to identify  
> a stream. This means my only hope is on each call to restart the  
> parser (presumably with yyrestart as you suggested) and to reset the  
> pointer in the stream to the position after reading the last token  
> but before caching and new tokens. Ie:
>
> void doScan( FILE *stream) {
>  yyrestart( stream);
>  yyparse();
>  reset_stream_pointer( stream);
> }
>
> I don't know what reset_stream_pointer() would look like. Is it  
> possible to do this? Otherwise I think our code is just broken and  
> not fixable without rewriting all our applications.
>
> Daniel
>
> On 17 Sep 2010, at 04:16, Chris verBurg wrote:
>
>> Hey Daniel,
>>
>> Unless you're really using buffers (yy_new_buffer, etc), I don't  
>> think you want YY_FLUSH_BUFFER.  Instead, I think you want yyrestart:
>>
>> void doScan( FILE *stream)
>> {
>>  static int lastFile;
>>  if ((int)stream != lastFile) {
>>    lastFile = stream;
>>    yyrestart(stream);
>>  }
>>  yyparse();
>> }
>>
>> That's my guess at least.  :)
>>
>> -Chris
>>
>>
>> On Thu, Sep 16, 2010 at 9:32 AM, Daniel Rolls <dan...@go... 
>> > wrote:
>> Hi,
>>
>> I have a non-reeentrant flex/bison parser that reads simple data
>> files. This parser can be accessed though simple library calls with
>> names like readInt, readChar, etc. where the argument to each call is
>> pointer of type *FILE. Each function sets yyin to the appropriate
>> stream and sets a start token like PARSE_INT, PARSE_CHAR etc. We have
>> successfully used this parser for years to parse multiple data files
>> in single programs. Only recently did I discover that if I did not
>> read a file to the end before parsing a new file flex can return data
>> from the wrong file and bison will return the wrong data to me. Both
>> the flex and bison manual discuss this issue. I tried to fix it by
>> calling YY_FLUSH_BUFFER every time the stream pointer changes. For an
>> initial test I have done it in a very crude way with the following  
>> code:
>>
>> void doScan( FILE *stream)
>> {
>>  static int lastFile;
>>  if ((int)stream != lastFile) {
>>    lastFile = stream;
>>    YY_FLUSH_BUFFER;
>>  }
>>  yyparse();
>> }
>>
>> Obviously permanent solution would need to track the files properly
>> and have a FILE* type for stream but when not overlapping reads
>> between files I would have expected this to fix my problem. Sadly, it
>> results in a segmentation fault from the following calls:
>>  yyparse -> yylex() -> yy_get_next_buffer() ->fread() ->
>> _IO_sgetn_internal()
>>
>> If I remove YY_FLUSH_BUFFER the crash disappears  but of course my
>> original problem returns. Am I using YY_FLUSH_BUFFER incorrectly?
>>
>> Many thanks,
>> Daniel
>>
>>
>> ------------------------------------------------------------------------------
>> Start uncovering the many advantages of virtual appliances
>> and start using them to simplify application deployment and
>> accelerate your shift to cloud computing.
>> http://p.sf.net/sfu/novell-sfdev2dev
>> _______________________________________________
>> Flex-help mailing list
>> Fle...@li...
>> https://lists.sourceforge.net/lists/listinfo/flex-help
>>
>