Memory leak

[John Brown]

Hello All,

I doubt that this is your issue, but I am using win_flex 2.5.37, so I am reporting it here.

According to mpatrol, my scanner generated by win_flex allocates but does not
free memory as follows:

unfreed allocations: 4 (61561 bytes)
0x02731310 (4 bytes) {malloc:21:0} [yyalloc|lex.yy.c|1857]

0x02731318 (48 bytes) {malloc:22:0} [yyalloc|lex.yy.c|1857]

0x02732098 (45123 bytes) {malloc:6:0} [-|-|-] // unknown source

0x02790000 (16386 bytes) {malloc:23:0} [yyalloc|lex.yy.c|1857]

// lex.yy.c
/* 1855 */ void yyalloc (yy_size_t size )
/* 1856 */ {
/* 1857 */ return (void ) malloc( size );
/* 1858 */}

// Calls to yyalloc in lex.yy.c
1396: b = (YY_BUFFER_STATE) yyalloc(sizeof( struct yy_buffer_state ) );
1405: b->yy_ch_buf = (char ) yyalloc(b->yy_buf_size + 2 );
1556: (yy_buffer_stack) = (struct yy_buffer_state)yyalloc
1604: b = (YY_BUFFER_STATE) yyalloc(sizeof( struct yy_buffer_state ) );
1653: buf = (char ) yyalloc(n );

I will be calling my win_bison 2.7 parser repeatedly, which in turn calls the win_flex
scanner. I suppose that it is tolerable if these allocations are one time only no matter
how many times the scanner is called, but:

1) Are they one time only initialisation?

2) What can be done to free this memory?

3) If I generated a reentrant scanner (which has explicit init and destroy functions),
would it help?

For what it is worth, my scaner is a GLR scanner.

Regards,
John Brown.

[Alex Zhondin]

Hi John!
Thank you for your questions, I will try to help you.
Actually I didn't investigate possible memory leakage in my lexers/parsers. But I made a little investigation in generated code now. Below my answers:

1) Are they one time only initialisation?

I think so. As I can see all these allocations related to buffers (yy_create_buffer/yyensure_buffer_stack). If generated lexer is not reentrant, there is only yylex function used in client code (bison generated parser). There is first time intialization of the buffer stack.

if ( !(yy_init) )
    {
        (yy_init) = 1;
        // some code
        if ( ! YY_CURRENT_BUFFER ) {
            yyensure_buffer_stack ();
            YY_CURRENT_BUFFER_LVALUE =
                yy_create_buffer(yyin,YY_BUF_SIZE );
        }
    }

2) What can be done to free this memory?

There is yylex_destroy function implemented in lexer generated file. But it never called. Flex docs tell us use yylex_init/yylex_destroy functions before/after using reentrant lexer. So I think we should call yylex_destroy when lexer is not needed any more. You can try it and tell us if it helps.

3) If I generated a reentrant scanner (which has explicit init and destroy functions), would it help?

Yes, Flex docs clearly specify to use yylex_init/yylex_destroy functions (as you mention). I think these functions do the same things - initialize/release buffer stack.

[John Brown]

When I call yydestroy() in my bison parser, no memory leaks are detected. The comments in yydestroy() indicate that a non-reentrant lexer will be initialised again automatically the next time yylex() is called, which is what I want. This seems to solve the problem. Thanks.