#195 Heap corruption for uncommon literals (SIGSEGV in cobc_free_mem)

[Simon Sobisch]

Got a core dump in running the testsuite for "literal too long" (current svn version).

Output when running directly:

prog.cob: 9: Error: Invalid literal: '                                   ...'
prog.cob: 9: Error: Literal length exceeds 8191 characters
Segmentation Fault (core dumped)

The following backtrace was produced by:

• recompiled after ./configure --enable-debug
• echo backtrace > backtrace.cmd
• gdb /tmp/gnu-cobol-2.0/cobc/.libs/cobc /tmp/gnu-cobol-2.0/tests/testsuite.dir/129/core -batch -x backtrace.cmd

Core was generated by `/tmp/gnu-cobol-2.0/cobc/.libs/cobc -fsyntax-only -debug -Wall -std=d'.
Program terminated with signal 11, Segmentation fault.
[New process 82904    ]
#0  0xfffffd7fff1f1418 in realfree () from /lib/64/libc.so.1
(gdb) backtrace
#0  0xfffffd7fff1f1418 in realfree () from /lib/64/libc.so.1
#1  0xfffffd7fff1f1a5c in _free_unlocked () from /lib/64/libc.so.1
#2  0xfffffd7fff1f19b9 in free () from /lib/64/libc.so.1
#3  0x0000000000415e1c in cobc_free (mptr=0x4c64f0) at cobc.c:666
#4  0x0000000000415a49 in cobc_free_mem () at cobc.c:498
#5  0x00000000004176da in cobc_clean_up (status=1) at cobc.c:1454
#6  0x00000000004177fe in cobc_sig_handler (sig=11) at cobc.c:1516
#7  0xfffffd7fff252616 in __sighndlr () from /lib/64/libc.so.1
#8  0xfffffd7fff245de6 in call_user_handler () from /lib/64/libc.so.1
#9  0xfffffd7fff246007 in sigacthandler () from /lib/64/libc.so.1
#10 0xffffffffffffffff in ?? ()
#11 0x000000000000000b in ?? ()
#12 0x0000000000000000 in ?? ()

[Simon Sobisch]

Analysis:

the error occurs in

::c
    for (reps = cobc_mainmem_base; reps; ) {
        repsl = reps;
        reps = reps->next;
        cobc_free (repsl);   // error occurs here
    }
    cobc_mainmem_base = NULL;

scanner.l does the following during initialisation:

::c
    if (!plexbuff) {
        plexbuff = cobc_malloc ((size_t)COB_MINI_BUFF);
        plexsize = COB_MINI_BUFF;
    }

and the following when parsing the literals:

::c
        if (!literal_error && i >= plexsize) {
            plexsize *= 2;
            if (unlikely (plexsize > cb_lit_length)) {
                plexsize = cb_lit_length;
            }
            plexbuff = cobc_realloc (plexbuff, plexsize);
        }

cobc. (cobc_realloc) is

::c
void *
cobc_realloc (void *prevptr, const size_t size)
{
    void    *mptr;

    mptr = realloc (prevptr, size);
    if (unlikely(!mptr)) {
        cobc_abort_pr (_("Cannot reallocate %d bytes of memory - Aborting"),
                (int)size);
        cobc_abort_terminate ();
    }
    return mptr;
}

This works fine in general but does not work when the pointer was allocated via cobc_malloc as this stores all it's allocation in a linked list cobc_mainmem_base which is defined in cobc.h as:

::c
/* Basic memory structure */
struct cobc_mem_struct {
    struct  cobc_mem_struct *next;          /* next pointer */
    void            *memptr;
    size_t          memlen;
};

ToDo:

• add the list to the function signature
• if the parameter is not NULL find the original entry and update it (both *memptr and memlen, plus the referencing item's *next
• Check for possible realloc-problems in other places (including libcob as this could be the reason for SIGSEGV seen in [patches:#23])

[Simon Sobisch]

The above is false information (which comes from only glancing on the code): cobc_malloc and cobc_realloc belong together and have nothing to do with cobc_mainmem_base, neither has plexbuff.

But the direction is likely correct, at least I assume some storage that was allocated with cobc_main_malloc was not reallocated with cobc_main_realloc / or not passed to cobc_main_free (which all work on cobc_mainmem_base).
Further checks needed...

[Simon Sobisch]

Further checks done:

• only cobc_main_malloc does the internal allocation
• all pointers in cobc_mem_struct are correct when calling cobc_free
• the free where the SIGSEGV occurs is a part of cb_extension_list where it's very unlikely that the memory got a call to free in other places
• most important: the SIGSEGV does only occur when you run this compilation with -fsyntax-only (which is done by the testsuite), without this flag (= more allocations) everything is fine !?!

Attached is a debugging version of cobc.[ch] leading to the output of testsuite.log which is attached, too, both with and without-fsyntax-only.

I won't do any further checks on this item for now and leave further checks and the fix to someone else, otherwise this bug stays here as a known bug...

[Simon Sobisch]

I've tried to run valgrind but it does not build on this machine (neither latest release not the solaris-port).

So this is the output from a different machine...

From valgrind --tool=exp-sgcheck --read-var-info=yes

==1262== exp-sgcheck, a stack and global array overrun detector
==1262== NOTE: This is an Experimental-Class Valgrind Tool
==1262== Copyright (C) 2003-2013, and GNU GPL'd, by OpenWorks Ltd et al.
==1262== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==1262== Command: /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc -fsyntax-only -debug -Wall -std=default prog.cob
==1262== 
prog.cob: 9: Error: Invalid literal: '                                   ...'
prog.cob: 9: Error: Literal length exceeds 8191 characters
==1262== 
==1262== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 9 from 9)

YAY, no stack problems found.

From valgrind --tool=memcheck --read-var-info=yes --track-origins=yes --leak-check=full

==1260== Memcheck, a memory error detector
==1260== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==1260== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==1260== Command: /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc -fsyntax-only -debug -Wall -std=default prog.cob
==1260== 
==1260== Invalid write of size 1
==1260==    at 0x8075C4F: yylex (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x805B7A8: yyparse (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x804EDCA: process_translate (cobc.c:3428)
==1260==    by 0x8050B8A: main (cobc.c:4636)
==1260==  Address 0x43903af is 0 bytes after a block of size 8,191 alloc'd
==1260==    at 0x4024D29: realloc (vg_replace_malloc.c:687)
==1260==    by 0x8049CC3: cobc_realloc (cobc.c:693)
==1260==    by 0x8073AC6: yylex (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x805B7A8: yyparse (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x804EDCA: process_translate (cobc.c:3428)
==1260==    by 0x8050B8A: main (cobc.c:4636)
==1260== 
==1260== Invalid write of size 1
==1260==    at 0x807627D: yylex (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x805B7A8: yyparse (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x804EDCA: process_translate (cobc.c:3428)
==1260==    by 0x8050B8A: main (cobc.c:4636)
==1260==  Address 0x43923e0 is 1 bytes after a block of size 8,191 alloc'd
==1260==    at 0x4024D29: realloc (vg_replace_malloc.c:687)
==1260==    by 0x8049CC3: cobc_realloc (cobc.c:693)
==1260==    by 0x8073AC6: yylex (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x805B7A8: yyparse (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x804EDCA: process_translate (cobc.c:3428)
==1260==    by 0x8050B8A: main (cobc.c:4636)
==1260== 
==1260== Invalid read of size 1
==1260==    at 0x4026523: strlen (mc_replace_strmem.c:404)
==1260==    by 0x8071C0E: error_literal (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x80762CE: yylex (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x805B7A8: yyparse (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x804EDCA: process_translate (cobc.c:3428)
==1260==    by 0x8050B8A: main (cobc.c:4636)
==1260==  Address 0x43923df is 0 bytes after a block of size 8,191 alloc'd
==1260==    at 0x4024D29: realloc (vg_replace_malloc.c:687)
==1260==    by 0x8049CC3: cobc_realloc (cobc.c:693)
==1260==    by 0x8073AC6: yylex (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x805B7A8: yyparse (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x804EDCA: process_translate (cobc.c:3428)
==1260==    by 0x8050B8A: main (cobc.c:4636)
==1260== 
prog.cob: 9: Error: Invalid literal: '                                   ...'
prog.cob: 9: Error: Literal length exceeds 8191 characters
==1260== Invalid write of size 1
==1260==    at 0x8075C14: yylex (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x805B7A8: yyparse (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x804EDCA: process_translate (cobc.c:3428)
==1260==    by 0x8050B8A: main (cobc.c:4636)
==1260==  Address 0x43923df is 0 bytes after a block of size 8,191 alloc'd
==1260==    at 0x4024D29: realloc (vg_replace_malloc.c:687)
==1260==    by 0x8049CC3: cobc_realloc (cobc.c:693)
==1260==    by 0x8073AC6: yylex (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x805B7A8: yyparse (in /tmp/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc)
==1260==    by 0x804EDCA: process_translate (cobc.c:3428)
==1260==    by 0x8050B8A: main (cobc.c:4636)
==1260== 
==1260== 
==1260== HEAP SUMMARY:
==1260==     in use at exit: 0 bytes in 0 blocks
==1260==   total heap usage: 286 allocs, 286 frees, 192,909 bytes allocated
==1260== 
==1260== All heap blocks were freed -- no leaks are possible
==1260== 
==1260== For counts of detected and suppressed errors, rerun with: -v
==1260== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 11 from 7)

I'll have to check this later.

[Simon Sobisch]

The bug is fixed with [r753] and [r754]. It occured if literal size > pplexsize (256/512/...) or > literal_length.

==16169== Memcheck, a memory error detector
==16169== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==16169== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==16169== Command: /reserve/oc_dev/gnu-cobol-2.0_totallyclean/cobc/.libs/cobc -fsyntax-only -debug -Wall -std=default prog.cob
==16169==
cobc: prog.cob: No such file or directory
==16169==
==16169== HEAP SUMMARY:
==16169==     in use at exit: 0 bytes in 0 blocks
==16169==   total heap usage: 37 allocs, 37 frees, 20,865 bytes allocated
==16169==
==16169== All heap blocks were freed -- no leaks are possible
==16169==
==16169== For counts of detected and suppressed errors, rerun with: -v
==16169== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 11 from 7)

[Simon Sobisch]

... the test case is broken again...

[Simon Sobisch]

Fixed with [r1036].