SourceForge has been redesigned. Learn more.
Close

#22 Leaks in tcllib functions using Trf acceleration

closed-fixed
Trf (15)
5
2009-05-07
2009-05-06
Pat Thoyts
No

The following code shows a leak in the tclsh process when Trf is the accelerator function used by tcllib. This example is crc32 but the same leak occurs with the md5 package. To make use of this in tcllib we attach this transform to an opened NUL or /dev/null channel and push the data through the nul channel and read off the variables at the end.

# Show leak in Trf accelerator usage in tcllib. This affects md5 and crc32
package require crc32

proc Main {} {
puts [array get ::crc::accel]; flush stdout
set v [expr {int(rand() * 16)}]
while {1} {
set v [::crc::crc32 $v]
}
}
Main

Discussion

  • Andreas Kupries

    Andreas Kupries - 2009-05-07

    The script from the bug report, as proper file, plus memory trace

     
  • Andreas Kupries

    Andreas Kupries - 2009-05-07

    Tcl script processing 'memory trace on' output into a semi-histogram of which call sites have how many unfreed allocations

     
  • Andreas Kupries

    Andreas Kupries - 2009-05-07

    Note: Trf CVS head currently uses Tcl_Alloc, Tcl_Free, and Tcl_Realloc. These functions are apparently immune to -DTCL_MEM_DEBUG. It was necessary to replace them with ckalloc, ckfree, and ckrealloc to get the expected behaviour, i.e. memory trace output for Trf.

    This may be a bug in the core. Will file a report on that.
    Will also commit the Trf sources modified in this way to have the correct functions in the sources.

     
  • Andreas Kupries

    Andreas Kupries - 2009-05-07

    Running LEAK.tcl (attached) for 20 seconds, capturing the stdout (memory trace) in a file (200 MB), then running it through processlog (also attached) and manually cutting out the sites in Tcl and low call-counts gives the following places with an extraordinary number of un-freed allocations:

    90 {{trf/generic/dig_opt.c 394 20}}
    900 {{trf/generic/dig_opt.c 394 21}}
    5347 {{trf/generic/dig_opt.c 394 22}}
    6346 {{trf/generic/registry.c 2649 140}}

    These are
    o->writeDestination = strcpy (ckalloc (1+strlen (value)), value);
    in SetOption (of dig_opt.c), and
    trans = NEW_TRANSFORM;
    in AttachTransform (registry.c).

    Seems the -xxxx-variable information is not cleaned up, and we seem to lose the whole transformation structure for attached transforms.

     
  • Andreas Kupries

    Andreas Kupries - 2009-05-07

    Leak {registry.c 2649} is fixed by adding a "ckfree (trans)" at the end of TrfClose().

     
  • Andreas Kupries

    Andreas Kupries - 2009-05-07

    Ok, the dig_opt.c leak was trickier. The xxxDestination information is allocated in CreateOptions, deallocated in DeleteOptions, which are all called correctly. However, in between, in Create(De/En)coder the informaiton is transfered out of the option structure into the (en/de)coder structure, field 'destHandle'. And this field is not freed in Delete(De/En)coder. Doing so fixed this leak as well.

     
  • Andreas Kupries

    Andreas Kupries - 2009-05-07

    Fixes committed to CVS. Version bumped to 2.1.4.

     
  • Andreas Kupries

    Andreas Kupries - 2009-05-07
    • status: open --> closed-fixed