#4168 unaligned malloc() -> Tcl panic

obsolete: 8.5.5
open
Jeffrey Hobbs
5
2008-10-21
2008-10-21
Don Porter
No

Testing memory hygiene in a program that
embeds Tcl on an x86_64 system with
ElectricFence, we encountered panics
in the Tcl library.

The problem boils down to this. Tcl's
Tcl_Alloc() routine (and the ckalloc()
form) is documented to return a pointer
aligned for any purpose. Some of the
Tcl source code is written assuming this
to be true.

Unfortunately, when malloc() returns a
pointer that is misaligned for some
purposes (notably for holding a pointer
value), the TclpAlloc() implementation
in tclThreadAlloc.c propagates that
misalignment up to the pointer returned
by ckalloc() leading to panics in the
code that assumes this cannot happen.

Misaligned malloc() returns is one of
the hazards introduced by ElectricFence
testing.

Since Tcl_Alloc() is documented to return
an aligned pointer, and it cannot always
be assumed that malloc() will cooperate,
it would make Tcl more robust, and more
useful for testing in partnership with
ElectricFence, if the allocator in
tclThreadAlloc.c included safeguards to
prevent misalignment from bubbling up
from below.

Discussion

  • Are you sure you have the right culprit here? What you
    describe sounds like a bug in ElectricFence or maybe
    somewhere else.

    > Tcl's Tcl_Alloc() routine (and the ckalloc() form) is
    > documented to return a pointer aligned for any purpose.

    This is the same guarantee that malloc gives. I checked the
    C standard as well as my local man pages (on Mac OS X) about
    this. See also
    <http://www.opengroup.org/onlinepubs/009695399/functions/malloc.html>.

    > Unfortunately, when malloc() returns a pointer that is
    > misaligned for some purposes (notably for holding a
    > pointer value)

    That implementation of malloc would be broken.

    Note that there are no simple functions in standard C that
    can be used to ensure alignment and that a portable
    implementation to do this is non-trivial (from the POV of
    the rather generic C standard at least). A malloc that does
    not ensure alignment would be unreasonable or even useless.

     
  • Don Porter
    Don Porter
    2008-10-24

    By that standard, yes,
    ElectricFence is "broken".

    However, EF is a useful tool,
    and this aspect of its
    brokenness is essential to
    how it functions.

    If Tcl can be revised with
    reasonable effort and without
    significant negative impact
    so that it tolerates this kind
    of brokenness, that would be
    desirable.

     
  • Sounds to me like EF is broken x86_64. A shame. Report it to the EF people and move on. (If their response is "we don't support EF on that architecture" then we'll just have to live without EF on x86_64.)

     
  • > However, EF is a useful tool,

    I'm sure it is. I haven't used it before, but I have heard
    about it.

    > and this aspect of its brokenness is essential to how it
    > functions.

    I just downloaded ElectricFence to understand what you mean.
    From the manual page it seems that this exact issue is
    addressed by its configuration parameter EF_ALIGNMENT and
    the section in the manual "WORD-ALIGNMENT AND OVERRUN
    DETECTION".

    To counteract the problem posed in that section, Tcl_Alloc
    could round up every allocation to a multiple of
    sizeof(void*) before calling malloc. OTOH setting
    EF_ALIGNMENT=8 seems to have the exact same effect (on a 64
    bit system) without requiring changes to Tcl. That seems to
    me to be the better option.

    Or did you have some other change in mind that is not just
    equivalent to EF_ALIGNMENT=8?