Re: [TCLCORE] Tcl C API Warts

[Trevor D. (Twylite) <tw...@cr...>]

Hi,

On 2012/12/13 12:57 PM, Gustaf Neumann wrote:
> Am 13.12.12 11:30, schrieb Trevor Davel (Twylite):
>> By making the macros malloc-compatible it is trivial to define them as
>> malloc(), free(), etc. and link against an external implementation.  BUT
>> the default implementation on Windows MUST NOT be malloc() and friends,
>> or alloc/free across DLL boundaries will crash the application.
> Not only tcl has the desire to use different mallocs, others have the
> same (e.g. firefox).
> Here are some options for win listed, none of these are very appealing
> http://stackoverflow.com/questions/858592/windows-malloc-replacement-e-g-tcmalloc-and-dynamic-crt-linking
These are solutions for providing a different malloc implementation for 
a code base that uses malloc() to allocate memory.  These workarounds 
exist precisely because the code base uses malloc() rather than a 
macro.  Furthermore none of them solve the problem of a crash if the 
malloc()/free() happens across a DLL boundary; in fact, (1) (2) and (3) 
exacerbate the problem by guaranteeing that each DLL will have a 
distinct heap.

> Makeing malloc() pluggable does not necessarily mean via external .dll,
> several malloc implementations could be shipped as source with tcl
> (similar to e.g. regepx). Using a macro (which might be defined
> differently depending on the OS) should be doable, but it should have
> the same signature.
My point is simply this: using malloc() - rather than a macro - in the 
code base is BAD if (a) the program has multiple heaps and (b) there is 
any possibility of allocating memory from one heap and trying to free it 
from another.

In context: using malloc() - rather than a macro - in the code base is 
BAD if (a.1) the software must run on Windows and (a.2) any dynamic 
linking occurs in the process and (b) there is any possibility of 
allocating memory in one module (DLL) and freeing it another.

Tcl clearly satisfies (a) and (b), and I contend Tcl + extensions will 
satisfy (c) because Unix-based developers (and many Windows-based 
developers) are unaware of the potential for each DLL to have an 
independent heap.

All the bad is mitigated by using Tcl-specific macros or Tcl-specific 
functions to manage all Tcl-specific memory resources that may migrate 
across DLL boundaries.  And this approach also makes it trivial for the 
tcl.dll to choose the memory allocation implementation (whether by 
external DLL or by compiling different source code).

> . Using a macro (which might be defined
> differently depending on the OS) should be doable, but it should have
> the same signature.
Yes - the macro should have the same signature on all platforms, and the 
signature for the Tcl_malloc() macro should match POSIX / ISO C malloc().

Then on *nix: #define Tcl_malloc malloc /* void* malloc(size_t size) */
And on Win: void *Tcl_malloc(size_t size) { return malloc(size); }
Optional: on all platforms use TclCustomMalloc instead of malloc above, 
then #define TclCustomMalloc to malloc or your preferred 
malloc-compatible library.

The Windows shim ensures that all calls to malloc() happen in the 
context of the tcl.dll, and thus uses a single malloc implementation and 
its associated heap(s).

Actually you want to use a Tcl_malloc() function on *nix as well.  By 
allowing a custom allocator you introduce the multiple heaps problem to 
Tcl (as opposed to it being a Windows phenomenon): different build 
configurations for an extension could lead to the extension having an 
independent heap.  Crash.

So really the only sane options are:
(1) All platforms manage memory via functions (not macros) void 
*Tcl_malloc() and Tcl_free(void*); or
(2) Ensure that alloc/free can never straddle a DLL boundary (experience 
suggests that coders aren't good at doing so).

In other words, Tcl already does this right at the API level.

Regards,
Twylite