#70 custom allocators with state

open-postponed
nobody
None
5
2004-12-17
2004-10-20
Corey Brenner
No

It is sometimes necessary to use different memory
allocation routines operating on one pool for different kinds
of allocation, or to use different pools with one memory
allocation routine (for instance, allocating data structures
with a fixed-size allocator, and strings from a string heap
utilizing a specialized string allocator).

I have made a non-intrusive patch to the expat
memory-handling suite which should be binary-compatible
for those who are dynamic-linking the expat library.

This patch allows one to provide "pool-allocation" function
pointers, and a void pointer which is passed to the
allocation routines as an extra argument, as part of the
memory-handling suite structure.

For instance:

/* my_allocator prototype, wrapper functions */
extern int my_allocator(int op, void** pp,
size_t siz, void* pool);

void* my_malloc (size_t siz, void* pool) {
void* p = NULL;
(void) my_allocator(OBTAIN, &p, siz, pool);
return p;
}
void* my_realloc (void* old, size_t siz, void* pool) {
(void) my_allocator(RESIZE, &old, siz, pool);
return old;
}
void my_free (void* mem, void* pool) {
(void) my_allocator(RETURN, &mem, 0, pool);
}

/* using it all */
int foo (void* pool) {
XML_Memory_Handling_Suite mhs = {
/* malloc_fcn */ NULL,
/* realloc_fcn */ NULL,
/* free_fcn */ NULL,
/* pool_malloc_fcn */ my_malloc,
/* pool_realloc_fcn */ my_realloc,
/* pool_free_fcn */ my_free,
/* pool */ NULL
};

XML_Parser parser;

mhs.pool = pool;

parser = XML_Parser_Create_MM(NULL, &mhs, NULL);
if (!parser) {
die_horribly();
}
/* ... */
}

Note: XML_Memory_Handling_Suite has changed, but not
in a way that will harm older programs which do not see
these additions. The memory-using macros provided in
this patch first attempt to utilize the old allocators, if
present, and then the new ones, so behavior for older
programs is consistent.

Documentation is updated to reflect the change.

Discussion

  • Corey Brenner
    Corey Brenner
    2004-10-20

     
    Attachments
  • Karl Waclawek
    Karl Waclawek
    2004-12-17

    • status: open --> open-postponed
     
  • Karl Waclawek
    Karl Waclawek
    2004-12-17

    Logged In: YES
    user_id=290026

    We are thinking of a future API change (after release 2.0)
    where the memory allocation call-backs would gain an extra
    parameter similar tothe userData parameter set on the
    parser.

    This looks like your proposal, only more generic.

    One use case we were thinking of is to prevent the
    "million laughs" attack by monitoting memory usage per
    parser instance.

     
  • Logged In: NO

    Did my patch not make it through?

    I did submit one. If not, please allow me to email it to
    someone. My patch does this without breaking the 1.9x API,
    so it can be used currently.

    Thanks,

    --Corey

     
  • Karl Waclawek
    Karl Waclawek
    2004-12-21

    Logged In: YES
    user_id=290026

    We are leaving your patch as a standalone patch for the
    moment, since - as already mentioned - we are thinking of
    doing something similar, but somewhat different, for a post
    2.0 release.