|
From: Nicholas N. <n.n...@gm...> - 2009-07-24 21:12:48
|
Hi, https://bugs.kde.org/show_bug.cgi?id=100628 concerns a program like this: #include <stdlib.h> #include "valgrind.h" int main(void) { char* x; x = malloc(1000); VALGRIND_MALLOCLIKE_BLOCK(x, /*szB*/ 16, /*rzB*/0, /*is_zeroed*/0); VALGRIND_MALLOCLIKE_BLOCK(x+100, /*szB*/ 32, /*rzB*/0, /*is_zeroed*/0); VALGRIND_MALLOCLIKE_BLOCK(x+200, /*szB*/ 64, /*rzB*/0, /*is_zeroed*/0); VALGRIND_MALLOCLIKE_BLOCK(x+300, /*szB*/128, /*rzB*/0, /*is_zeroed*/0); return 0; } It's a simplified version of a real problem, where a user implements a custom allocator on top of malloc(). They use malloc() to allocate a big chunk of memory, and then hand out pieces of it, using MALLOCLIKE_BLOCK to mark those pieces as logically separate heap blocks. Currently the leak checker barfs on it because it can't handle overlapping heap blocks. I have a patch that tolerates overlaps such as these, where a custom block falls entirely within a malloc()'d block. But there's a design choice to be made. In such a case, what do we do with the malloc()'d block? (1) Ignore it completely when leak checking (2) Consider it when leak checking It's unclear to me which of these is preferable. (1) treats the malloc() much as a mmap() or brk() would be treated, which is arguably desirable, as many allocators are built on mmap() and/or brk() and so treating malloc() in the same way makes some sense. But (2) involves less special-casing, and is thus slightly easier to do than (1). Anyone have any preferences? Nick |
|
From: Bart V. A. <bar...@gm...> - 2009-07-25 05:11:17
|
On Fri, Jul 24, 2009 at 11:12 PM, Nicholas Nethercote <n.n...@gm...> wrote: > https://bugs.kde.org/show_bug.cgi?id=100628 concerns a program like this: > > #include <stdlib.h> > #include "valgrind.h" > > int main(void) > { > char* x; > > x = malloc(1000); > VALGRIND_MALLOCLIKE_BLOCK(x, /*szB*/ 16, /*rzB*/0, /*is_zeroed*/0); > VALGRIND_MALLOCLIKE_BLOCK(x+100, /*szB*/ 32, /*rzB*/0, /*is_zeroed*/0); > VALGRIND_MALLOCLIKE_BLOCK(x+200, /*szB*/ 64, /*rzB*/0, /*is_zeroed*/0); > VALGRIND_MALLOCLIKE_BLOCK(x+300, /*szB*/128, /*rzB*/0, /*is_zeroed*/0); > > return 0; > } > > It's a simplified version of a real problem, where a user implements a > custom allocator on top of malloc(). They use malloc() to allocate a > big chunk of memory, and then hand out pieces of it, using > MALLOCLIKE_BLOCK to mark those pieces as logically separate heap > blocks. Currently the leak checker barfs on it because it can't > handle overlapping heap blocks. > > I have a patch that tolerates overlaps such as these, where a custom > block falls entirely within a malloc()'d block. But there's a design > choice to be made. In such a case, what do we do with the malloc()'d > block? > > (1) Ignore it completely when leak checking > (2) Consider it when leak checking > > It's unclear to me which of these is preferable. (1) treats the > malloc() much as a mmap() or brk() would be treated, which is arguably > desirable, as many allocators are built on mmap() and/or brk() and so > treating malloc() in the same way makes some sense. But (2) involves > less special-casing, and is thus slightly easier to do than (1). > > Anyone have any preferences? My suggestion is to require that the user makes it clear to Valgrind that the memory allocated by the malloc() call in the example won't be used directly but will be handed out in pieces to the client program. Bart. |
|
From: John R. <jr...@bi...> - 2009-07-25 13:43:16
|
Nicholas Nethercote wrote: [snip] > It's a simplified version of a real problem, where a user implements a > custom allocator on top of malloc(). ... > I have a patch that tolerates overlaps such as these, where a custom > block falls entirely within a malloc()'d block. But there's a design > choice to be made. In such a case, what do we do with the malloc()'d > block? > > (1) Ignore it completely when leak checking > (2) Consider it when leak checking 0. Have a way for the custom allocator to say, "This interval will be used as an arena for custom allocation," and "This interval no longer will be used as such an arena." A custom allocator must tell memcheck these facts. If not then memcheck may adopt any policy at all, including random policy. 1. Overlap is allowed only for a custom arena and the blocks that it allocates. Arenas can nest. An arena can be cleared: all blocks free()d as a group in one operation without specifying each allocated block separately. 2. Each arena is considered to be its own root with respect to blocks that it allocates, and also each arena is considered to be an opaque allocated block within its nearest enclosing arena. Thus the leak report has a separate section for each arena, and an arena can be leaked within its nearest enclosing arena yet still contain its own non-leaked blocks. -- |
|
From: Nicholas N. <n.n...@gm...> - 2009-07-26 22:54:12
|
On Sat, Jul 25, 2009 at 11:37 PM, John Reiser<jr...@bi...> wrote: > >> I have a patch that tolerates overlaps such as these, where a custom >> block falls entirely within a malloc()'d block. But there's a design >> choice to be made. In such a case, what do we do with the malloc()'d >> block? >> >> (1) Ignore it completely when leak checking >> (2) Consider it when leak checking > > 0. Have a way for the custom allocator to say, "This interval > will be used as an arena for custom allocation," and "This interval > no longer will be used as such an arena." A custom allocator > must tell memcheck these facts. If not then memcheck may adopt > any policy at all, including random policy. > > 1. Overlap is allowed only for a custom arena and the blocks that > it allocates. Arenas can nest. An arena can be cleared: all blocks > free()d as a group in one operation without specifying each allocated > block separately. > > 2. Each arena is considered to be its own root with respect to blocks > that it allocates, and also each arena is considered to be > an opaque allocated block within its nearest enclosing arena. > Thus the leak report has a separate section for each arena, > and an arena can be leaked within its nearest enclosing arena > yet still contain its own non-leaked blocks. Hmm. That sounds a lot like the existing MEMPOOL client requests, except maybe for the nesting. And I just tested them and found they can handle the given program -- the strategy used is (1) -- any malloc'd block marked as a mempool is ignored by the leak checker. The MEMPOOL requests probably totally subsume MALLOCLIKE/FREELIKE. It's hard to tell without looking deeper as the documentation for the MEMPOOL requests is paltry. I imagine MALLOCLIKE/FREELIKE could be implemented in terms of MEMPOOL. I'll take a look. Thanks for the suggestion. Nick |