|
From: Nicholas L. <nl...@bl...> - 2016-09-15 19:40:47
|
In the Valgrind User Manual, the Massif chapter mentions: "certain space leaks ... aren't detected by traditional leak-checkers, such as Memcheck's. That's because the memory isn't ever actually lost -- a pointer remains to it -- but it's not in use. ... Massif can help identify these leaks." The sample program and output graph sections explain how to visually interpret the output after it's run through ms_print, as well as the concepts of detailed and heap snapshots, but there isn't any example given of finding space leaks. What are the high-level steps for doing so? Would users examine the detailed snapshots and analyze the breakdown of memory consumption in specific functions to find references that haven't been used in a while? Thanks, Nicholas, QNX Software Systems Ltd. |
|
From: Julian S. <js...@ac...> - 2016-09-16 08:35:04
|
This is all a bit cryptic, but what it means is: Memcheck will tell you about leaks where all pointers to a block are lost, so it can never be freed. But it doesn't say anything about a different kind of leak, in which blocks are continuously allocated over the lifetime of the program, but only ever freed at the end. So memory use ramps up forever despite the fact that in the end, all memory is freed. Massif can tell you about these leaks, because they show up as ever- increasing residency from some specific allocation point. Try using Milian Wolff's excellent massif-visualizer GUI. That makes it very easy to examine the output of Massif in detail and find such leaks. At least on Fedora 23 you can ask the package manager to install it. I'm guessing here, but I suspect you can take the Massif output from a run such as on a phone, move it to a host machine, and view with massif-visualizer there. J On 15/09/16 21:40, Nicholas Lamb wrote: > In the Valgrind User Manual, the Massif chapter mentions: > "certain space leaks ... aren't detected by traditional leak-checkers, such as Memcheck's. > That's because the memory isn't ever actually lost -- a pointer remains to it -- > but it's not in use. ... Massif can help identify these leaks." > > The sample program and output graph sections explain how to visually interpret > the output after it's run through ms_print, as well as the concepts of > detailed > and heap snapshots, but there isn't any example given of finding space leaks. > What are the high-level steps for doing so? Would users examine the detailed > snapshots and analyze the breakdown of memory consumption in specific functions > to find references that haven't been used in a while? |
|
From: Nicholas L. <nl...@bl...> - 2016-09-16 15:37:19
|
I think Memcheck actually can say something about accessible but unreleased blocks, but by default it doesn't. The show-leak-kinds option, according to the Valgrind manual, can contain the 'reachable' value. In this case, Memcheck reports blocks that could have been freed but weren't. But I guess it depends on what you mean by "only freed at the end". If you mean there's an explicit call to free() or delete, then Memcheck obviously wouldn't report these blocks. If you mean the programmer makes no such call, knowing that the OS cleans up the memory when the program exits, these blocks would qualify as 'reachable' leaks. I suppose the advantage of Massif is that it reveals the breakdown of heap memory distribution, so you can know how much memory is taken by accessible but unreleased blocks. I'm using a variant of the Eclipse IDE that provides good visualization of Massif's output, so I guess it's just a matter of examining the detailed snapshots to find memory blocks that account for a large portion of overall used memory but haven't been used for a while. - Nicholas. -----Original Message----- From: Julian Seward [mailto:js...@ac...] Sent: Friday, September 16, 2016 4:35 AM To: Nicholas Lamb <nl...@bl...>; val...@li... Subject: Re: [Valgrind-users] Using Massif to find space leaks This is all a bit cryptic, but what it means is: Memcheck will tell you about leaks where all pointers to a block are lost, so it can never be freed. But it doesn't say anything about a different kind of leak, in which blocks are continuously allocated over the lifetime of the program, but only ever freed at the end. So memory use ramps up forever despite the fact that in the end, all memory is freed. Massif can tell you about these leaks, because they show up as ever- increasing residency from some specific allocation point. Try using Milian Wolff's excellent massif-visualizer GUI. That makes it very easy to examine the output of Massif in detail and find such leaks. At least on Fedora 23 you can ask the package manager to install it. I'm guessing here, but I suspect you can take the Massif output from a run such as on a phone, move it to a host machine, and view with massif-visualizer there. J On 15/09/16 21:40, Nicholas Lamb wrote: > In the Valgrind User Manual, the Massif chapter mentions: > "certain space leaks ... aren't detected by traditional leak-checkers, such as Memcheck's. > That's because the memory isn't ever actually lost -- a pointer > remains to it -- > but it's not in use. ... Massif can help identify these leaks." > > The sample program and output graph sections explain how to visually > interpret the output after it's run through ms_print, as well as the > concepts of detailed and heap snapshots, but there isn't any example > given of finding space leaks. > What are the high-level steps for doing so? Would users examine the > detailed snapshots and analyze the breakdown of memory consumption in > specific functions > to find references that haven't been used in a while? |
|
From: Philippe W. <phi...@sk...> - 2016-09-17 08:31:26
|
On Fri, 2016-09-16 at 15:37 +0000, Nicholas Lamb wrote:
> I think Memcheck actually can say something about accessible but unreleased blocks, but by default it doesn't.
> The show-leak-kinds option, according to the Valgrind manual, can contain the 'reachable' value. In this case,
> Memcheck reports blocks that could have been freed but weren't.
Yes, this is correct : memcheck --show-leak-kinds=... option can show
the allocated not (yet) released memory. To find (small) increase of
reachable memory (e.g. in regression tests), you can even do a
'delta leak search' showing the difference with the memory state at the
previous leak search.
> I suppose the advantage of Massif is that it reveals the breakdown of heap memory distribution,
> so you can know how much memory is taken by accessible but unreleased blocks. I'm using a
> variant of the Eclipse IDE that provides good visualization of Massif's output, so I guess
> it's just a matter of examining the detailed snapshots to find memory blocks that account
> for a large portion of overall used memory but haven't been used for a while.
Effectively, the main differences between massif and memcheck leak
search are:
* massif reports the memory use in a 'tree like layout' which
gives a better visualisation that the memcheck 'flat report' given
per allocation stack trace.
* massif makes reports at regular interval to show memory usage
evolution, and shows the peak usage.
* memcheck delta leak search is more precise to detect small
increase.
Note that I am busy adding an 'execution tree' module
pub_tool_xtree.h
that generalises the way massif records and present 'numbers'.
The idea is that such an xtree can be used by various tools
(a.o. memcheck and massif). Various output format will be available
for an xtree (callgrind format, massif format, ..).
memcheck can already produce a
'kcachegrind/callgrind memory status'.
I have just started implementing the massif output format.
I will then change massif to use this module rather than its
current 'Xpt'.
After that, probably that all tools that are tracking malloc/free
(memcheck, helgrind, ...) will be able to produce massif like
memory reports with very little additional code.
This is all (evening) work in progress, estimate time of arrival
unknown :)
Philippe
|
|
From: Julian S. <js...@ac...> - 2016-09-23 06:02:48
|
On 16/09/16 17:37, Nicholas Lamb wrote: > I suppose the advantage of Massif is that it reveals the breakdown > of heap memory distribution, so you can know how much memory is taken > by accessible but unreleased blocks. I'm using a variant of the Eclipse > IDE that provides good visualization of Massif's output, so I guess it's > just a matter of examining the detailed snapshots to find memory blocks > that account for a large portion of overall used memory but haven't > been used for a while. You can get yet another kind of insight into your heap behaviour using the exp-dhat tool. This observes both heap blocks and the memory accesses made to them, and so can tell you * places where blocks are allocated and freed very soon afterwards -- possible inefficiency -- maybe that data could be stack-allocated instead? * blocks which are allocated and later freed but remain partially or entirely unused, or under-used. From this information you can find sometimes surprising facts like (for example) "50% of our hash tables never have any entries in them", etc. J |