|
From: tunaNO@SPAMindra.com - 2004-06-07 21:48:39
|
i'm trying to use massif to analyze the behavior of some rather large
applications. for many "interesting" problem sizes, massif/valgrind
(this is in the context of valgrind 2.1.1) blows out with:
valgrind: vg_mylibc.c:1681 (vgPlain_get_memory_from_mmap): Assertion
`p >= (void *)vgPlain_valgrind_mmap_end && p < (void *)vgPlain_valgrind_end'
failed.
vg_mylibc.c:1681 corresponds to a bounds-checking assertion in
"VG_(get_memory_from_mmap)"; i've instrumented that code and verified
that the problem appears to be an _overflow_ of those bounds.
because i don't see the same problems running the same applications
and problem sizes under calltree/valgrind, i'm assuming this is a
massif-specific issue. digging around, i see a call to
"VG_(get_memory_from_mmap)" at ms_main.c:372 coming out of
"perm_malloc", which is prefaced with the following comment:
// Cheap allocation for blocks that never need to be freed. Saves
// about 10% for Konqueror startup with --depth=40.
"perm_malloc" looks to be called from "new_XPt", which appears to be
the code that is used to allocate new execution/allocation points in
the instrumentation scheme used by massif. so one assumes that the
assertion at vg_mylibc.c:1681 is ultimately failing because we've run
out of space to allocate more XPt objects, and perhaps that there is
something interesting about these applications (long runtimes? large
number of allocation points?) that is causing enough XPt objects to be
allocated for me to run afoul of such problems.
it is unclear to me at this point whether or not the periodic calls to
"halve_censi" may in fact be removing some XPt objects from further
consideration without providing a means for ever reusing them, thus
exacerbating the problems suggested above.
without digging into ms_main.c too much further (yet!), it appears to
me that there might be a handful of different ways of attacking this
problem:
1) assuming "halve_censi" (or some other mechanism?) does indeed
remove some XPt objects from further consideration without
providing a means for ever reusing them, provide such a
means:
a) simplest solution here is probably a freelist that
no-longer-used XPt objects are pushed onto, and new XPt objects
are popped from when possible.
b) alternate approach would be to eliminate the calls to
"perm_malloc" in favor of calls to a conventional "malloc",
taking care to insert corresponding calls to "free" for XPt
objects that will no longer be used.
2) provide a means of expanding the size of the memory segment that
is available for "VG_(get_memory_from_mmap)" to work with.
of these, 1a would seem to be the most straightforward and promising
approach, but that's only if the assumption about XPt object use holds
true, which i'm completely unclear on.
thoughts/comments/suggestions?
tx,
kirk
ps. i'm not subscribed to val...@li..., so
please include my e-mail address (modulo obvious anti-spam -- remove
the NO and SPAM) in your response. just in case, i'll also try to keep
an eye on the valgrind-users archives ...
|
|
From: Nicholas N. <nj...@ca...> - 2004-06-08 07:40:33
|
On Mon, 7 Jun 2004 tunaNO@SPAMindra.com wrote: > i'm trying to use massif to analyze the behavior of some rather large > applications. for many "interesting" problem sizes, massif/valgrind > (this is in the context of valgrind 2.1.1) blows out with: > > valgrind: vg_mylibc.c:1681 (vgPlain_get_memory_from_mmap): Assertion > `p >= (void *)vgPlain_valgrind_mmap_end && p < (void *)vgPlain_valgrind_end' > failed. > > vg_mylibc.c:1681 corresponds to a bounds-checking assertion in > "VG_(get_memory_from_mmap)"; i've instrumented that code and verified > that the problem appears to be an _overflow_ of those bounds. > > because i don't see the same problems running the same applications > and problem sizes under calltree/valgrind, i'm assuming this is a > massif-specific issue. I think it's a Valgrind core issue, that perhaps only Massif exposes. It may be a manifestation of bug #82301 (bugs.kde.org/show_bug.cgi?id=82301). Josef, is this the same error you are getting with Calltree on big programs? As a temporary workaround, you could try installing Valgrind 2.0.0, which lays out memory differently, and then plumbing in Massif (it wasn't added until 2.1.0), which will require some judicious copying and editing of configure.in and Makefile.am. > without digging into ms_main.c too much further (yet!), it appears to > me that there might be a handful of different ways of attacking this > problem: > > 1) assuming "halve_censi" (or some other mechanism?) does indeed > remove some XPt objects from further consideration without > providing a means for ever reusing them, provide such a > means: > > a) simplest solution here is probably a freelist that > no-longer-used XPt objects are pushed onto, and new XPt objects > are popped from when possible. > > b) alternate approach would be to eliminate the calls to > "perm_malloc" in favor of calls to a conventional "malloc", > taking care to insert corresponding calls to "free" for XPt > objects that will no longer be used. > > 2) provide a means of expanding the size of the memory segment that > is available for "VG_(get_memory_from_mmap)" to work with. > > of these, 1a would seem to be the most straightforward and promising > approach, but that's only if the assumption about XPt object use holds > true, which i'm completely unclear on. Hmm, I hadn't considered that some XPts could become non-live. My gut feeling is that the number wouldn't be that significant, but I would be happy to be proved wrong. The number of XPts is bound by the size of the program, rather than the running time. It's also possible that an XPt could become dead and then alive again later, in which case it would be reallocated. I think option 2 would be the better one. N |
|
From: kirk j. <tu...@in...> - 2004-06-08 16:06:15
|
KJ = kirk johnson
NN = nicholas nethercote
KJ > i'm trying to use massif to analyze the behavior of some rather
> large applications. for many "interesting" problem sizes,
> massif/valgrind (this is in the context of valgrind 2.1.1) blows
> out with:
>
> valgrind: vg_mylibc.c:1681 (vgPlain_get_memory_from_mmap):
> Assertion `p >= (void *)vgPlain_valgrind_mmap_end && p < (void
> *)vgPlain_valgrind_end' failed.
>
> vg_mylibc.c:1681 corresponds to a bounds-checking assertion in
> "VG_(get_memory_from_mmap)"; i've instrumented that code and
> verified that the problem appears to be an _overflow_ of those
> bounds.
>
> because i don't see the same problems running the same
> applications and problem sizes under calltree/valgrind, i'm
> assuming this is a massif-specific issue.
NN > I think it's a Valgrind core issue, that perhaps only Massif
> exposes.
entirely possible.
NN > It may be a manifestation of bug #82301
> (bugs.kde.org/show_bug.cgi?id=82301).
yeah, i'm seeing a different failure mode, but i think the basic
problem is the same -- i'm running out of mmap space.
NN > As a temporary workaround, you could try installing Valgrind
> 2.0.0, which lays out memory differently, and then plumbing in
> Massif (it wasn't added until 2.1.0), which will require some
> judicious copying and editing of configure.in and Makefile.am.
another option would be to try some version of the patch that's listed
at the end of bug #82301.
NN > Hmm, I hadn't considered that some XPts could become non-live.
> My gut feeling is that the number wouldn't be that significant,
> but I would be happy to be proved wrong.
mostly a guess on my part, no data to support it -- i was simply
trying to come up with an explanation for why we were going through
128 MB worth of XPt objects ...
NN > The number of XPts is bound by the size of the program, rather
> than the running time. It's also possible that an XPt could
> become dead and then alive again later, in which case it would
> be reallocated.
exactly the sort of thing i was wondering about.
NN > I think option 2 would be the better one.
good for a short term fix, but if a significant number of XPt objects
are going non-live and/or then being reallocated, then it may not be
the right long-term fix ...
tx for the pointers, i'll see if i can get something working today and
let you know what happens.
kirk
|
|
From: kirk j. <tu...@in...> - 2004-06-10 07:48:19
|
NN = nicholas nethercote
KJ = kirk johnson
NN > As a temporary workaround, you could try installing Valgrind
> 2.0.0, which lays out memory differently, and then plumbing in
> Massif (it wasn't added until 2.1.0), which will require some
> judicious copying and editing of configure.in and Makefile.am.
KJ > another option would be to try some version of the patch that's
> listed at the end of bug #82301.
i tried the approach listed at the end of bug #82301 -- i tweaked
VALGRIND_MAPSIZE to 256 MB and info.map_base to 0xa8000000 -- but it
didn't change the failure mode. in fact, even if i upped
VALGRIND_MAPSIZE to 512 MB (and made the corresponding change to
info.map_base), that still doesn't help. even worse, things still fail
at essentially the same point in the program execution.
digging a little deeper, it ends up being rather obvious why this is
-- the VG_(get_memory_from_mmap) code is allocating from between
VG_(valgrind_mmap_end) and VG_(valgrind_end), and the values of these
variables are not affected by the dinking around with VALGRIND_MAPSIZE
and info.map_base. said dinking _does_ cause the value of
VG_(valgrind_base) to change, but since that doesn't affect the range
of memory available for VG_(get_memory_from_mmap) to allocate from, it
doesn't address the problem at hand.
brief thought was given to the idea of replacing the calls to
perm_malloc with calls to VG_(malloc), but that wouldn't help either
-- in the end, memory allocated by VG_(malloc) is also coming out of
the pool managed by VG_(get_memory_from_mmap).
looking more closely at coregrind/vg_main.c, i see that
VG_(valgrind_mmap_end) and VG_(valgrind_end) are set up in
layout_client_space(). alas, the values assigned to these variables
appear to be out of my control -- VG_(valgrind_mmap_end) is coming
from the location of kickstart_end (apparently a "magic" symbol
defined by the linker/loader?), while VG_(valgrind_end) is coming from
the location at which main() is passed its arguments. thus, we're left
with a handful of possible solutions:
1) find a way to change the locations of kickstart_end and/or the
arguments that are passed to main(), thus increasing the amount of
space availble to VG_(get_memory_from_mmap). although this doesn't
_solve_ the problem, perhaps it puts it off far enough to work
around my immediate difficulties.
2) revise the memory allocation scheme such that once
VG_(get_memory_from_mmap) runs out of space between
VG_(valgrind_mmap_end) and VG_(valgrind_end), additional memory
segments can be allocated from elsewhere (either the same pool
that the client heap is being drawn from or from unused parts of
the VG_(valgrind_base) to VG_(valgrind_mmap_end) range). this
would seem to run at least partly counter to the
full-virtualization efforts (and may in fact be closer to how
things were done in valgrind-2.0.0; i haven't looked at the code
to see).
3) get lucky and discover that non-trivial numbers of XPt objects are
going non-live (and possibly being reallocated later); provide a
means of reclaiming these XPt objects and any associated memory
(e.g., free the child data, freelist the XPt objects). obviously
this only helps if non-trivial numbers of XPt objects are going
non-live, an assumption that i have not investigated further.
thoughts?
tx for the help,
kirk
|