|
From: <and...@co...> - 2007-12-21 11:48:30
|
Hello all. I've read in the documentation that the Callgrind module supports =20 interactive dumping through callgrind_control. I would like to do the same with Memcheck. For example, I have a =20 program that during it's execution consumes more and more memory but =20 when it exits it frees up all of it, so memcheck does not detect leaks =20= (there aren't any). I need to be able to signal memcheck to dump the =20 memory usage and who allocated it in the middle of the program. It =20 this possible? I know I can do that by calling valgrind methods in the =20= code but I would much prefer not having to alter the program's source =20= code. There is also the possibility of sending a signal to the process =20 causing it to terminate without letting it run the cleanup code but =20 this means stopping the process... Best regards, Andr=E9 Cruz |
|
From: Jason C. <co...@cc...> - 2007-12-21 12:03:29
|
I could be wrong, but I believe the 'massif' tool is meant for this kind of memory analysis? http://valgrind.org/docs/manual/ms-manual.html Massif has been re-written recently (see earlier mails on the list). J. > Hello all. > > I've read in the documentation that the Callgrind module supports > interactive dumping through callgrind_control. > > I would like to do the same with Memcheck. For example, I have a > program that during it's execution consumes more and more memory but > when it exits it frees up all of it, so memcheck does not detect leaks > (there aren't any). I need to be able to signal memcheck to dump the > memory usage and who allocated it in the middle of the program. It > this possible? I know I can do that by calling valgrind methods in the > code but I would much prefer not having to alter the program's source > code. > > There is also the possibility of sending a signal to the process > causing it to terminate without letting it run the cleanup code but > this means stopping the process... > > > Best regards, > André Cruz > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Valgrind-users mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-users > LEGAL NOTICE Unless expressly stated otherwise, information contained in this message is confidential. If this message is not intended for you, please inform pos...@cc... and delete the message. The Cambridge Crystallographic Data Centre is a company Limited by Guarantee and a Registered Charity. Registered in England No. 2155347 Registered Charity No. 800579 Registered office 12 Union Road, Cambridge CB2 1EZ. |
|
From: Julian S. <js...@ac...> - 2007-12-21 13:10:29
|
On Friday 21 December 2007 13:03, Jason Cole wrote: > I could be wrong, but I believe the 'massif' tool is meant for this kind > of memory analysis? Yes. Massif is intended for exactly this kind of problem. And it is much improved in 3.3.0, so make sure you use that version. J |
|
From: Nicholas N. <nj...@cs...> - 2007-12-22 00:31:58
|
On Fri, 21 Dec 2007, Julian Seward wrote: > On Friday 21 December 2007 13:03, Jason Cole wrote: >> I could be wrong, but I believe the 'massif' tool is meant for this kind >> of memory analysis? > > Yes. Massif is intended for exactly this kind of problem. And it > is much improved in 3.3.0, so make sure you use that version. But Andre's initial point is still valid -- a lot of people seem to find the callgrind_control stuff useful. It would be nice if we made it part of the core so that multiple tools would be able to use it. For example, it would be great for the user to be able to control when Massif does its detailed heap snapshots -- currently Massif just does them every so often. N |
|
From: Josef W. <Jos...@gm...> - 2007-12-22 01:52:59
|
On Saturday 22 December 2007, Nicholas Nethercote wrote: > On Fri, 21 Dec 2007, Julian Seward wrote: > > > On Friday 21 December 2007 13:03, Jason Cole wrote: > >> I could be wrong, but I believe the 'massif' tool is meant for this kind > >> of memory analysis? > > > > Yes. Massif is intended for exactly this kind of problem. And it > > is much improved in 3.3.0, so make sure you use that version. > > But Andre's initial point is still valid -- a lot of people seem to find the > callgrind_control stuff useful. It would be nice if we made it part of the > core so that multiple tools would be able to use it. While I think this is a very good idea, we should use another technique; actually, I am not very proud of how this stuff is implemented. Callgrind's interactive support is done by regular polling for the existance of a given file from tool code (which is called from instrumentation). This is problematic in at least 2 aspects: * Polling is wasting CPU cycles, and does a lot of system calls. * It does not work at all when the client is sleeping. This is really bad. It would be better if VG could spawn a control thread and wait for command connections e.g. by listening on a socket. The hook call into the tool should be done in a serialized way to the client threads, to have a consistent state and avoid any racy behavior. In Callgrind, to see whether a simulation is running, another temporary file is created in /tmp, "callgrind.info.<pid>". callgrind_control searches for such files, and checks via /proc/<pid>/maps that there really is such a process running under valgrind. The callgrind.info file specifies the exact file which has to be created to send interactive commands to the given Callgrind run. I do not think this would be needed when a VG run could be contacted via sockets, as these are global for a machine... Josef > For example, it would > be great for the user to be able to control when Massif does its detailed > heap snapshots -- currently Massif just does them every so often. > > N > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Valgrind-users mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-users > |
|
From: Nicholas N. <nj...@cs...> - 2008-01-04 00:15:46
|
On Sat, 22 Dec 2007, Josef Weidendorfer wrote: >> But Andre's initial point is still valid -- a lot of people seem to find the >> callgrind_control stuff useful. It would be nice if we made it part of the >> core so that multiple tools would be able to use it. > > While I think this is a very good idea, we should use another technique; > actually, I am not very proud of how this stuff is implemented. > > Callgrind's interactive support is done by regular polling for the existance > of a given file from tool code (which is called from instrumentation). > > This is problematic in at least 2 aspects: > * Polling is wasting CPU cycles, and does a lot of system calls. > * It does not work at all when the client is sleeping. This is really bad. > > It would be better if VG could spawn a control thread and wait for command > connections e.g. by listening on a socket. The hook call into the tool > should be done in a serialized way to the client threads, to have a > consistent state and avoid any racy behavior. > > In Callgrind, to see whether a simulation is running, another temporary > file is created in /tmp, "callgrind.info.<pid>". callgrind_control searches > for such files, and checks via /proc/<pid>/maps that there really is such a > process running under valgrind. > The callgrind.info file specifies the exact file which has to be created to > send interactive commands to the given Callgrind run. > > I do not think this would be needed when a VG run could be contacted via > sockets, as these are global for a machine... So if I understand correctly: - Callgrind creates a file in /tmp - callgrind_control searches for such files and checks /proc/<pid>/maps for the process - the file specifies to callgrind what to do (?) That is a bit complex. Basically, this requires IPC (inter-process communication). Using files has one definite potential virtue: it's really simple and portable, if done carefully -- all OSes support files. But the no-response-when-sleeping issue is bad. Other possibilities: - Signals -- we already reserve the top two signals for Valgrind, so another is possible. But they're not very informative, so we'd need an auxiliary mechanism (eg. a file) to pass extra info. - Pipes. These seem good. I guess it would be like how signals are currently handled -- the Valgrind tool would be set up to read from the pipe, and every so often the core would check to see if any new input has come in on the pipe. I'm not sure how the Valgrind tool would know what the pipe's name is (assuming it's named). Maybe having a standard name like "valgrind-<pid>-pipe" or somesuch would work. - Sockets. I don't know much about them. Having to spawn a control thread sounds complicated, though. Our current one-thread-per-thread scheme is nice and stable, I'd be reluctant to lose that. - Shared memory. Seems like a bad idea. But I really don't know much about IPC, so others may have more informed opinions. Nick |
|
From: Josef W. <Jos...@gm...> - 2008-01-04 01:09:35
|
On Friday 04 January 2008, Nicholas Nethercote wrote: > So if I understand correctly: > > - Callgrind creates a file in /tmp > - callgrind_control searches for such files and checks /proc/<pid>/maps > for the process > - the file specifies to callgrind what to do (?) > > That is a bit complex. Even more complex: "the file specifies the *name of a file* which regularly is polled by callgrind for existance, and with the command as content". Aside from that, yes, it is complex. I started with the file approach because it does not need any file descriptor open all the time. Descriptors are closed directly after each single action. This "design" still is from Valgrind 1.x days... > Basically, this requires IPC (inter-process communication). Using files has > one definite potential virtue: it's really simple and portable, if done > carefully -- all OSes support files. Yes. But it does not feel like the right thing for interactive control. > But the no-response-when-sleeping > issue is bad. If we really can use a signal, this gets a non-issue (and also does not need the control thread). > > Other possibilities: > > - Signals -- we already reserve the top two signals for Valgrind, so another > is possible. But they're not very informative, so we'd need an auxiliary > mechanism (eg. a file) to pass extra info. Yes. > - Pipes. These seem good. I guess it would be like > how signals are currently handled -- the Valgrind tool would be set > up to read from the pipe, and every so often the core would check > to see if any new input has come in on the pipe. I'm not sure how > the Valgrind tool would know what the pipe's name is (assuming it's > named). Maybe having a standard name like "valgrind-<pid>-pipe" or > somesuch would work. In principle, this looks similar to the file approach. Named pipes are persistant, and you need a way for cleanup in the case of a VG crash. Perhaps it is better to make the command sender responsible for creating the pipe, and valgrind checks for the existance under a defined name (such as your suggestion). Doesn't "read ... every so often" also have the sleep problem? We should add a signal to trigger the check. > - Sockets. I don't know much about them. Having to spawn a control thread > sounds complicated, though. Our current one-thread-per-thread scheme > is nice and stable, I'd be reluctant to lose that. With a signal, a dedicated control thread is not needed. Hmmm... Allowing network connections to control a valgrind tool is perhaps not the best thing (security issues come to mind). And unix domain sockets are similar to named pipes for the given task (?)... > - Shared memory. Seems like a bad idea. Not needed. Interactive commands should require only low bandwidth. [Shared memory IPC would be really good for sending chunks of events for further processing in another process]. > But I really don't know much about IPC, so others may have more informed > opinions. I am also not sure what's the best... Perhaps a combination of pipe and signal could work fine. Josef > > Nick > |
|
From: Julian S. <js...@ac...> - 2008-01-04 01:39:59
|
> > Callgrind's interactive support is done by regular polling for the > > existance of a given file from tool code (which is called from > > instrumentation). > > > > This is problematic in at least 2 aspects: > > * Polling is wasting CPU cycles, and does a lot of system calls. > > * It does not work at all when the client is sleeping. This is really > > bad. Although the polling scheme has these 2 disadvantages, the other approaches involving signals and extra threads worry me. The signal/syscall/thread interactions are already a nightmare of complexity and I'm less than enthusiastic about messing with that stuff. At least the current Callgrind scheme localises its complexity in one place. The performance aspects are not a big deal, because we can make the scheduler polling-aware - let it decide when to poll, and, if the intervals get too close together, increase the scheduling quantum (or something similar) so as to limit the polling rate. The only real problem is the case where all client threads are sleeping. J |
|
From: Josef W. <Jos...@gm...> - 2008-01-04 01:59:50
|
On Friday 04 January 2008, Julian Seward wrote: > At least the current Callgrind scheme localises its complexity > in one place. The performance aspects are not a big deal, because > we can make the scheduler polling-aware - let it decide when to poll, > and, if the intervals get too close together, increase the scheduling > quantum (or something similar) so as to limit the polling rate. Yes, that sounds fine. > The only real problem is the case where all client threads are > sleeping. What do you suggest as possible solution? Does the complexity issue also cover "just" a VG-specific signal to trigger polling for an interactive command? Hmm... how does this work at all? If there is a signal from another process, how can VG decide whether it is for the client or VG itself? For VG-internal signals, this is no issue... Josef > > J > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Valgrind-users mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-users > |