You can subscribe to this list here.
| 2003 |
Jan
|
Feb
|
Mar
(58) |
Apr
(261) |
May
(169) |
Jun
(214) |
Jul
(201) |
Aug
(219) |
Sep
(198) |
Oct
(203) |
Nov
(241) |
Dec
(94) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2004 |
Jan
(137) |
Feb
(149) |
Mar
(150) |
Apr
(193) |
May
(95) |
Jun
(173) |
Jul
(137) |
Aug
(236) |
Sep
(157) |
Oct
(150) |
Nov
(136) |
Dec
(90) |
| 2005 |
Jan
(139) |
Feb
(130) |
Mar
(274) |
Apr
(138) |
May
(184) |
Jun
(152) |
Jul
(261) |
Aug
(409) |
Sep
(239) |
Oct
(241) |
Nov
(260) |
Dec
(137) |
| 2006 |
Jan
(191) |
Feb
(142) |
Mar
(169) |
Apr
(75) |
May
(141) |
Jun
(169) |
Jul
(131) |
Aug
(141) |
Sep
(192) |
Oct
(176) |
Nov
(142) |
Dec
(95) |
| 2007 |
Jan
(98) |
Feb
(120) |
Mar
(93) |
Apr
(96) |
May
(95) |
Jun
(65) |
Jul
(62) |
Aug
(56) |
Sep
(53) |
Oct
(95) |
Nov
(106) |
Dec
(87) |
| 2008 |
Jan
(58) |
Feb
(149) |
Mar
(175) |
Apr
(110) |
May
(106) |
Jun
(72) |
Jul
(55) |
Aug
(89) |
Sep
(26) |
Oct
(96) |
Nov
(83) |
Dec
(93) |
| 2009 |
Jan
(97) |
Feb
(106) |
Mar
(74) |
Apr
(64) |
May
(115) |
Jun
(83) |
Jul
(137) |
Aug
(103) |
Sep
(56) |
Oct
(59) |
Nov
(61) |
Dec
(37) |
| 2010 |
Jan
(94) |
Feb
(71) |
Mar
(53) |
Apr
(105) |
May
(79) |
Jun
(111) |
Jul
(110) |
Aug
(81) |
Sep
(50) |
Oct
(82) |
Nov
(49) |
Dec
(21) |
| 2011 |
Jan
(87) |
Feb
(105) |
Mar
(108) |
Apr
(99) |
May
(91) |
Jun
(94) |
Jul
(114) |
Aug
(77) |
Sep
(58) |
Oct
(58) |
Nov
(131) |
Dec
(62) |
| 2012 |
Jan
(76) |
Feb
(93) |
Mar
(68) |
Apr
(95) |
May
(62) |
Jun
(109) |
Jul
(90) |
Aug
(87) |
Sep
(49) |
Oct
(54) |
Nov
(66) |
Dec
(84) |
| 2013 |
Jan
(67) |
Feb
(52) |
Mar
(93) |
Apr
(65) |
May
(33) |
Jun
(34) |
Jul
(52) |
Aug
(42) |
Sep
(52) |
Oct
(48) |
Nov
(66) |
Dec
(14) |
| 2014 |
Jan
(66) |
Feb
(51) |
Mar
(34) |
Apr
(47) |
May
(58) |
Jun
(27) |
Jul
(52) |
Aug
(41) |
Sep
(78) |
Oct
(30) |
Nov
(28) |
Dec
(26) |
| 2015 |
Jan
(41) |
Feb
(42) |
Mar
(20) |
Apr
(73) |
May
(31) |
Jun
(48) |
Jul
(23) |
Aug
(55) |
Sep
(36) |
Oct
(47) |
Nov
(48) |
Dec
(41) |
| 2016 |
Jan
(32) |
Feb
(34) |
Mar
(33) |
Apr
(22) |
May
(14) |
Jun
(31) |
Jul
(29) |
Aug
(41) |
Sep
(17) |
Oct
(27) |
Nov
(38) |
Dec
(28) |
| 2017 |
Jan
(28) |
Feb
(30) |
Mar
(16) |
Apr
(9) |
May
(27) |
Jun
(57) |
Jul
(28) |
Aug
(43) |
Sep
(31) |
Oct
(20) |
Nov
(24) |
Dec
(18) |
| 2018 |
Jan
(34) |
Feb
(50) |
Mar
(18) |
Apr
(26) |
May
(13) |
Jun
(31) |
Jul
(13) |
Aug
(11) |
Sep
(15) |
Oct
(12) |
Nov
(18) |
Dec
(13) |
| 2019 |
Jan
(12) |
Feb
(29) |
Mar
(51) |
Apr
(22) |
May
(13) |
Jun
(20) |
Jul
(13) |
Aug
(12) |
Sep
(21) |
Oct
(6) |
Nov
(9) |
Dec
(5) |
| 2020 |
Jan
(13) |
Feb
(5) |
Mar
(25) |
Apr
(4) |
May
(40) |
Jun
(27) |
Jul
(5) |
Aug
(17) |
Sep
(21) |
Oct
(1) |
Nov
(5) |
Dec
(15) |
| 2021 |
Jan
(28) |
Feb
(6) |
Mar
(11) |
Apr
(5) |
May
(7) |
Jun
(8) |
Jul
(5) |
Aug
(5) |
Sep
(11) |
Oct
(9) |
Nov
(10) |
Dec
(12) |
| 2022 |
Jan
(7) |
Feb
(13) |
Mar
(8) |
Apr
(7) |
May
(12) |
Jun
(27) |
Jul
(14) |
Aug
(27) |
Sep
(27) |
Oct
(17) |
Nov
(17) |
Dec
|
| 2023 |
Jan
(10) |
Feb
(18) |
Mar
(9) |
Apr
(26) |
May
|
Jun
(13) |
Jul
(18) |
Aug
(5) |
Sep
(6) |
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
|
|
|
|
|
1
(3) |
2
(1) |
|
3
|
4
(8) |
5
(4) |
6
(11) |
7
(9) |
8
(5) |
9
|
|
10
(4) |
11
(5) |
12
(1) |
13
(4) |
14
(4) |
15
(1) |
16
|
|
17
|
18
|
19
(3) |
20
(8) |
21
(7) |
22
|
23
|
|
24
(5) |
25
(14) |
26
(8) |
27
(18) |
28
(11) |
29
(15) |
|
|
From: Patrick O. <pat...@in...> - 2008-02-21 08:20:09
|
Hello all, I hesitated to write this email because it could be seen as a plug for a commercial product and I wasn't sure whether that would be welcome here. I promise to keep the advertising to a mininum... For the purpose of full disclosure: I used to be the main developer of the MPI correctness checking feature in the Intel(R) Trace Analyzer and Collector, which works with Intel(R) MPI. Since 7.1 distributed memory checking with Valgrind is also supported. Julian might remember that in the distant past I confirmed that the vget/set functionality would be important and should be added again; he was so kind to add it again in the 3.x releases. This is what the distributed memory checking uses today: it takes the definedness information on the sender, transmits it on a side channel, then on the receiver restores the definedness information. This way false positives about sending partially initialized data which is not used at the recipient is avoided. It also avoids false negatives at the recipient where it incorrectly assumes that all received data is initialized. The drawback of course is delayed error reporting: a policy of "all outgoing data must be initialized" is easier to enforce and check. One other use of Valgrind is to make memory inaccessible while it is owned semantically by MPI. That way incorrect accesses by the application to that memory is flagged immediately by Valgrind. I have a white paper lying around which describes these features in more detail, but it hasn't been published outside of Intel. If there is interest in it I could try to get it released. On Wed, 2008-02-20 at 13:43 +0100, Julian Seward wrote: > MPI is a difficult (but not impossible) target for Memcheck. > The key reasons are > > (a) MPI libraries often replace malloc/free with their own > allocator. This seriously reduces Memcheck's ability to > find problems, and it also causes many false positives. I don't think this is a problem with Intel MPI; if there is, the Intel MPI team is in the ideal position to fix it, because nowadays they are responsible for both products. > (b) MPI libraries usually do DMA direct from card to userspace. > Again, Memcheck cannot see this happening, and you get many > false errors and missed errors. With the approach that I described above this is not an issue because before the application gets access to that memory the definedness information is updated. The MPI's internal access to it might trigger reports; these are suppressed. > All that said, developers of these various MPI libraries can surely > give you better answers than those above. It may be worth asking > them directly. I took the first sentence as an invitation to speak up here at all, but further inquiries regarding the products that I mentioned are indeed better directed towards the normal Intel support channels. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. The email footer below is automatically added to comply with company policy; for this email the "intended recipient(s)" are all human and non-human inhabitants of planet Earth. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. |
|
From: Nicholas N. <nj...@cs...> - 2008-02-21 06:05:15
|
On Wed, 20 Feb 2008, Sanjay H A wrote: > I am new to Valgrind. > I want to obtain Basic Block Trace of the application. > How to do that ( I installed Valgrind on x86_64 platform). > Can anyone point me to the document which says how to do that. valgrind --tool=lackey --trace-superblocks=yes <your-program> > Also I want to instrument some statements to application. will valgrind > comes handy for that? Most likely. Nick |
|
From: Sanjay H A <san...@ly...> - 2008-02-21 02:10:53
|
Hi All <br>I am new to Valgrind. <br>I want to obtain Basic Block Trace of the application. <br>How to do that ( I installed Valgrind on x86_64 platform). <br>Can anyone point me to the document which says how to do that. <br> <br>Also I want to instrument some statements to application. will valgrind comes handy for that? <br> <br>Thanx in advance <br>Sanjay |
|
From: rich.fromm <nos...@ya...> - 2008-02-20 23:38:46
|
rich.fromm wrote:
>
> No problems are being reported, but I suspect that the problem is that the
> code that I care about isn't actually being instrumented at all.
>
> As other *.so files are read, there are messages output like:
>
> Reading syms from foo.so
> Reading debug info from bar.so
>
> But I never see any such message for the *.so containing the JNI code.
>
> ...
>
> I tried adding "--trace-syscalls=yes" to my execution line, and I don't
> see
> any evidence of the *.so file containing the JNI code being opened.
>
I have found out how to at least allegedly get valgrind to profile my *.so,
by
setting the LD_PRELOAD environment variable to contain the name of the file.
Then I see the message:
Reading syms from <filename>
and if I run with --trace-syscalls=yes I see a sys_open() call on the file.
valgrind still isn't reporting any leaks in my code, so I still need to
explore further, but I thought I'd share this datapoint in case it benefits
anyone else.
LD_PRELOAD is documented in the LD.SO(8) man page:
LD_PRELOAD
A whitespace-separated list of additional, user-specified, ELF
shared libraries to be loaded before all others. This can be
used to selectively override functions in other shared
libraries. For setuid/setgid ELF binaries, only libraries in
the standard search directories that are also setgid will be
loaded.
- Rich
--
View this message in context: http://www.nabble.com/memcheck-of-JNI-library-loaded-via-System.loadLibrary%28%29-tp15582655p15601589.html
Sent from the Valgrind - Users mailing list archive at Nabble.com.
|
|
From: Julian S. <js...@ac...> - 2008-02-20 21:52:25
|
On Wednesday 20 February 2008 22:16, Nicholas Nethercote wrote: > On Wed, 20 Feb 2008, Julian Seward wrote: > > (b) MPI libraries usually do DMA direct from card to userspace. > > Again, Memcheck cannot see this happening, and you get many > > false errors and missed errors. > > Does the --ignore-ranges option help with that? Yes; and indeed this is the exact reason for its existence, so you can tell memcheck the address ranges where the card is mapped, and it will effectively ignore those (ignore all writes, and assume all reads return initialised data). But it's definitely a method of last resort: * you have to somehow figure out where the card is mapped, and pray that it doesn't get mapped at different locations in different runs. In some cases I've seen, more than one address range are involved; this just makes it more difficult. * it's indiscrimate. Presumably some parts of the address range are the actual end-user I/O buffers, and other parts are for controlling the card. Inside such an area, Memcheck can't check anything - if you are overrunning the user data areas, reading data from the user data area that wasn't sent to the card (so you're really reading uninitialised values, etc) Far better for the MPI library to communicate via standard syscalls (over TCP or UDP) and in short be an absolutely standard userspace process. At least for debugging runs, since that has a bad effect on performance for production runs. I believe the flags I specified for OpenMPI do achieve that; and using them it seemed easy to get reasonable results (essentially zero false errors) with little effort. Throw in the MPI wrapper library and you additionally get MPI level semantic checks on the library calls. J |
|
From: Nicholas N. <nj...@cs...> - 2008-02-20 21:16:01
|
On Wed, 20 Feb 2008, Julian Seward wrote: > (b) MPI libraries usually do DMA direct from card to userspace. > Again, Memcheck cannot see this happening, and you get many > false errors and missed errors. Does the --ignore-ranges option help with that? N |
|
From: Julian S. <js...@ac...> - 2008-02-20 12:46:44
|
(this also partially addresses Robert Anderson's earlier query) (If you haven't already, read the section in the Memcheck manual on MPI http://www.valgrind.org/docs/manual/mc-manual.html#mc-manual.mpiwrap) MPI is a difficult (but not impossible) target for Memcheck. The key reasons are (a) MPI libraries often replace malloc/free with their own allocator. This seriously reduces Memcheck's ability to find problems, and it also causes many false positives. (b) MPI libraries usually do DMA direct from card to userspace. Again, Memcheck cannot see this happening, and you get many false errors and missed errors. The most effective approach seems to be to adjust your MPI arrangements to avoid these problems, even if this loses some performance. At the moment it appears that the state of the art for MPI libraries does not simultaneously provide maximum performance and adequate visibility for Memcheck to work well. I have had some success (in my very limited testing) with OpenMPI. It has flags which avoid (b) (and possibly (a), since malloc/free replacements are usually done to support (b)). At least when I tried OpenMPI a couple of years ago, the following helped: mpirun --mca btl tcp,self -np a.out which (according to an OpenMPI developer), "disables shared memory by telling Open MPI to only use loopback and tcp". Note that you should ask for tcp between all MPI processes, even those running on the same core, since Valgrind does not handle synchronisation through shared memory properly. Other people have had varying degrees of success using Memcheck with direct-to-userspace card transfers (b), although that generally makes life more difficult and requires suppressing large numbers of false errors to make it usable. I have heard before about (a) w.r.t. MPICH2 and it strikes me as essentially unsolvable; the only (effective) fix (that I know of) is to ask the library not to supply its own allocator. All that said, developers of these various MPI libraries can surely give you better answers than those above. It may be worth asking them directly. J On Wednesday 20 February 2008 12:49, Joan wrote: > Hi everybody > > I've been using valgrind for debugging some application that I've wrote... > Now I'm programming with MPICH 1.0 environment, distribute memory. > I'd like to know how can I do for debugging my application with valgrind > under that situation. I've been looking for on the net but what I've found > ( http://www.hlrs.de/people/keller/MPI/mpich_valgrind.html) has not been > useful. > > Could somebody explain me how to do it? > > Thank you very much |
|
From: Ashley P. <api...@co...> - 2008-02-20 12:37:41
|
On Wed, 2008-02-20 at 13:19 +0100, Joan wrote: > Hi Ashley, > > In fact I'm using mpirun for starting my applicattion, but it doesn't > work. I don't know what exactly could be. It looks to be working to me, you haven't compiled your application with -g which would allow valgrind to resolve file/line numbers but it is running your progam under valgrind. What was the full command line you used, you aren't getting complete stack traces, it looks like you are using the --num-callers=1 option. For parallel environments I'd recommend using the --log-file-qualifier or --log-file option depending on the version of valgrind you are using, this will give you one output file per rank which is easier to deal with. Ashley, > I'm checking it with the source code that appears in > http://www.hlrs.de/people/keller/MPI/mpich_valgrind.html and that's > what I get from valgrind: > > ==17677== Memcheck, a memory error detector. > ==17677== Copyright (C) 2002-2006, and GNU GPL'd, by Julian > Seward et al. > ==17677== Using LibVEX rev 1658, a library for dynamic binary > translation. > ==17677== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks > LLP. > ==17677== Using valgrind-3.2.1-Debian, a dynamic binary > instrumentation framework. > ==17677== Copyright (C) 2000-2006, and GNU GPL'd, by Julian > Seward et al. > ==17677== For more details, rerun with: -v > ==17677== > ==17677== Invalid read of size 4 > ==17677== at 0x4010DE9: (within /lib/ld-2.3.6.so) > ==17677== Address 0x42BEAD4 is 36 bytes inside a block of > size 37 alloc'd > ==17677== at 0x401D38B: malloc (vg_replace_malloc.c:149) > ==17677== > ==17677== Invalid read of size 4 > ==17677== at 0x4010E17: (within /lib/ld-2.3.6.so) > ==17677== Address 0x4315A9C is 28 bytes inside a block of > size 31 alloc'd > ==17677== at 0x401D38B: malloc (vg_replace_malloc.c:149) > ==17677== > ==17677== Invalid read of size 4 > ==17677== at 0x4010DD3: (within /lib/ld-2.3.6.so) > ==17677== Address 0x4316118 is 32 bytes inside a block of > size 35 alloc'd > ==17677== at 0x401D38B: malloc (vg_replace_malloc.c:149) > ==17677== > ==17677== Syscall param writev(vector[...]) points to > uninitialised byte(s) > ==17677== at 0x4000792: (within /lib/ld-2.3.6.so) > ==17677== Address 0xBECC8568 is on thread 1's stack > ==17677== > ==17677== Invalid write of size 1 > ==17677== at 0x80AD60F: MPID_CH_Eagerb_recv_short > (in /home/aprat/provaValgrind/prova) > ==17677== Address 0x4323290 is 0 bytes after a block of size > 40 alloc'd > ==17677== at 0x401D38B: malloc (vg_replace_malloc.c:149) > ==17678== > ==17678== ERROR SUMMARY: 4 errors from 3 contexts (suppressed: > 37 from 1) > ==17678== malloc/free: in use at exit: 549,776 bytes in 30 > blocks. > ==17678== malloc/free: 97 allocs, 67 frees, 559,818 bytes > allocated. > ==17678== For counts of detected errors, rerun with: -v > ==17678== searching for pointers to 30 not-freed blocks. > ==17678== checked 179,208 bytes. > ==17678== > ==17678== LEAK SUMMARY: > ==17678== definitely lost: 156 bytes in 11 blocks. > ==17678== possibly lost: 0 bytes in 0 blocks. > ==17678== still reachable: 549,620 bytes in 19 blocks. > ==17678== suppressed: 0 bytes in 0 blocks. > ==17678== Use --leak-check=full to see details of leaked > memory. > ==17677== > ==17677== ERROR SUMMARY: 14 errors from 5 contexts > (suppressed: 42 from 2) > ==17677== malloc/free: in use at exit: 3,044 bytes in 16 > blocks. > ==17677== malloc/free: 172 allocs, 156 frees, 606,678 bytes > allocated. > ==17677== For counts of detected errors, rerun with: -v > ==17677== searching for pointers to 16 not-freed blocks. > ==17677== checked 174,272 bytes. > ==17677== > ==17677== LEAK SUMMARY: > ==17677== definitely lost: 204 bytes in 14 blocks. > ==17677== possibly lost: 0 bytes in 0 blocks. > ==17677== still reachable: 2,840 bytes in 2 blocks. > ==17677== suppressed: 0 bytes in 0 blocks. > ==17677== Use --leak-check=full to see details of leaked > memory. > > It does not really seems what I should see, doesn't it? |
|
From: Ashley P. <api...@co...> - 2008-02-20 11:59:45
|
On Wed, 2008-02-20 at 12:49 +0100, Joan wrote: > Hi everybody, > > I've been using valgrind for debugging some application that I've > wrote... Now I'm programming with MPICH 1.0 environment, distribute > memory. > I'd like to know how can I do for debugging my application with > valgrind under that situation. I've been looking for on the net but > what I've found > (http://www.hlrs.de/people/keller/MPI/mpich_valgrind.html) has not > been useful. > > Could somebody explain me how to do it? It depends on the parallel environment you are using, i.e. how you start your parallel program. Rainers page used the command below, if you are using the mpirun provided with mpich-1 you could try that, if not let me know how you are starting jobs and I'll see if I can help you. /opt/mpich-1.2.6-gcc-ch_shmem/bin/mpirun -np 2 -dbg=valgrind ./mpi_murks Ashley Pittman. |
|
From: Joan <joa...@gm...> - 2008-02-20 11:49:25
|
Hi everybody, I've been using valgrind for debugging some application that I've wrote... Now I'm programming with MPICH 1.0 environment, distribute memory. I'd like to know how can I do for debugging my application with valgrind under that situation. I've been looking for on the net but what I've found ( http://www.hlrs.de/people/keller/MPI/mpich_valgrind.html) has not been useful. Could somebody explain me how to do it? Thank you very much -- Blog: http://earenjoy.blogspot.com |
|
From: rich.fromm <nos...@ya...> - 2008-02-20 05:21:26
|
I have a Java application that runs code in a C library via JNI. I have reason to believe that the C code is leaking memory, and I would like to use valgrind to instrument it. >From reading the FAQ, and various threads on this e-mail list, it seems like this should be possible in theory. I am prefacing my java execution call with the following arguments: valgrind --tool=memcheck --leak-check=full --smc-check=all -v No problems are being reported, but I suspect that the problem is that the code that I care about isn't actually being instrumented at all. As other *.so files are read, there are messages output like: Reading syms from foo.so Reading debug info from bar.so But I never see any such message for the *.so containing the JNI code. It is loaded via a call to System.loadLibrary() in the static section of a Java class, in case that matters. I tried adding "--trace-children=yes" to my execution line, and now there are all sorts of errors reported in /usr/lib/j2sdk1.5-sun/jre/lib/i386/client/libjvm.so, but I suspect that these are false positives, and just indicative of things that Sun's JVM does that valgrind can't understand. I tried adding "--trace-syscalls=yes" to my execution line, and I don't see any evidence of the *.so file containing the JNI code being opened. I'm using the Sun JVM via the sun-j2sdk1.5 Debian package, version 1.5.0+update06, on debian 3.1 (sarge). The C code including the *.so file was compiled with gcc 3.3.5. Thanks for any help that anyone might be able to provide. - Rich Fromm -- View this message in context: http://www.nabble.com/memcheck-of-JNI-library-loaded-via-System.loadLibrary%28%29-tp15582655p15582655.html Sent from the Valgrind - Users mailing list archive at Nabble.com. |
|
From: Robert W. <rj...@du...> - 2008-02-19 18:11:16
|
> I wanted to know if there are any way to cross-compile valgrind to > 'mipsel-linux' http://www.valgrind.org/info/platforms.html Short answer: no. Long answer: you'd have to port it, which is a very large task. |
|
From: A. E. C. <aec...@gm...> - 2008-02-19 17:16:00
|
Hi all, I wanted to know if there are any way to cross-compile valgrind to 'mipsel-linux' Thanks! -- Andrés Estévez Costas http://ultimastecnoloxias.blogspot.com http://andresestevez.blogspot.com |
|
From: Sébastien S. <sa...@us...> - 2008-02-19 14:00:47
|
Hi Josef, Josef Weidendorfer a écrit : > Ok. From an more abstract view point, "ELF object" and "Source File" are > just different groupings of functions. For the python functions, you chose > to make these groupings equal. And the benefit would be that independent > of whether you want to see the "ELF object" or "Source File" grouping for > your C functions, you simultaneous get the "Python Source" grouping. Yes? Yes, exactly; it is just for practical reasons. It provides a different way of sorting functions. >> I am going to try to explain what I did: >> >> PyEval_EvalFrameEx, PyObject_CallMethod, PyObject_Call >> ... > > OK, now I hope I understand. Great. > Am I right to say that your additions are specific to a given python implementation > (the function names to catch etc.). Or is there some official specification which > says that every implementation of a python interpreter has to call these functions > on the given conditions with exactly this name? > Further, you take a stack value and cast it to some python structure to get the > function/file names. This also seems to be very specific to the given python > implementation, as well as specific to the ABI of the current architecture (e.g. > the patch probably would be different on x86-64). You are right; The additions I did are specific to CPython 2.5.1 and need to be recompiled to match the ABI of the architecture. That being said, the Python C API is rather stable so provided that you recompile it with the right version of headers, the patch should work as is with Python 2.4 or only require some minimal changes (possibily also with 2.3). I didn't have time to improve the patch yet, but I have plenty of ideas to so. regards -- Sébastien Sablé |
|
From: Josef W. <Jos...@gm...> - 2008-02-15 01:35:48
|
On Thursday 14 February 2008, Nicholas Nethercote wrote: > Is there any way to encode this information in a file, and have Callgrind > read that file? For this python extension, Sebastien needs to access the content of some variables on the stack when entering a given function. Potentially without data debug info, how would you do it? We already have a shared lib which is dynamically linked to client code when running a given VG tool. Could this somehow be used for an extension mechanism? E.g. by registering some functions on initialisation? The big problem here of course is that callgrind would like to run such a function on the real processor, and not on the simulated one... Hmm. In this case, it should even work to define function wrappers which execute client requests. Josef > Then it wouldn't require any official notion of tool > extension. > > Nick > |
|
From: Nicholas N. <nj...@cs...> - 2008-02-14 21:14:47
|
On Thu, 14 Feb 2008, Josef Weidendorfer wrote: > Yes, that's reasonable. The first thing would be to decide about the > technical issues allowing for extensions of VG tools. > You want to have access to python structures. So the extension probably > has to be a binary blob callable in a callgrind run. > As VG tools a static binaries, there is no way to dynamically link code > to a tool... Hmm.. I have to think about it (Julian?). Is there any way to encode this information in a file, and have Callgrind read that file? Then it wouldn't require any official notion of tool extension. Nick |
|
From: Robert A. <rw...@gm...> - 2008-02-14 21:08:36
|
I am debugging an MPI program using valgrind, and getting unusable results
(tons of false positives and negatives).
Here's an example. The program:
#include <cstdlib>
using namespace std;
int main()
{
int* a = (int*)malloc(1*sizeof(int));
a[10] = 1;
}
runs clean:
==31559== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 12 from 5)
when compiled using MPI, but correctly reports the invalid write bug when
compiled without MPI libraries.
By looking at the output of -v, I notice that malloc never gets redirected
when compiled with MPI, but it does get redirected when compiled without
MPI.
Without MPI:
% valgrind -v ./a.out 2>&1 | grep malloc | grep REDIR
--2240-- REDIR: 0x510C3F0 (malloc) redirected to 0x4A18D8C (malloc)
With MPI:
% valgrind -v ./a.out 2>&1 | grep malloc | grep REDIR
%
malloc appears to resolve to an implementation in an mpich shared library on
my system. Is there a way to get memcheck to redirect from this malloc to
valgrind's malloc? Is this a bug in valgrind that it doesn't do this
automatically?
Thanks,
Bob
|
|
From: Josef W. <Jos...@gm...> - 2008-02-14 17:48:15
|
[Julian: below there is the question what would be the best solution to make VG tools extendable, where the extension would be provided in a separate package. Any ideas?] Hi Sebastien, On Thursday 14 February 2008, Sébastien Sablé wrote: > With my last patch, the python functions defined in the foo.py script > will be associated to the foo.py source file and foo.py ELF object. Ok. From an more abstract view point, "ELF object" and "Source File" are just different groupings of functions. For the python functions, you chose to make these groupings equal. And the benefit would be that independent of whether you want to see the "ELF object" or "Source File" grouping for your C functions, you simultaneous get the "Python Source" grouping. Yes? > I haven't looked in details yet. However when using > compress-strings=yes, some functions would be reported in the wrong > source file/ELF object. Hmm. It could be that this is not about parsing the file, but because of some optimizations in KCachegrind where I took some semantics as given. > > I have no idea about phython internals, so I can not comment on the usefulness > > of catching these additional functions. > > So you are "just" adding further artificial functions to the callgraph? > > > Oh, then I misunderstood. When is "PyEval_EvalFrameEx" actually called? I thought > > this is a special run mode of Python, which needs modifications on the Python side... > > I am going to try to explain what I did: > > PyEval_EvalFrameEx, PyObject_CallMethod, PyObject_Call > ... OK, now I hope I understand. Am I right to say that your additions are specific to a given python implementation (the function names to catch etc.). Or is there some official specification which says that every implementation of a python interpreter has to call these functions on the given conditions with exactly this name? Further, you take a stack value and cast it to some python structure to get the function/file names. This also seems to be very specific to the given python implementation, as well as specific to the ABI of the current architecture (e.g. the patch probably would be different on x86-64). > > So if I want to try to abstract your needs for python from callgrind by adding > > client requests to define artificial function names for the call graph, this > > would mean an actual change of python, which currently is not needed? > > I don't think any modification should be needed on the application being > analyzed. What would be great, would be if in callgrind each time a C > function call is detected (around bbcc.c:753), this function name (and > its args) could be passed to an extension for analysis. This extension > would dynamically decide if the function should be added to the call > stack as is, or if it should be skipped or renamed. Yes, that's reasonable. The first thing would be to decide about the technical issues allowing for extensions of VG tools. You want to have access to python structures. So the extension probably has to be a binary blob callable in a callgrind run. As VG tools a static binaries, there is no way to dynamically link code to a tool... Hmm.. I have to think about it (Julian?). The other question is who should provide the python specific callgrind extension. As it dependson the given python implementation, I would say that it should be distributed with some python-devel package, using the extension support in callgrind. Josef |
|
From: Sébastien S. <sa...@us...> - 2008-02-14 10:30:21
|
Hi Josef,
Josef Weidendorfer a écrit :
> Sorry, I do not understand this. Why is the "filename" field not enough?
> Or is this a workaround for a usability problem in KCachegrind?
There is a drop box widget in Kcachegrind where you can select how you
want to sort the functions. You can choose between (No Grouping), ELF
object, Source File, ...
The "filename" arg passed to get_fn_node_inseg in callgrind corresponds
to "Source File".
However the ELF object is determined by the obj_node in the SegInfo, and
since I was passing a NULL si, all the python functions were associated
to the "(unknown)" ELF File.
With my last patch, the python functions defined in the foo.py script
will be associated to the foo.py source file and foo.py ELF object.
It provides a different way of sorting which I find useful.
>> For some reason it only works with the
>> --compress-strings=no option.
>
> What it the problem here? The parser in KCachegrind?
I haven't looked in details yet. However when using
compress-strings=yes, some functions would be reported in the wrong
source file/ELF object.
> I have no idea about phython internals, so I can not comment on the usefulness
> of catching these additional functions.
> So you are "just" adding further artificial functions to the callgraph?
> Oh, then I misunderstood. When is "PyEval_EvalFrameEx" actually called? I thought
> this is a special run mode of Python, which needs modifications on the Python side...
I am going to try to explain what I did:
PyEval_EvalFrameEx, PyObject_CallMethod, PyObject_Call... are some C
functions in the Python runtime which are called at a low level to
execute some functions or methods. The Python functions which is
executed by them can be determined by the args passed to those functions.
These functions are called very often so when you run an unmodified
callgrind on a python application, you get cycles made of those
functions and it is difficult to analyze the trace.
What I am doing is detect those C functions thanks to callgrind, then
instead of pushing the name of those functions in the call stack, I get
the name of the python functions which are executed by looking into the
args of the function and I push this information instead in the call stack.
For example PyObject_CallMethod is used in the python runtime to call
the method of a class.
class Foo:
def bar(self):
pass
The following python code
foo.bar()
will be converted at the C Level in:
PyObject_CallMethod(object, "bar")
where object is of type Foo, so that in the C struct object->type->name
is "Foo".
So when callgrind encounters the C function PyObject_CallMethod, I get
the first and second args of this functions, and instead of pushing
PyObject_CallMethod in the call stack, I push the "virtual" function
"Foo.bar" instead.
As a result the trace has much less cycles, and I can actually see which
python functions get called instead of the internal of the python runtime.
So I am not exactly adding some events, but rather renaming them with a
more meaningful name.
I also try to skip some C functions in the python runtime which do not
bring information.
I hope this explanation makes things clearer.
> So if I want to try to abstract your needs for python from callgrind by adding
> client requests to define artificial function names for the call graph, this
> would mean an actual change of python, which currently is not needed?
I don't think any modification should be needed on the application being
analyzed. What would be great, would be if in callgrind each time a C
function call is detected (around bbcc.c:753), this function name (and
its args) could be passed to an extension for analysis. This extension
would dynamically decide if the function should be added to the call
stack as is, or if it should be skipped or renamed.
regards
--
Sébastien Sablé
|
|
From: Josef W. <Jos...@gm...> - 2008-02-13 23:32:01
|
On Wednesday 13 February 2008, Nicholas Nethercote wrote: > On Wed, 13 Feb 2008, Josef Weidendorfer wrote: > > >> A patch in contrib/ would be fine for the moment. But I am sure that a > >> generic way to push higher level functions would interest a lot of > >> people (so that there could be some callgrind plug-ins to profile > >> python2.5, python2.3, php, perl, whatever...). > > > > Yes. > > > > However, there currently does not exist any further extension of VG > > tools to allow for "callgrind plug-ins". But it should be possible to > > make a kind of library version of callgrind, to base other VG tools > > on it. > > > > Or even better: Support it via just an configuration file (or command line > > options). Now that Julian is adding support for reading debug info for data, > > it should be possible to look up arbitrary variables. > > I'd strongly encourage you to find a way, if possible, to make this feature > generic if you plan to include it. Yes, of course. I have not yet decided how to include it if at all. If I add client requests which looks like a general solution (as far as callgrind goes), the needed macros need to be included into python. While this is the better solution, it is not really helpful if such a modified python never finds its way into distributions. > Code that is not normally run (eg. due > to being in contrib/, or only included if certain configure options are > used) will be tested less and be much less robust. This is one of those > things that is really important in practice. Yes. There is probably no big difference if it would be in contrib/ or if Sebastian maintains it itself, or if the patch finds its way into contrib/ of python source: It won't be run normally, as you said. > If the feature can be extended to other languages than Python that obviously > makes it much better too. IMHO there is a lot of potential for it: a JITter could notify about the source relation of generated code. Probably even useful for self-hosting Valgrind runs (making it even slower ;-) Josef |
|
From: Nicholas N. <nj...@cs...> - 2008-02-13 22:56:28
|
On Wed, 13 Feb 2008, Josef Weidendorfer wrote: >> A patch in contrib/ would be fine for the moment. But I am sure that a >> generic way to push higher level functions would interest a lot of >> people (so that there could be some callgrind plug-ins to profile >> python2.5, python2.3, php, perl, whatever...). > > Yes. > > However, there currently does not exist any further extension of VG > tools to allow for "callgrind plug-ins". But it should be possible to > make a kind of library version of callgrind, to base other VG tools > on it. > > Or even better: Support it via just an configuration file (or command line > options). Now that Julian is adding support for reading debug info for data, > it should be possible to look up arbitrary variables. I'd strongly encourage you to find a way, if possible, to make this feature generic if you plan to include it. Code that is not normally run (eg. due to being in contrib/, or only included if certain configure options are used) will be tested less and be much less robust. This is one of those things that is really important in practice. If the feature can be extended to other languages than Python that obviously makes it much better too. Nick |
|
From: Josef W. <Jos...@gm...> - 2008-02-13 19:37:23
|
Hi Sébastien, On Wednesday 13 February 2008, Sébastien Sablé wrote: > I improved my patch so that python calls can now be sorted by "ELF > object" in Kcachegrind, each "ELF object" representing a different > python script. Sorry, I do not understand this. Why is the "filename" field not enough? Or is this a workaround for a usability problem in KCachegrind? > For some reason it only works with the > --compress-strings=no option. What it the problem here? The parser in KCachegrind? > I also found that I could get much more information about the Python > context when calling Python C/API. This is useful for example to profile > Python extension written in C or Pyrex. (this is done by instrumenting > additional functions like PyObject_Call, PyObject_GetAttr...). > It has the extra advantage of breaking many cycles, so that the trace is > much easier to read in Kcachegrind. I have no idea about phython internals, so I can not comment on the usefulness of catching these additional functions. So you are "just" adding further artificial functions to the callgraph? > There is probably a lot more that can be done, and I will complete this > patch in coming days. I intend to post an announce about it on > python-users and pyrex mailing list once a more complete patch is > available in order to get feedback from Python users. > > Josef Weidendorfer a écrit : > > But to be true, the current solution looks quite "hacky" to me. To use it, > > one has to modify things in the python interpreter, and there is > > this special case for python in callgrind. Up to now, callgrind is independent > > from the language used, as long as there exists debug information for the binary. > > This patch does not need any modification on the Python side. Oh, then I misunderstood. When is "PyEval_EvalFrameEx" actually called? I thought this is a special run mode of Python, which needs modifications on the Python side... > You can > run callgrind with this patch on the standard Python interpreter or any > C application linked with the Python libraries and it should work. > However you are right that it is "hacky" in that it brings some code > specific to Python in callgrind and some dependencies on the Python headers. So if I want to try to abstract your needs for python from callgrind by adding client requests to define artificial function names for the call graph, this would mean an actual change of python, which currently is not needed? > A patch in contrib/ would be fine for the moment. But I am sure that a > generic way to push higher level functions would interest a lot of > people (so that there could be some callgrind plug-ins to profile > python2.5, python2.3, php, perl, whatever...). Yes. However, there currently does not exist any further extension of VG tools to allow for "callgrind plug-ins". But it should be possible to make a kind of library version of callgrind, to base other VG tools on it. Or even better: Support it via just an configuration file (or command line options). Now that Julian is adding support for reading debug info for data, it should be possible to look up arbitrary variables. Josef > > regards > > -- > Sébastien Sablé > |
|
From: Sébastien S. <sa...@us...> - 2008-02-13 17:35:34
|
Hi Josef, I improved my patch so that python calls can now be sorted by "ELF object" in Kcachegrind, each "ELF object" representing a different python script. For some reason it only works with the --compress-strings=no option. I also found that I could get much more information about the Python context when calling Python C/API. This is useful for example to profile Python extension written in C or Pyrex. (this is done by instrumenting additional functions like PyObject_Call, PyObject_GetAttr...). It has the extra advantage of breaking many cycles, so that the trace is much easier to read in Kcachegrind. There is probably a lot more that can be done, and I will complete this patch in coming days. I intend to post an announce about it on python-users and pyrex mailing list once a more complete patch is available in order to get feedback from Python users. Josef Weidendorfer a écrit : > But to be true, the current solution looks quite "hacky" to me. To use it, > one has to modify things in the python interpreter, and there is > this special case for python in callgrind. Up to now, callgrind is independent > from the language used, as long as there exists debug information for the binary. This patch does not need any modification on the Python side. You can run callgrind with this patch on the standard Python interpreter or any C application linked with the Python libraries and it should work. However you are right that it is "hacky" in that it brings some code specific to Python in callgrind and some dependencies on the Python headers. > So, one way would be to add the patch as user contribution into a contrib/ directory. > And if there is enough pressure for a distribution, it could provide the patched callgrind. > > Or I try to refactor most parts of callgrind to make it available as modules for other > tools. And callgrind itself would shrink dramatically in code size. Not really realistic. > > Another way would be to try to achieve the same/similar result with a more general > solution not specific for python, but for any type of interpreter. > E.g. using client requests, which allow to specify function calls/returns > happening in a high-level call-graph. I suppose that your PyEval_EvalFrameEx > function would be the ideal place to notify callgrind about these events. > Hmm... this would add another dimension to the code position such as > (high-level function, low-level function), and thus to some "explosion" of positions. > In fact, it is quite similar to another dimension I wanted to add since quite some time: > accessed data. I just wonder if there is an easy extension of the format and > visualization, or if it is not really worth it... A patch in contrib/ would be fine for the moment. But I am sure that a generic way to push higher level functions would interest a lot of people (so that there could be some callgrind plug-ins to profile python2.5, python2.3, php, perl, whatever...). regards -- Sébastien Sablé |
|
From: Karl K. <mai...@bl...> - 2008-02-12 17:41:31
|
Hello!Today I tried to execute callgrind but it dies with an assertion (Assertion 'sigNum == vgCallgrind_current_state.sig' failed). I have put debug statements to the source (exactly into src/threads.c - which is attached). It produces a periodic output until line threads.c:236 is executed. A view moments after this line is executed the program stops.I am working on SuSE 10 with - valgrind 3.0.0 (src) and callgrind 0.10.1 (src) and have also tried - valgrind 3.3.0 (rpm & src)both with the same result.The modified threads.c is attached, also the valgrind log file. The file debug_trace.log is the program output.Additional info: valgrind with memcheck and cachegrind works without problems.Thanks in advance,Karl |
|
From: Nicholas N. <nj...@cs...> - 2008-02-11 21:12:28
|
On Mon, 11 Feb 2008, Josef Weidendorfer wrote: > But to be true, the current solution looks quite "hacky" to me. To use it, > one has to modify things in the python interpreter, and there is this > special case for python in callgrind. Up to now, callgrind is independent > from the language used, as long as there exists debug information for the > binary. I agree. We also try to avoid conditional compilation, and dependencies on external software, including headers. But it is useful to mixed Python/C programmers, so it would be nice if there is a cleaner way to do it. N |