|
From: Ahmad N. <ahm...@gm...> - 2019-02-08 15:33:14
|
Hi, I wrote a really simple Pin tool to calculate the number of dynamically allocated bytes in a program. I instrumented GIMP with this tool and it reported 77 MB of allocations. I did the same experiment with Valgrind which reported 117 MB. My Pin tool is similar to the example in Pin. It searches for malloc(), calloc() and memalign() in each loaded image and adds instructions before them to calculate the total size of the allocations. I am really confused and need help! Regards. |
|
From: David F. <fa...@kd...> - 2019-02-08 16:56:08
|
On vendredi 8 février 2019 16:32:50 CET Ahmad Nouralizadeh wrote: > Hi, > I wrote a really simple Pin tool to calculate the number of dynamically > allocated bytes in a program. I instrumented GIMP with this tool and it > reported 77 MB of allocations. I did the same experiment with Valgrind > which reported 117 MB. > My Pin tool is similar to the example in Pin. It searches for malloc(), > calloc() and memalign() in each loaded image and adds instructions before > them to calculate the total size of the allocations. > I am really confused and need help! If you're on Linux, I recommend using heaptrack for this :-) https://github.com/KDAB/heaptrack This doesn't really answer your question, sorry about that, but you might want to see which of those tools heaptrack agrees with, it might help finding out who is wrong... -- David Faure, fa...@kd..., http://www.davidfaure.fr Working on KDE Frameworks 5 |
|
From: Ahmad N. <ahm...@gm...> - 2019-02-08 17:32:24
|
Thanks David, But heaptrack even reports a larger number: 153 MB! On Fri, Feb 8, 2019 at 8:09 PM David Faure <fa...@kd...> wrote: > On vendredi 8 février 2019 16:32:50 CET Ahmad Nouralizadeh wrote: > > Hi, > > I wrote a really simple Pin tool to calculate the number of dynamically > > allocated bytes in a program. I instrumented GIMP with this tool and it > > reported 77 MB of allocations. I did the same experiment with Valgrind > > which reported 117 MB. > > My Pin tool is similar to the example in Pin. It searches for malloc(), > > calloc() and memalign() in each loaded image and adds instructions before > > them to calculate the total size of the allocations. > > I am really confused and need help! > > If you're on Linux, I recommend using heaptrack for this :-) > https://github.com/KDAB/heaptrack > > This doesn't really answer your question, sorry about that, but you might > want > to see which of those tools heaptrack agrees with, it might help finding > out > who is wrong... > > -- > David Faure, fa...@kd..., http://www.davidfaure.fr > Working on KDE Frameworks 5 > > > > |
|
From: David F. <fa...@kd...> - 2019-02-08 18:45:09
|
LOL that was the risk, getting a third, completely different, number ;) Well, you mention that your tool only looks at "each loaded image", while heaptrack and valgrind look at ALL allocations. On vendredi 8 février 2019 18:32:01 CET Ahmad Nouralizadeh wrote: > Thanks David, > But heaptrack even reports a larger number: 153 MB! > > On Fri, Feb 8, 2019 at 8:09 PM David Faure <fa...@kd...> wrote: > > On vendredi 8 février 2019 16:32:50 CET Ahmad Nouralizadeh wrote: > > > Hi, > > > I wrote a really simple Pin tool to calculate the number of dynamically > > > allocated bytes in a program. I instrumented GIMP with this tool and it > > > reported 77 MB of allocations. I did the same experiment with Valgrind > > > which reported 117 MB. > > > My Pin tool is similar to the example in Pin. It searches for malloc(), > > > calloc() and memalign() in each loaded image and adds instructions > > > before > > > them to calculate the total size of the allocations. > > > I am really confused and need help! > > > > If you're on Linux, I recommend using heaptrack for this :-) > > https://github.com/KDAB/heaptrack > > > > This doesn't really answer your question, sorry about that, but you might > > want > > to see which of those tools heaptrack agrees with, it might help finding > > out > > who is wrong... > > > > -- > > David Faure, fa...@kd..., http://www.davidfaure.fr > > Working on KDE Frameworks 5 -- David Faure, fa...@kd..., http://www.davidfaure.fr Working on KDE Frameworks 5 |
|
From: Ahmad N. <ahm...@gm...> - 2019-02-08 18:58:48
|
By image, I mean the binary code of the program to be traced and all the shared libraries accessed by that program. As soon as they are loaded, they will be searched for calls to malloc,... and some code will be added before and after each call. The code is used to store stats, such as the allocation size. How is it possible to miss an allocation? Every possible malloc,... call point is covered. On Fri, Feb 8, 2019 at 10:14 PM David Faure <fa...@kd...> wrote: > LOL that was the risk, getting a third, completely different, number ;) > > Well, you mention that your tool only looks at "each loaded image", > while heaptrack and valgrind look at ALL allocations. > > > On vendredi 8 février 2019 18:32:01 CET Ahmad Nouralizadeh wrote: > > Thanks David, > > But heaptrack even reports a larger number: 153 MB! > > > > On Fri, Feb 8, 2019 at 8:09 PM David Faure <fa...@kd...> wrote: > > > On vendredi 8 février 2019 16:32:50 CET Ahmad Nouralizadeh wrote: > > > > Hi, > > > > I wrote a really simple Pin tool to calculate the number of > dynamically > > > > allocated bytes in a program. I instrumented GIMP with this tool and > it > > > > reported 77 MB of allocations. I did the same experiment with > Valgrind > > > > which reported 117 MB. > > > > My Pin tool is similar to the example in Pin. It searches for > malloc(), > > > > calloc() and memalign() in each loaded image and adds instructions > > > > before > > > > them to calculate the total size of the allocations. > > > > I am really confused and need help! > > > > > > If you're on Linux, I recommend using heaptrack for this :-) > > > https://github.com/KDAB/heaptrack > > > > > > This doesn't really answer your question, sorry about that, but you > might > > > want > > > to see which of those tools heaptrack agrees with, it might help > finding > > > out > > > who is wrong... > > > > > > -- > > > David Faure, fa...@kd..., http://www.davidfaure.fr > > > Working on KDE Frameworks 5 > > > -- > David Faure, fa...@kd..., http://www.davidfaure.fr > Working on KDE Frameworks 5 > > > > |
|
From: John R. <jr...@bi...> - 2019-02-08 21:04:06
|
On 2/8/19 10:58 AM, Ahmad Nouralizadeh wrote: > By image, I mean the binary code of the program to be traced and all the shared libraries accessed by that program. As soon as they are loaded, they will be searched for calls to malloc,... and some code will be added before and after each call It would be more robust to re-write the first several instructions of malloc itself, instead of trying to find all the calls. In particular, a tail-merged call that jumps to the "call malloc@PLT" instruction might not be recognized by the insertion of code "before and after each call to malloc". Also, malloc() can be called through a pointer: void *(*f)(size_t) = malloc; char *x = f(10); and the "before ... and after" recognizer probably will miss some of those. ld-linux (the PT_INTERP) might have its own malloc, separate from libc.so.6. In some (but not all) cases mmap(0,size,prot,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) should be considered an allocation. Do all the measurement programs agree? |
|
From: Tom H. <to...@co...> - 2019-02-08 19:01:47
|
Right but how does your program get control? Does it manage to see all the allocations done by the dynamic linker before main is entered? Tom On 08/02/2019 18:58, Ahmad Nouralizadeh wrote: > By image, I mean the binary code of the program to be traced and all the > shared libraries accessed by that program. As soon as they are loaded, > they will be searched for calls to malloc,... and some code will be > added before and after each call. The code is used to store stats, such > as the allocation size. How is it possible to miss an allocation? Every > possible malloc,... call point is covered. > > On Fri, Feb 8, 2019 at 10:14 PM David Faure <fa...@kd... > <mailto:fa...@kd...>> wrote: > > LOL that was the risk, getting a third, completely different, number ;) > > Well, you mention that your tool only looks at "each loaded image", > while heaptrack and valgrind look at ALL allocations. > > > On vendredi 8 février 2019 18:32:01 CET Ahmad Nouralizadeh wrote: > > Thanks David, > > But heaptrack even reports a larger number: 153 MB! > > > > On Fri, Feb 8, 2019 at 8:09 PM David Faure <fa...@kd... > <mailto:fa...@kd...>> wrote: > > > On vendredi 8 février 2019 16:32:50 CET Ahmad Nouralizadeh wrote: > > > > Hi, > > > > I wrote a really simple Pin tool to calculate the number of > dynamically > > > > allocated bytes in a program. I instrumented GIMP with this > tool and it > > > > reported 77 MB of allocations. I did the same experiment with > Valgrind > > > > which reported 117 MB. > > > > My Pin tool is similar to the example in Pin. It searches for > malloc(), > > > > calloc() and memalign() in each loaded image and adds > instructions > > > > before > > > > them to calculate the total size of the allocations. > > > > I am really confused and need help! > > > > > > If you're on Linux, I recommend using heaptrack for this :-) > > > https://github.com/KDAB/heaptrack > > > > > > This doesn't really answer your question, sorry about that, but > you might > > > want > > > to see which of those tools heaptrack agrees with, it might > help finding > > > out > > > who is wrong... > > > > > > -- > > > David Faure, fa...@kd... <mailto:fa...@kd...>, > http://www.davidfaure.fr > > > Working on KDE Frameworks 5 > > > -- > David Faure, fa...@kd... <mailto:fa...@kd...>, > http://www.davidfaure.fr > Working on KDE Frameworks 5 > > > > > > _______________________________________________ > Valgrind-users mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-users > -- Tom Hughes (to...@co...) http://compton.nu/ |
|
From: Ahmad N. <ahm...@gm...> - 2019-02-08 19:27:41
|
Hi Tom, Yes. ld.so is also an image that is looked for malloc calls. On Fri, Feb 8, 2019 at 10:31 PM Tom Hughes <to...@co...> wrote: > Right but how does your program get control? Does it manage to see > all the allocations done by the dynamic linker before main is entered? > > Tom > > On 08/02/2019 18:58, Ahmad Nouralizadeh wrote: > > By image, I mean the binary code of the program to be traced and all the > > shared libraries accessed by that program. As soon as they are loaded, > > they will be searched for calls to malloc,... and some code will be > > added before and after each call. The code is used to store stats, such > > as the allocation size. How is it possible to miss an allocation? Every > > possible malloc,... call point is covered. > > > > On Fri, Feb 8, 2019 at 10:14 PM David Faure <fa...@kd... > > <mailto:fa...@kd...>> wrote: > > > > LOL that was the risk, getting a third, completely different, number > ;) > > > > Well, you mention that your tool only looks at "each loaded image", > > while heaptrack and valgrind look at ALL allocations. > > > > > > On vendredi 8 février 2019 18:32:01 CET Ahmad Nouralizadeh wrote: > > > Thanks David, > > > But heaptrack even reports a larger number: 153 MB! > > > > > > On Fri, Feb 8, 2019 at 8:09 PM David Faure <fa...@kd... > > <mailto:fa...@kd...>> wrote: > > > > On vendredi 8 février 2019 16:32:50 CET Ahmad Nouralizadeh > wrote: > > > > > Hi, > > > > > I wrote a really simple Pin tool to calculate the number of > > dynamically > > > > > allocated bytes in a program. I instrumented GIMP with this > > tool and it > > > > > reported 77 MB of allocations. I did the same experiment with > > Valgrind > > > > > which reported 117 MB. > > > > > My Pin tool is similar to the example in Pin. It searches for > > malloc(), > > > > > calloc() and memalign() in each loaded image and adds > > instructions > > > > > before > > > > > them to calculate the total size of the allocations. > > > > > I am really confused and need help! > > > > > > > > If you're on Linux, I recommend using heaptrack for this :-) > > > > https://github.com/KDAB/heaptrack > > > > > > > > This doesn't really answer your question, sorry about that, but > > you might > > > > want > > > > to see which of those tools heaptrack agrees with, it might > > help finding > > > > out > > > > who is wrong... > > > > > > > > -- > > > > David Faure, fa...@kd... <mailto:fa...@kd...>, > > http://www.davidfaure.fr > > > > Working on KDE Frameworks 5 > > > > > > -- > > David Faure, fa...@kd... <mailto:fa...@kd...>, > > http://www.davidfaure.fr > > Working on KDE Frameworks 5 > > > > > > > > > > > > _______________________________________________ > > Valgrind-users mailing list > > Val...@li... > > https://lists.sourceforge.net/lists/listinfo/valgrind-users > > > > > -- > Tom Hughes (to...@co...) > http://compton.nu/ > |
|
From: Philippe W. <phi...@sk...> - 2019-02-09 10:08:36
|
On Fri, 2019-02-08 at 21:02 +0330, Ahmad Nouralizadeh wrote: > Thanks David, > But heaptrack even reports a larger number: 153 MB! > > On Fri, Feb 8, 2019 at 8:09 PM David Faure <fa...@kd...> wrote: > > On vendredi 8 février 2019 16:32:50 CET Ahmad Nouralizadeh wrote: > > > Hi, > > > I wrote a really simple Pin tool to calculate the number of dynamically > > > allocated bytes in a program. I instrumented GIMP with this tool and it > > > reported 77 MB of allocations. I did the same experiment with Valgrind > > > which reported 117 MB. > > > My Pin tool is similar to the example in Pin. It searches for malloc(), > > > calloc() and memalign() in each loaded image and adds instructions before > > > them to calculate the total size of the allocations. > > > I am really confused and need help! What I suggest is to try with much smaller executables and investigate when you see a difference. For example, do the following in a valgrind build: valgrind --xtree-memory=full ./memcheck/tests/trivialleak This will output the total allocations observed by valgrind. It will also produce a file xtmemory.kcg.<PID> that gives the detailed information about the malloc/free calls (to visualise with kcachegrind). Compare this with what is given by the 2 other measurements. Try with somewhat more complex programs if you see no difference, till you find a difference with (let's hope) something simple enough that you can understand where the difference is coming from. With valgrind, you can also trace all the malloc/free calls using --trace-malloc=yes. Philippe |
|
From: Ahmad N. <ahm...@gm...> - 2019-02-10 11:24:55
|
Thanks, everybody. I will try your suggestions. On Saturday, February 9, 2019, Philippe Waroquiers < phi...@sk...> wrote: > On Fri, 2019-02-08 at 21:02 +0330, Ahmad Nouralizadeh wrote: > > Thanks David, > > But heaptrack even reports a larger number: 153 MB! > > > > On Fri, Feb 8, 2019 at 8:09 PM David Faure <fa...@kd...> wrote: > > > On vendredi 8 février 2019 16:32:50 CET Ahmad Nouralizadeh wrote: > > > > Hi, > > > > I wrote a really simple Pin tool to calculate the number of > dynamically > > > > allocated bytes in a program. I instrumented GIMP with this tool and > it > > > > reported 77 MB of allocations. I did the same experiment with > Valgrind > > > > which reported 117 MB. > > > > My Pin tool is similar to the example in Pin. It searches for > malloc(), > > > > calloc() and memalign() in each loaded image and adds instructions > before > > > > them to calculate the total size of the allocations. > > > > I am really confused and need help! > > What I suggest is to try with much smaller executables and investigate > when you see a difference. > > For example, do the following in a valgrind build: > valgrind --xtree-memory=full ./memcheck/tests/trivialleak > This will output the total allocations observed by valgrind. > It will also produce a file xtmemory.kcg.<PID> > that gives the detailed information about the malloc/free calls > (to visualise with kcachegrind). > > Compare this with what is given by the 2 other measurements. > Try with somewhat more complex programs if you see no difference, > till you find a difference with (let's hope) something simple > enough that you can understand where the difference is coming from. > > With valgrind, you can also trace all the malloc/free calls > using --trace-malloc=yes. > > Philippe > > > |