You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(122) |
Nov
(152) |
Dec
(69) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(6) |
Feb
(25) |
Mar
(73) |
Apr
(82) |
May
(24) |
Jun
(25) |
Jul
(10) |
Aug
(11) |
Sep
(10) |
Oct
(54) |
Nov
(203) |
Dec
(182) |
| 2004 |
Jan
(307) |
Feb
(305) |
Mar
(430) |
Apr
(312) |
May
(187) |
Jun
(342) |
Jul
(487) |
Aug
(637) |
Sep
(336) |
Oct
(373) |
Nov
(441) |
Dec
(210) |
| 2005 |
Jan
(385) |
Feb
(480) |
Mar
(636) |
Apr
(544) |
May
(679) |
Jun
(625) |
Jul
(810) |
Aug
(838) |
Sep
(634) |
Oct
(521) |
Nov
(965) |
Dec
(543) |
| 2006 |
Jan
(494) |
Feb
(431) |
Mar
(546) |
Apr
(411) |
May
(406) |
Jun
(322) |
Jul
(256) |
Aug
(401) |
Sep
(345) |
Oct
(542) |
Nov
(308) |
Dec
(481) |
| 2007 |
Jan
(427) |
Feb
(326) |
Mar
(367) |
Apr
(255) |
May
(244) |
Jun
(204) |
Jul
(223) |
Aug
(231) |
Sep
(354) |
Oct
(374) |
Nov
(497) |
Dec
(362) |
| 2008 |
Jan
(322) |
Feb
(482) |
Mar
(658) |
Apr
(422) |
May
(476) |
Jun
(396) |
Jul
(455) |
Aug
(267) |
Sep
(280) |
Oct
(253) |
Nov
(232) |
Dec
(304) |
| 2009 |
Jan
(486) |
Feb
(470) |
Mar
(458) |
Apr
(423) |
May
(696) |
Jun
(461) |
Jul
(551) |
Aug
(575) |
Sep
(134) |
Oct
(110) |
Nov
(157) |
Dec
(102) |
| 2010 |
Jan
(226) |
Feb
(86) |
Mar
(147) |
Apr
(117) |
May
(107) |
Jun
(203) |
Jul
(193) |
Aug
(238) |
Sep
(300) |
Oct
(246) |
Nov
(23) |
Dec
(75) |
| 2011 |
Jan
(133) |
Feb
(195) |
Mar
(315) |
Apr
(200) |
May
(267) |
Jun
(293) |
Jul
(353) |
Aug
(237) |
Sep
(278) |
Oct
(611) |
Nov
(274) |
Dec
(260) |
| 2012 |
Jan
(303) |
Feb
(391) |
Mar
(417) |
Apr
(441) |
May
(488) |
Jun
(655) |
Jul
(590) |
Aug
(610) |
Sep
(526) |
Oct
(478) |
Nov
(359) |
Dec
(372) |
| 2013 |
Jan
(467) |
Feb
(226) |
Mar
(391) |
Apr
(281) |
May
(299) |
Jun
(252) |
Jul
(311) |
Aug
(352) |
Sep
(481) |
Oct
(571) |
Nov
(222) |
Dec
(231) |
| 2014 |
Jan
(185) |
Feb
(329) |
Mar
(245) |
Apr
(238) |
May
(281) |
Jun
(399) |
Jul
(382) |
Aug
(500) |
Sep
(579) |
Oct
(435) |
Nov
(487) |
Dec
(256) |
| 2015 |
Jan
(338) |
Feb
(357) |
Mar
(330) |
Apr
(294) |
May
(191) |
Jun
(108) |
Jul
(142) |
Aug
(261) |
Sep
(190) |
Oct
(54) |
Nov
(83) |
Dec
(22) |
| 2016 |
Jan
(49) |
Feb
(89) |
Mar
(33) |
Apr
(50) |
May
(27) |
Jun
(34) |
Jul
(53) |
Aug
(53) |
Sep
(98) |
Oct
(206) |
Nov
(93) |
Dec
(53) |
| 2017 |
Jan
(65) |
Feb
(82) |
Mar
(102) |
Apr
(86) |
May
(187) |
Jun
(67) |
Jul
(23) |
Aug
(93) |
Sep
(65) |
Oct
(45) |
Nov
(35) |
Dec
(17) |
| 2018 |
Jan
(26) |
Feb
(35) |
Mar
(38) |
Apr
(32) |
May
(8) |
Jun
(43) |
Jul
(27) |
Aug
(30) |
Sep
(43) |
Oct
(42) |
Nov
(38) |
Dec
(67) |
| 2019 |
Jan
(32) |
Feb
(37) |
Mar
(53) |
Apr
(64) |
May
(49) |
Jun
(18) |
Jul
(14) |
Aug
(53) |
Sep
(25) |
Oct
(30) |
Nov
(49) |
Dec
(31) |
| 2020 |
Jan
(87) |
Feb
(45) |
Mar
(37) |
Apr
(51) |
May
(99) |
Jun
(36) |
Jul
(11) |
Aug
(14) |
Sep
(20) |
Oct
(24) |
Nov
(40) |
Dec
(23) |
| 2021 |
Jan
(14) |
Feb
(53) |
Mar
(85) |
Apr
(15) |
May
(19) |
Jun
(3) |
Jul
(14) |
Aug
(1) |
Sep
(57) |
Oct
(73) |
Nov
(56) |
Dec
(22) |
| 2022 |
Jan
(3) |
Feb
(22) |
Mar
(6) |
Apr
(55) |
May
(46) |
Jun
(39) |
Jul
(15) |
Aug
(9) |
Sep
(11) |
Oct
(34) |
Nov
(20) |
Dec
(36) |
| 2023 |
Jan
(79) |
Feb
(41) |
Mar
(99) |
Apr
(169) |
May
(48) |
Jun
(16) |
Jul
(16) |
Aug
(57) |
Sep
(19) |
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
|
|
1
(2) |
2
(4) |
3
(1) |
4
(7) |
5
|
|
6
|
7
(4) |
8
|
9
(3) |
10
(6) |
11
(13) |
12
(6) |
|
13
(1) |
14
|
15
(1) |
16
|
17
(4) |
18
(3) |
19
(5) |
|
20
(5) |
21
(5) |
22
(5) |
23
(6) |
24
|
25
(1) |
26
(1) |
|
27
(1) |
28
(4) |
29
(5) |
30
|
|
|
|
|
From: <sv...@va...> - 2016-11-12 11:22:57
|
Author: philippe
Date: Sat Nov 12 11:22:50 2016
New Revision: 16133
Log:
Outer/inner setup : allow append to outer args, use a bigger --num-callers
* Similarly to what was done in revision r16104 for vg_perf, have a way
to append to predefined vg_regtest outer args, rather than only be able
to replace them.
* increase the num callers for outer tool to 40.
(This is in preparation of a future commit, that will allow outer tool
to append the inner guest stack trace to reported errors in the inner)
Modified:
trunk/perf/vg_perf.in
trunk/tests/vg_regtest.in
Modified: trunk/perf/vg_perf.in
==============================================================================
--- trunk/perf/vg_perf.in (original)
+++ trunk/perf/vg_perf.in Sat Nov 12 11:22:50 2016
@@ -71,8 +71,9 @@
--outer-valgrind: run these Valgrind(s) under the given outer valgrind.
These Valgrind(s) must be configured with --enable-inner.
--outer-tool: tool to use by the outer valgrind (default cachegrind).
- --outer-args: use this as outer tool args. If the outer args are starting with +,
- the given outer args are appended to the outer args predefined by vg_perf.
+ --outer-args: use this as outer tool args. If the outer args are starting
+ with +, the given outer args are appended to the outer args predefined
+ by vg_perf.
Any tools named in --tools must be present in all directories specified
with --vg. (This is not checked.)
Modified: trunk/tests/vg_regtest.in
==============================================================================
--- trunk/tests/vg_regtest.in (original)
+++ trunk/tests/vg_regtest.in Sat Nov 12 11:22:50 2016
@@ -43,7 +43,9 @@
# --outer-valgrind: run this valgrind under the given outer valgrind.
# This valgrind must be configured with --enable-inner.
# --outer-tool: tool to use by the outer valgrind (default memcheck).
-# --outer-args: use this as outer tool args.
+# --outer-args: use this as outer tool args. If the outer args are starting
+# with +, the given outer args are appended to the outer args predefined
+# by vg_regtest.
# --loop-till-fail: loops on the test(s) till one fail, then exit
# This is useful to obtain detailed trace or --keep-unfiltered
# output of a non deterministic test failure
@@ -182,6 +184,7 @@
my $outer_valgrind;
my $outer_tool = "memcheck";
my $outer_args;
+my $run_outer_args = "";
my $valgrind_lib = "$tests_dir/.in_place";
my $keepunfiltered = 0;
@@ -255,8 +258,8 @@
if (defined $outer_valgrind) {
$outer_valgrind = validate_program($tests_dir, $outer_valgrind, 1, 1);
- if (not defined $outer_args) {
- $outer_args =
+ if ((not defined $outer_args) || ($outer_args =~ /^\+/)) {
+ $run_outer_args =
" --command-line-only=yes"
. " --run-libc-freeres=no --sim-hints=enable-outer"
. " --smc-check=all-non-file"
@@ -265,7 +268,16 @@
. " --suppressions="
. validate_program($tests_dir,"./tests/outer_inner.supp",1,0)
. " --memcheck:leak-check=full --memcheck:show-reachable=no"
+ . " --num-callers=40"
. " ";
+ # we use a (relatively) big --num-callers, to allow the outer to report
+ # also the inner guest stack trace, when reporting an error.
+ if (defined $outer_args) {
+ $outer_args =~ s/^\+(.*)/$1/;
+ $run_outer_args = $run_outer_args . $outer_args;
+ }
+ } else {
+ $run_outer_args = $outer_args;
}
}
@@ -505,8 +517,8 @@
mysystem( "$envvars VALGRIND_LIB_INNER=$valgrind_lib "
. "$outer_valgrind "
. "--tool=" . $outer_tool . " "
- . "$outer_args "
. "--log-file=" . "$name.outer.log "
+ . "$run_outer_args "
. "$valgrind --command-line-only=yes --memcheck:leak-check=no "
. "--sim-hints=no-inner-prefix "
. "--tool=$tool $extraopts $vgopts "
|
|
From: <sv...@va...> - 2016-11-12 09:09:39
|
Author: iraisr
Date: Sat Nov 12 09:09:29 2016
New Revision: 16132
Log:
Provide a paragraph about Solaris agent thread in advanced concepts
n-i-bz
Modified:
trunk/docs/xml/manual-core-adv.xml
Modified: trunk/docs/xml/manual-core-adv.xml
==============================================================================
--- trunk/docs/xml/manual-core-adv.xml (original)
+++ trunk/docs/xml/manual-core-adv.xml Sat Nov 12 09:09:29 2016
@@ -1057,6 +1057,12 @@
OS X or Android.
</para>
+ <para>Unblocking processes blocked in system calls is implemented
+ via agent thread on Solaris. This is quite a different approach
+ than using ptrace on Linux, but leads to equivalent result - Valgrind
+ gdbserver is invoked. Note that agent thread is a Solaris OS
+ feature and cannot be disabled.
+ </para>
</listitem>
<listitem>
|
|
From: Ivo R. <iv...@iv...> - 2016-11-12 06:46:15
|
2016-11-11 8:11 GMT-07:00 <sv...@va...>: > > Log: > Update documentation and NEWS for xtree concept. > > Final patch of the xtree serie, which provides the documentation. > > Thanks to Ivo, Julian and Josef for the review comments. > And big thanks to Philippe who actually made this happen! I. |
|
From: <sv...@va...> - 2016-11-11 15:11:57
|
Author: philippe
Date: Fri Nov 11 15:11:49 2016
New Revision: 16131
Log:
Update documentation and NEWS for xtree concept.
Final patch of the xtree serie, which provides the documentation.
The xtree concept was committed in the revisions
16120 : Support pool of unique string in pub_tool_deduppoolalloc.h
16121 : Implement a cache 'address -> symbol name' in m_debuginfo.c
16122 : Add VG_(strIsMemberXA) in pub_tool_xarray.h
16123 : Addition of the pub_tool_xtree.h and pub_tool_xtmemory.h modules, and of the --xtree-memory* options
16124 : Addition of the options --xtree-memory and --xtree-memory-file
16125 : Small changes in callgrind_annotate and callgrind manual
16126 : Locally define vgPlain_scrcmp in 2 unit tests
16127 : Support for xtree memory profiling and xtmemory gdbsrv monitor command in helgrind
16128 : Support for xtree memory profiling and xtmemory gdbsrv monitor command in memcheck
16129 : Update massif implementation to xtree
Some smaller follow-up patches to be expected to add some regtests,
and refine documentation.
Thanks to Ivo, Julian and Josef for the review comments.
Added:
trunk/docs/images/kcachegrind_xtree.png (with props)
Modified:
trunk/NEWS
trunk/docs/Makefile.am
trunk/docs/xml/manual-core-adv.xml
trunk/docs/xml/manual-core.xml
Modified: trunk/NEWS
==============================================================================
--- trunk/NEWS (original)
+++ trunk/NEWS Fri Nov 11 15:11:49 2016
@@ -2,10 +2,60 @@
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~--
Release 3.13.0 is under development, not yet released.
+3.13.0 is a feature release with many improvements and the usual
+collection of bug fixes.
+
+This release supports X86/Linux, AMD64/Linux, ARM32/Linux,
+ARM64/Linux, PPC32/Linux, PPC64BE/Linux, PPC64LE/Linux, S390X/Linux,
+MIPS32/Linux, MIPS64/Linux, ARM/Android, ARM64/Android,
+MIPS32/Android, X86/Android, X86/Solaris, AMD64/Solaris, X86/MacOSX
+10.10 and AMD64/MacOSX 10.10. There is also preliminary support for
+X86/MacOSX 10.11/12, AMD64/MacOSX 10.11/12 and TILEGX/Linux.
+
+A significant change in 3.13 is the addition of the 'xtree' concept:
+An xtree is a tree of stacktraces with data associated to the stacktraces.
+This xtree is used by various tools (memcheck, helgrind, massif) to
+report the heap consumption of your program. The xtree reporting
+is controlled by the new options --xtree-memory=none|allocs|full and
+--xtree-memory-file=<file>.
+An heap xtree memory profiling can also be produced on demand using
+a gdbserver monitor command.
+The xtree can be output in 2 formats: 'callgrind format'
+and 'massif format. The existing visualisers for these formats (e.g.
+callgrind_annotate, kcachegrind, ms_print) can be used to visualise
+and analyse these reports.
+For more details, read the user manual.
+
* ================== PLATFORM CHANGES =================
* ==================== TOOL CHANGES ====================
+* Memcheck:
+
+ - Support for --xtree-memory profiling.
+
+ - A new monitor command 'xtmemory [<filename>]>' produces a
+ heap usage profile report.
+
+* Massif:
+
+ - Support for --xtree-memory profiling.
+
+ - A new monitor command 'xtmemory [<filename>]>' produces a
+ heap usage profile report.
+
+ - For some workloads (typically, for big applications), Massif
+ memory consumption and CPU consumption decreases significantly.
+
+* Helgrind:
+
+ - Support for --xtree-memory profiling.
+
+ - A new monitor command 'xtmemory [<filename>]>' produces a
+ heap usage profile report.
+
+
+
* ==================== OTHER CHANGES ====================
* ==================== FIXED BUGS ====================
@@ -23,6 +73,7 @@
371412 Rename wrap_sys_shmat to sys_shmat like other wrappers
371869 support '%' in symbol Z-encoding
+371916 execution tree xtree concept
372120 c++ demangler demangles symbols which are not c++
Modified: trunk/docs/Makefile.am
==============================================================================
--- trunk/docs/Makefile.am (original)
+++ trunk/docs/Makefile.am Fri Nov 11 15:11:49 2016
@@ -19,6 +19,7 @@
images/next.png \
images/prev.png \
images/up.png \
+ images/kcachegrind_xtree.png \
internals/3_0_BUGSTATUS.txt \
internals/3_1_BUGSTATUS.txt \
internals/3_2_BUGSTATUS.txt \
@@ -173,6 +174,7 @@
export XML_CATALOG_FILES=$(XML_CATALOG_FILES) && \
mkdir -p $(myprintdir) && \
mkdir -p $(myprintdir)/images && \
+ cp $(myimgdir)/*.png $(myprintdir)/images && \
$(XSLTPROC) $(XSLTPROC_FLAGS) -o $(myprintdir)/index.fo $(XSL_FO_STYLE) $(myxmldir)/index.xml && \
(cd $(myprintdir) && \
( pdfxmltex index.fo && \
Added: trunk/docs/images/kcachegrind_xtree.png
==============================================================================
Binary file - no diff available.
Modified: trunk/docs/xml/manual-core-adv.xml
==============================================================================
--- trunk/docs/xml/manual-core-adv.xml (original)
+++ trunk/docs/xml/manual-core-adv.xml Fri Nov 11 15:11:49 2016
@@ -1390,6 +1390,13 @@
to a huge value and continue execution.</para>
</listitem>
+ <listitem>
+ <para><varname>xtmemory [<filename> default xtmemory.kcg]</varname>
+ requests the tool to produce an xtree heap memory report.
+ See <xref linkend="manual-core.xtree"/> for
+ a detailed explanation about execution trees. </para>
+ </listitem>
+
</itemizedlist>
<para>The following Valgrind monitor commands are useful for
Modified: trunk/docs/xml/manual-core.xml
==============================================================================
--- trunk/docs/xml/manual-core.xml (original)
+++ trunk/docs/xml/manual-core.xml Fri Nov 11 15:11:49 2016
@@ -1011,7 +1011,7 @@
<option>--xml=yes</option>. Any <option>%p</option> or
<option>%q</option> sequences appearing in the filename are expanded
in exactly the same way as they are for <option>--log-file</option>.
- See the description of <option>--log-file</option> for details.
+ See the description of <xref linkend="opt.log-file"/> for details.
</para>
</listitem>
</varlistentry>
@@ -1686,6 +1686,74 @@
</listitem>
</varlistentry>
+ <varlistentry id="opt.xtree-memory" xreflabel="--xtree-memory">
+ <term>
+ <option><![CDATA[--xtree-memory=none|allocs|full [none] ]]></option>
+ </term>
+ <listitem>
+ <para> Tools replacing Valgrind's <function>malloc,
+ realloc,</function> etc, can optionally produce an execution
+ tree detailing which piece of code is responsible for heap
+ memory usage. See <xref linkend="manual-core.xtree"/>
+ for a detailed explanation about execution trees. </para>
+
+ <para> When set to <varname>none</varname>, no memory execution
+ tree is produced.</para>
+
+ <para> When set to <varname>allocs</varname>, the memory
+ execution tree gives the current number of allocated bytes and
+ the current number of allocated blocks. </para>
+
+ <para> When set to <varname>full</varname>, the memory execution
+ tree gives 6 different measurements : the current number of
+ allocated bytes and blocks (same values as
+ for <varname>allocs</varname>), the total number of allocated
+ bytes and blocks, the total number of freed bytes and
+ blocks.</para>
+
+ <para>Note that the overhead in cpu and memory to produce
+ an xtree depends on the tool. The overhead in cpu is small for
+ the value <varname>allocs</varname>, as the information needed
+ to produce this report is maintained in any case by the tool.
+ For massif and helgrind, specifying <varname>full</varname>
+ implies to capture a stack trace for each free operation,
+ while normally these tools only capture an allocation stack
+ trace. For memcheck, the cpu overhead for the
+ value <varname>full</varname> is small, as this can only be
+ used in combination with
+ <option>--keep-stacktraces=alloc-and-free</option> or
+ <option>--keep-stacktraces=alloc-then-free</option>, which
+ already records a stack trace for each free operation. The
+ memory overhead varies between 5 and 10 words per unique
+ stacktrace in the xtree, plus the memory needed to record the
+ stack trace for the free operations, if needed specifically
+ for the xtree.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="opt.xtree-memory-file" xreflabel="--xtree-memory-file">
+ <term>
+ <option><![CDATA[--xtree-memory-file=<filename> [default:
+ xtmemory.kcg.%p] ]]></option>
+ </term>
+ <listitem>
+ <para>Specifies that Valgrind should produce the xtree memory
+ report in the specified file. Any <option>%p</option> or
+ <option>%q</option> sequences appearing in the filename are expanded
+ in exactly the same way as they are for <option>--log-file</option>.
+ See the description of <xref linkend="opt.log-file"/>
+ for details. </para>
+ <para>If the filename contains the extension <option>.ms</option>,
+ then the produced file format will be a massif output file format.
+ If the filename contains the extension <option>.kcg</option>
+ or no extension is provided or recognised,
+ then the produced file format will be a callgrind output format.</para>
+ <para>See <xref linkend="manual-core.xtree"/>
+ for a detailed explanation about execution trees formats. </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
<!-- end of xi:include in the manpage -->
@@ -2673,11 +2741,231 @@
</sect1>
+<sect1 id="manual-core.xtree" xreflabel="Execution Trees">
+<title>Execution Trees</title>
+<para>An execution tree (xtree) is made of a set of stack traces, each
+ stack trace is associated with some resource consumptions or event
+ counts. Depending on the xtree, different event counts/resource
+ consumptions can be recorded in the xtree.</para>
+
+<para> A typical usage for an xtree is to show a graphical or textual
+ representation of the heap usage of a program. The below figure is
+ a heap usage xtree graphical representation produced by
+ kcachegrind. In the kcachegrind output, you can see that main
+ current heap usage (allocated indirectly) is 528 bytes : 388 bytes
+ allocated indirectly via a call to function f1 and 140 bytes
+ indirectly allocated via a call to function f2. f2 has allocated
+ memory by calling g2, while f1 has allocated memory by calling g11
+ and g12. g11, g12 and g1 have directly called a memory allocation
+ function (malloc), and so have a non zero 'Self' value. Note that when
+ kcachegrind shows an xtree, the 'Called' column and call nr indications in
+ the Call Graph are not significant (always set to 0 or 1, independently
+ of the real nr of calls. A future version of kcachegrind will not show
+ anymore such irrelevant xtree call number information.</para>
+
+<graphic fileref="images/kcachegrind_xtree.png" scalefit="1"/>
+
+<para>An xtree heap memory report is produced at the end of the
+ execution when required using the
+ option <option>--xtree-memory</option>. It can also be produced on
+ demand using the <option>xtmemory</option> monitor command (see
+ <xref linkend="manual-core-adv.valgrind-monitor-commands"/>). Currently,
+ an xtree heap memory report can be produced by
+ the <option>memcheck</option>, <option>helgrind</option>
+ and <option>massif</option> tools.</para>
+
+ <para>The xtrees produced by the option
+ <xref linkend="opt.xtree-memory"/> or the <option>xtmemory</option>
+ monitor command are showing the following events/resource
+ consumption describing heap usage:</para>
+<itemizedlist>
+ <listitem>
+ <para><option>curB</option> current number of Bytes allocated. The
+ number of allocated bytes is added to the <option>curB</option>
+ value of a stack trace for each allocation. It is decreased when
+ a block allocated by this stack trace is released (by another
+ "freeing" stack trace)</para>
+ </listitem>
+
+ <listitem>
+ <para><option>curBk</option> current number of Blocks allocated,
+ maintained similary to curB : +1 for each allocation, -1 when
+ the block is freed.</para>
+ </listitem>
+
+ <listitem>
+ <para><option>totB</option> total allocated Bytes. This is
+ increased for each allocation with the number of allocated bytes.</para>
+ </listitem>
+
+ <listitem>
+ <para><option>totBk</option> total allocated Blocks, maintained similary
+ to totB : +1 for each allocation.</para>
+ </listitem>
+
+ <listitem>
+ <para><option>totFdB</option> total Freed Bytes, increased each time
+ a block is released by this ("freeing") stack trace : + nr freed bytes
+ for each free operation.</para>
+ </listitem>
+
+ <listitem>
+ <para><option>totFdBk</option> total Freed Blocks, maintained similarly
+ to totFdB : +1 for each free operation.</para>
+ </listitem>
+</itemizedlist>
+<para>Note that the last 4 counts are produced only when the
+ <option>--xtree-memory=full</option> was given at startup.</para>
+<para>Xtrees can be saved in 2 file formats, the "Callgrind Format" and
+the "Massif Format".</para>
+<itemizedlist>
+
+ <listitem>
+ <para>Callgrind Format</para>
+ <para>An xtree file in the Callgrind Format contains a single callgraph,
+ associating each stack trace with the values recorded
+ in the xtree. </para>
+ <para>Different Callgrind Format file visualisers are available:</para>
+ <para>Valgrind distribution includes the <option>callgrind_annotate</option>
+ command line utility that reads in the xtree data, and prints a sorted
+ lists of functions, optionally with source annotation. Note that due to
+ xtree specificities, you must give the option
+ <option>--inclusive=yes</option> to callgrind_annotate.</para>
+ <para>For graphical visualization of the data, you can use
+ <ulink url="&cl-gui-url;">KCachegrind</ulink>, which is a KDE/Qt based
+ GUI that makes it easy to navigate the large amount of data that
+ an xtree can contain.</para>
+ </listitem>
+
+ <listitem>
+ <para>Massif Format</para>
+ <para>An xtree file in the Massif Format contains one detailed tree
+ callgraph data for each type of event recorded in the xtree. So,
+ for <option>--xtree-memory=alloc</option>, the output file will
+ contain 2 detailed trees (for the counts <option>curB</option>
+ and <option>curBk</option>),
+ while <option>--xtree-memory=full</option> will give a file
+ with 6 detailed trees.</para>
+ <para>Different Massif Format file visualisers are available. Valgrind
+ distribution includes the <option>ms_print</option>
+ command line utility that produces an easy to read reprentation of
+ a massif output file. See <xref linkend="ms-manual.running-massif"/> and
+ <xref linkend="ms-manual.using"/> for more details
+ about visualising Massif Format output files.</para>
+ </listitem>
+</itemizedlist>
+
+<para>Note that it is recommended to use the "Callgrind Format" as it
+ is more compact than the Massif Format, and the Callgrind Format
+ visualiser are more versatile that the Massif Format
+ visualisers. kcachegrind is particularly easy to use to analyse
+ big xtree data.</para>
+
+<para>To clarify the xtree concept, the below gives several extracts of
+ the output produced by the following commands:
+<screen><![CDATA[
+valgrind --xtree-memory=full --xtree-memory-file=xtmemory.kcg mfg
+callgrind_annotate --auto=yes --inclusive=yes --sort=curB:100,curBk:100,totB:100,totBk:100,totFdB:100,totFdBk:100 xtmemory.kcg
+]]></screen>
+</para>
+<para>The below extract shows that the program mfg has allocated in
+ total 770 bytes in 60 different blocks. Of these 60 blocks, 19 were
+ freed, releasing a total of 242 bytes. The heap currently contains
+ 528 bytes in 41 blocks.</para>
+<screen><![CDATA[
+--------------------------------------------------------------------------------
+curB curBk totB totBk totFdB totFdBk
+--------------------------------------------------------------------------------
+ 528 41 770 60 242 19 PROGRAM TOTALS
+]]></screen>
+
+<para>The below gives more details about which functions have
+ allocated or released memory. As an example, we see that main has
+ (directly or indirectly) allocated 770 bytes of memory and freed
+ (directly or indirectly) 242 bytes of memory. The function f1 has
+ (directly or indirectly) allocated 570 bytes of memory, and has not
+ (directly or indirectly) freed memory. Of the 570 bytes allocated
+ by function f1, 388 bytes (34 blocks) have not been
+ released.</para>
+<screen><![CDATA[
+--------------------------------------------------------------------------------
+curB curBk totB totBk totFdB totFdBk file:function
+--------------------------------------------------------------------------------
+ 528 41 770 60 242 19 mfg.c:main
+ 388 34 570 50 0 0 mfg.c:f1
+ 220 20 330 30 0 0 mfg.c:g11
+ 168 14 240 20 0 0 mfg.c:g12
+ 140 7 200 10 0 0 mfg.c:g2
+ 140 7 200 10 0 0 mfg.c:f2
+ 0 0 0 0 131 10 mfg.c:freeY
+ 0 0 0 0 111 9 mfg.c:freeX
+]]></screen>
+
+<para>The below gives a more detailed information about the callgraph
+ and which source lines/calls have (directly or indirectly) allocated or
+ released memory. The below shows that the 770 bytes allocated by
+ main have been indirectly allocated by calls to f1 and f2.
+ Similarly, we see that the 570 bytes allocated by f1 have been
+ indirectly allocated by calls to g11 and g12. Of the 330 bytes allocated
+ by the 30 calls to g11, 168 bytes have not been freed.
+ The function freeY (called once by main) has released in total
+ 10 blocks and 131 bytes. </para>
+<screen><![CDATA[
+--------------------------------------------------------------------------------
+-- Auto-annotated source: /home/philippe/valgrind/littleprogs/ + mfg.c
+--------------------------------------------------------------------------------
+curB curBk totB totBk totFdB totFdBk
+....
+ . . . . . . static void freeY(void)
+ . . . . . . {
+ . . . . . . int i;
+ . . . . . . for (i = 0; i < next_ptr; i++)
+ . . . . . . if(i % 5 == 0 && ptrs[i] != NULL)
+ 0 0 0 0 131 10 free(ptrs[i]);
+ . . . . . . }
+ . . . . . . static void f1(void)
+ . . . . . . {
+ . . . . . . int i;
+ . . . . . . for (i = 0; i < 30; i++)
+ 220 20 330 30 0 0 g11();
+ . . . . . . for (i = 0; i < 20; i++)
+ 168 14 240 20 0 0 g12();
+ . . . . . . }
+ . . . . . . int main()
+ . . . . . . {
+ 388 34 570 50 0 0 f1();
+ 140 7 200 10 0 0 f2();
+ 0 0 0 0 111 9 freeX();
+ 0 0 0 0 131 10 freeY();
+ . . . . . . return 0;
+ . . . . . . }
+]]></screen>
+
+<para>Heap memory xtrees are helping to understand how your (big)
+ program is using the heap. A full heap memory xtree helps to pin
+ point some code that allocates a lot of small objects : allocating
+ such small objects might be replaced by more efficient technique,
+ such as allocating a big block using malloc, and then diviving this
+ block into smaller blocks in order to decrease the cpu and/or memory
+ overhead of allocating a lot of small blocks. Such full xtree information
+ complements e.g. what callgrind can show: callgrind can show the number
+ of calls to a function (such as malloc) but does not indicate the volume
+ of memory allocated (or freed).</para>
+
+<para>A full heap memory xtree also can identify the code that allocates
+ and frees a lot of blocks : the total foot print of the program might
+ not reflect the fact that the same memory was over and over allocated
+ then released.</para>
+
+<para>Finally, Xtree visualisers such as kcachegrind are helping to
+ identify big memory consumers, in order to possibly optimise the
+ amount of memory needed by your program.</para>
+</sect1>
<sect1 id="manual-core.install" xreflabel="Building and Installing">
<title>Building and Installing Valgrind</title>
|
|
From: <sv...@va...> - 2016-11-11 14:47:41
|
Author: iraisr
Date: Fri Nov 11 14:47:33 2016
New Revision: 16130
Log:
Provide Solaris specifics in coregrind/m_gdbserver/README_DEVELOPERS
n-i-bz
Modified:
trunk/coregrind/m_gdbserver/README_DEVELOPERS
Modified: trunk/coregrind/m_gdbserver/README_DEVELOPERS
==============================================================================
--- trunk/coregrind/m_gdbserver/README_DEVELOPERS (original)
+++ trunk/coregrind/m_gdbserver/README_DEVELOPERS Fri Nov 11 14:47:33 2016
@@ -32,7 +32,7 @@
The standard gdb distribution has a standalone gdbserver (a small
executable) which implements this protocol and the needed system calls
to allow gdb to remotely debug process running on a linux or MacOS or
-...
+Solaris...
Activation of gdbserver code inside valgrind
--------------------------------------------
@@ -206,6 +206,12 @@
Do not kill -9 vgdb while it has interrupted the valgrind process,
otherwise the valgrind process will very probably stay stopped or die.
+On Solaris, this forced invocation is implemented via agent thread.
+The process is first stopped (all the threads at once), and special agent
+thread is created which will force gbdserver invocation. After its
+work is done, the agent thread is destroyed and process resumed.
+Agent thread functionality is a Solaris OS feature, used also by debuggers.
+Therefore vgdb-invoker-solaris implementation is really small.
Implementation is based on the gdbserver code from gdb 6.6
----------------------------------------------------------
@@ -257,7 +263,7 @@
The main thing to do is to make a file valgrind-low-hal9000.c.
Start from an existing file (e.g. valgrind-low-x86.c).
The data structures 'struct reg regs'
-and 'const char *expedite_regs' are build from files
+and 'const char *expedite_regs' are built from files
in the gdb sources, e.g. for an new arch hal9000
cd gdb/regformats
sh ./regdat.sh reg-hal9000.dat hal9000
@@ -287,6 +293,9 @@
things are needed e.g. to attach to threads etc).
A courageous Mac aficionado is welcome on this aspect.
+For Solaris, only architecture specific functionality in vgdb-invoker-solaris.c
+needs to be implemented, similar to Linux above.
+
Optional:
To let gdb see the Valgrind shadow registers, xml description
files have to be provided + valgrind-low-hal9000.c has
|
|
From: <sv...@va...> - 2016-11-11 14:36:43
|
Author: philippe
Date: Fri Nov 11 14:36:36 2016
New Revision: 16128
Log:
Support for xtree memory profiling and xtmemory gdbsrv monitor command in memcheck
* memcheck will produce xtree memory profiling according to the options
--xtree-memory.
* addition of the xtmemory gdbserver monitor command.
(this is the second real xtree functional difference)
Modified:
trunk/gdbserver_tests/mchelp.stdoutB.exp
trunk/memcheck/mc_include.h
trunk/memcheck/mc_main.c
trunk/memcheck/mc_malloc_wrappers.c
Modified: trunk/gdbserver_tests/mchelp.stdoutB.exp
==============================================================================
--- trunk/gdbserver_tests/mchelp.stdoutB.exp (original)
+++ trunk/gdbserver_tests/mchelp.stdoutB.exp Fri Nov 11 14:36:36 2016
@@ -52,6 +52,8 @@
shows places pointing inside <len> (default 1) bytes at <addr>
(with len 1, only shows "start pointers" pointing exactly to <addr>,
with len > 1, will also show "interior pointers")
+ xtmemory [<filename>]
+ dump xtree memory profile in <filename> (default xtmemory.kcg)
general valgrind monitor commands:
help [debug] : monitor command help. With debug: + debugging commands
v.wait [<ms>] : sleep <ms> (default 0) then continue
@@ -121,4 +123,6 @@
shows places pointing inside <len> (default 1) bytes at <addr>
(with len 1, only shows "start pointers" pointing exactly to <addr>,
with len > 1, will also show "interior pointers")
+ xtmemory [<filename>]
+ dump xtree memory profile in <filename> (default xtmemory.kcg)
monitor command request to kill this process
Modified: trunk/memcheck/mc_include.h
==============================================================================
--- trunk/memcheck/mc_include.h (original)
+++ trunk/memcheck/mc_include.h Fri Nov 11 14:36:36 2016
@@ -144,6 +144,8 @@
void MC_(make_mem_defined) ( Addr a, SizeT len );
void MC_(copy_address_range_state) ( Addr src, Addr dst, SizeT len );
+void MC_(xtmemory_report) ( const HChar* filename, Bool fini );
+
void MC_(print_malloc_stats) ( void );
/* nr of free operations done */
SizeT MC_(get_cmalloc_n_frees) ( void );
Modified: trunk/memcheck/mc_main.c
==============================================================================
--- trunk/memcheck/mc_main.c (original)
+++ trunk/memcheck/mc_main.c Fri Nov 11 14:36:36 2016
@@ -47,6 +47,9 @@
#include "pub_tool_replacemalloc.h"
#include "pub_tool_tooliface.h"
#include "pub_tool_threadstate.h"
+#include "pub_tool_xarray.h"
+#include "pub_tool_xtree.h"
+#include "pub_tool_xtmemory.h"
#include "mc_include.h"
#include "memcheck.h" /* for client requests */
@@ -6400,6 +6403,8 @@
" shows places pointing inside <len> (default 1) bytes at <addr>\n"
" (with len 1, only shows \"start pointers\" pointing exactly to <addr>,\n"
" with len > 1, will also show \"interior pointers\")\n"
+" xtmemory [<filename>]\n"
+" dump xtree memory profile in <filename> (default xtmemory.kcg)\n"
"\n");
}
@@ -6515,7 +6520,7 @@
command. This ensures a shorter abbreviation for the user. */
switch (VG_(keyword_id)
("help get_vbits leak_check make_memory check_memory "
- "block_list who_points_at xb",
+ "block_list who_points_at xb xtmemory",
wcmd, kwd_report_duplicated_matches)) {
case -2: /* multiple matches */
return True;
@@ -6874,6 +6879,13 @@
return True;
}
+ case 8: { /* xtmemory */
+ HChar* filename;
+ filename = VG_(strtok_r) (NULL, " ", &ssaveptr);
+ MC_(xtmemory_report)(filename, False);
+ return True;
+ }
+
default:
tl_assert(0);
return False;
@@ -7889,6 +7901,17 @@
/* Do not check definedness of guest state if --undef-value-errors=no */
if (MC_(clo_mc_level) >= 2)
VG_(track_pre_reg_read) ( mc_pre_reg_read );
+
+ if (VG_(clo_xtree_memory) == Vg_XTMemory_Full) {
+ if (MC_(clo_keep_stacktraces) == KS_none
+ || MC_(clo_keep_stacktraces) == KS_free)
+ VG_(fmsg_bad_option)("--keep-stacktraces",
+ "To use --xtree-memory=full, you must"
+ " keep at least the alloc stacktrace\n");
+ // Activate full xtree memory profiling.
+ VG_(XTMemory_Full_init)(VG_(XT_filter_1top_and_maybe_below_main));
+ }
+
}
static void print_SM_info(const HChar* type, Int n_SMs)
@@ -7997,6 +8020,7 @@
static void mc_fini ( Int exitcode )
{
+ MC_(xtmemory_report) (VG_(clo_xtree_memory_file), True);
MC_(print_malloc_stats)();
if (MC_(clo_leak_check) != LC_Off) {
Modified: trunk/memcheck/mc_malloc_wrappers.c
==============================================================================
--- trunk/memcheck/mc_malloc_wrappers.c (original)
+++ trunk/memcheck/mc_malloc_wrappers.c Fri Nov 11 14:36:36 2016
@@ -36,12 +36,16 @@
#include "pub_tool_libcbase.h"
#include "pub_tool_libcassert.h"
#include "pub_tool_libcprint.h"
+#include "pub_tool_libcproc.h"
#include "pub_tool_mallocfree.h"
#include "pub_tool_options.h"
#include "pub_tool_replacemalloc.h"
#include "pub_tool_threadstate.h"
#include "pub_tool_tooliface.h" // Needed for mc_include.h
#include "pub_tool_stacktrace.h" // For VG_(get_and_pp_StackTrace)
+#include "pub_tool_xarray.h"
+#include "pub_tool_xtree.h"
+#include "pub_tool_xtmemory.h"
#include "mc_include.h"
@@ -303,20 +307,33 @@
default: tl_assert (0);
}
mc->where[0] = VG_(record_ExeContext) ( tid, 0/*first_ip_delta*/ );
+ if (UNLIKELY(VG_(clo_xtree_memory) == Vg_XTMemory_Full))
+ VG_(XTMemory_Full_alloc)(mc->szB, mc->where[0]);
}
void MC_(set_freed_at) (ThreadId tid, MC_Chunk* mc)
{
- UInt pos;
+ Int pos;
+ ExeContext* ec_free;
+
switch (MC_(clo_keep_stacktraces)) {
case KS_none: return;
- case KS_alloc: return;
+ case KS_alloc: pos = -1; break;
case KS_free: pos = 0; break;
case KS_alloc_then_free: pos = 0; break;
case KS_alloc_and_free: pos = 1; break;
default: tl_assert (0);
}
- mc->where[pos] = VG_(record_ExeContext) ( tid, 0/*first_ip_delta*/ );
+ /* We need the execontext for the free operation, either to store
+ it in the mc chunk and/or for full xtree memory profiling.
+ Note: we are guaranteed to find the ec_alloc in mc->where[0], as
+ mc_post_clo_init verifies the consistency of --xtree-memory and
+ --keep-stacktraces. */
+ ec_free = VG_(record_ExeContext) ( tid, 0/*first_ip_delta*/ );
+ if (UNLIKELY(VG_(clo_xtree_memory) == Vg_XTMemory_Full))
+ VG_(XTMemory_Full_free)(mc->szB, mc->where[0], ec_free);
+ if (pos >= 0)
+ mc->where[pos] = ec_free;
}
UInt MC_(n_where_pointers) (void)
@@ -651,6 +668,9 @@
if (oldSizeB == newSizeB)
return;
+ if (UNLIKELY(VG_(clo_xtree_memory) == Vg_XTMemory_Full))
+ VG_(XTMemory_Full_resize_in_place)(oldSizeB, newSizeB, mc->where[0]);
+
mc->szB = newSizeB;
if (newSizeB < oldSizeB) {
MC_(make_mem_noaccess)( p + newSizeB, oldSizeB - newSizeB + rzB );
@@ -1117,6 +1137,25 @@
return True;
}
+static void xtmemory_report_next_block(XT_Allocs* xta, ExeContext** ec_alloc)
+{
+ MC_Chunk* mc = VG_(HT_Next)(MC_(malloc_list));
+ if (mc) {
+ xta->nbytes = mc->szB;
+ xta->nblocks = 1;
+ *ec_alloc = MC_(allocated_at)(mc);
+ } else
+ xta->nblocks = 0;
+}
+
+void MC_(xtmemory_report) ( const HChar* filename, Bool fini )
+{
+ // Make xtmemory_report_next_block ready to be called.
+ VG_(HT_ResetIter)(MC_(malloc_list));
+
+ VG_(XTMemory_report)(filename, fini, xtmemory_report_next_block,
+ VG_(XT_filter_1top_and_maybe_below_main));
+}
/*------------------------------------------------------------*/
/*--- Statistics printing ---*/
|
|
From: <sv...@va...> - 2016-11-11 14:33:34
|
Author: philippe
Date: Fri Nov 11 14:33:27 2016
New Revision: 16127
Log:
Support for xtree memory profiling and xtmemory gdbsrv monitor command in helgrind
* helgrind will produce xtree memory profiling according to the options
--xtree-memory.
* addition of the xtmemory gdbserver monitor command.
(this is the first real xtree functional difference)
Modified:
trunk/helgrind/hg_main.c
Modified: trunk/helgrind/hg_main.c
==============================================================================
--- trunk/helgrind/hg_main.c (original)
+++ trunk/helgrind/hg_main.c Fri Nov 11 14:33:27 2016
@@ -57,6 +57,8 @@
#include "pub_tool_aspacemgr.h" // VG_(am_is_valid_for_client)
#include "pub_tool_poolalloc.h"
#include "pub_tool_addrinfo.h"
+#include "pub_tool_xtree.h"
+#include "pub_tool_xtmemory.h"
#include "hg_basics.h"
#include "hg_wordset.h"
@@ -4157,6 +4159,8 @@
md->thr = map_threads_lookup( tid );
VG_(HT_add_node)( hg_mallocmeta_table, (VgHashNode*)md );
+ if (UNLIKELY(VG_(clo_xtree_memory) == Vg_XTMemory_Full))
+ VG_(XTMemory_Full_alloc)(md->szB, md->where);
/* Tell the lower level memory wranglers. */
evh__new_mem_heap( p, szB, is_zeroed );
@@ -4211,6 +4215,10 @@
tl_assert(md->payload == (Addr)p);
szB = md->szB;
+ if (UNLIKELY(VG_(clo_xtree_memory) == Vg_XTMemory_Full)) {
+ ExeContext* ec_free = VG_(record_ExeContext)( tid, 0 );
+ VG_(XTMemory_Full_free)(md->szB, md->where, ec_free);
+ }
/* Nuke the metadata block */
old_md = (MallocMeta*)
@@ -4885,6 +4893,25 @@
sizeof(GNAT_dmml) );
}
}
+
+static void xtmemory_report_next_block(XT_Allocs* xta, ExeContext** ec_alloc)
+{
+ const MallocMeta* md = VG_(HT_Next)(hg_mallocmeta_table);
+ if (md) {
+ xta->nbytes = md->szB;
+ xta->nblocks = 1;
+ *ec_alloc = md->where;
+ } else
+ xta->nblocks = 0;
+}
+static void HG_(xtmemory_report) ( const HChar* filename, Bool fini )
+{
+ // Make xtmemory_report_next_block ready to be called.
+ VG_(HT_ResetIter)(hg_mallocmeta_table);
+ VG_(XTMemory_report)(filename, fini, xtmemory_report_next_block,
+ VG_(XT_filter_1top_and_maybe_below_main));
+}
+
static void print_monitor_help ( void )
{
VG_(gdb_printf)
@@ -4895,6 +4922,8 @@
" with no lock_addr, show status of all locks\n"
" accesshistory <addr> [<len>] : show access history recorded\n"
" for <len> (or 1) bytes at <addr>\n"
+" xtmemory [<filename>]\n"
+" dump xtree memory profile in <filename> (default xtmemory.kcg)\n"
"\n");
}
@@ -4913,7 +4942,7 @@
starts with the same first letter(s) as an already existing
command. This ensures a shorter abbreviation for the user. */
switch (VG_(keyword_id)
- ("help info accesshistory",
+ ("help info accesshistory xtmemory",
wcmd, kwd_report_duplicated_matches)) {
case -2: /* multiple matches */
return True;
@@ -4986,6 +5015,13 @@
return True;
}
+ case 3: { /* xtmemory */
+ HChar* filename;
+ filename = VG_(strtok_r) (NULL, " ", &ssaveptr);
+ HG_(xtmemory_report)(filename, False);
+ return True;
+ }
+
default:
tl_assert(0);
return False;
@@ -5668,6 +5704,7 @@
static void hg_fini ( Int exitcode )
{
+ HG_(xtmemory_report) (VG_(clo_xtree_memory_file), True);
if (VG_(clo_verbosity) == 1 && !VG_(clo_xml)) {
VG_(message)(Vg_UserMsg,
"For counts of detected and suppressed errors, "
@@ -5740,6 +5777,9 @@
laog__init();
initialise_data_structures(hbthr_root);
+ if (VG_(clo_xtree_memory) == Vg_XTMemory_Full)
+ // Activate full xtree memory profiling.
+ VG_(XTMemory_Full_init)(VG_(XT_filter_1top_and_maybe_below_main));
}
static void hg_info_location (Addr a)
|
|
From: <sv...@va...> - 2016-11-11 14:27:00
|
Author: philippe
Date: Fri Nov 11 14:26:54 2016
New Revision: 16126
Log:
Locally define vgPlain_scrcmp in 2 unit tests
Humph, this should have been part of :
16122 Add VG_(strIsMemberXA) in pub_tool_xarray.h
which means that between 16122 and this revision, these 2 unit tests
will (very probably) not compile.
That will make bissect not easy :(
Modified:
trunk/drd/tests/unit_bitmap.c
trunk/memcheck/tests/unit_oset.c
Modified: trunk/drd/tests/unit_bitmap.c
==============================================================================
--- trunk/drd/tests/unit_bitmap.c (original)
+++ trunk/drd/tests/unit_bitmap.c Fri Nov 11 14:26:54 2016
@@ -44,6 +44,8 @@
abort();
}
+Int VG_(strcmp)( const HChar* s1, const HChar* s2 )
+{ return strcmp(s1, s2); }
void* VG_(memset)(void *s, Int c, SizeT sz)
{ return memset(s, c, sz); }
void* VG_(memcpy)(void *d, const void *s, SizeT sz)
Modified: trunk/memcheck/tests/unit_oset.c
==============================================================================
--- trunk/memcheck/tests/unit_oset.c (original)
+++ trunk/memcheck/tests/unit_oset.c Fri Nov 11 14:26:54 2016
@@ -28,6 +28,7 @@
#define vgPlain_memset memset
#define vgPlain_memcpy memcpy
#define vgPlain_memmove memmove
+#define vgPlain_strcmp strcmp
// Crudely replace some functions (in m_xarray.c, but not needed for
// this unit test) by (hopefully) failing asserts.
|
|
From: <sv...@va...> - 2016-11-11 14:22:40
|
Author: philippe
Date: Fri Nov 11 14:22:34 2016
New Revision: 16125
Log:
Small changes in callgrind_annotate and callgrind manual
* callgrind_annotate: ignore the lines giving the long names of the
events: such lines are used by kcachegrind for the GUI, so are better
accepted (and ignored) by callgrind_annotate
* Document in callgrind_annotate manual that thresholds can be given
per event kind.
* Avoid a division by zero in callgrind_annotate, for 'special data'
such as produced by (some) xtrees.
Modified:
trunk/callgrind/callgrind_annotate.in
trunk/callgrind/docs/cl-manual.xml
Modified: trunk/callgrind/callgrind_annotate.in
==============================================================================
--- trunk/callgrind/callgrind_annotate.in (original)
+++ trunk/callgrind/callgrind_annotate.in Fri Nov 11 14:22:34 2016
@@ -429,6 +429,9 @@
$has_line = ($positions =~ /line/);
$has_addr = ($positions =~ /(addr|instr)/);
}
+ elsif (/^event:\s+.*$/) {
+ # ignore lines giving a long name to an event
+ }
elsif (/^events:\s+(.*)$/) {
$events = $1;
@@ -1216,9 +1219,17 @@
if ($did_annotations) {
my $percent_printed_CC;
foreach (my $i = 0; $i < @$summary_CC; $i++) {
- $percent_printed_CC->[$i] =
- sprintf("%.0f",
- $printed_totals_CC->[$i] / $summary_CC->[$i] * 100);
+ # Some files (in particular the files produced by --xtree-memory)
+ # have non additive self costs, so have a special case for these
+ # to print all functions and also to avoid a division by 0.
+ if ($summary_CC->[$i] == 0
+ || $printed_totals_CC->[$i] > $summary_CC->[$i]) {
+ $percent_printed_CC->[$i] = "100";
+ } else {
+ $percent_printed_CC->[$i] =
+ sprintf("%.0f",
+ $printed_totals_CC->[$i] / $summary_CC->[$i] * 100);
+ }
}
my $pp_CC_col_widths = compute_CC_col_widths($percent_printed_CC);
print($fancy);
Modified: trunk/callgrind/docs/cl-manual.xml
==============================================================================
--- trunk/callgrind/docs/cl-manual.xml (original)
+++ trunk/callgrind/docs/cl-manual.xml Fri Nov 11 14:22:34 2016
@@ -1257,6 +1257,8 @@
</term>
<listitem>
<para>Sort columns by events A,B,C [event column order].</para>
+ <para>Optionally, each event is followed by a : and a threshold,
+ to specify different thresholds depending on the event.</para>
</listitem>
</varlistentry>
|
Author: philippe
Date: Fri Nov 11 14:15:45 2016
New Revision: 16124
Log:
Addition of the options --xtree-memory and --xtree-memory-file
Option is added, parsed, explained in --help
but is not used yet by any other code.
So, apart of the --help, no functional effect.
Modified:
trunk/coregrind/m_main.c
trunk/coregrind/m_replacemalloc/replacemalloc_core.c
trunk/none/tests/cmdline1.stdout.exp
trunk/none/tests/cmdline2.stdout.exp
Modified: trunk/coregrind/m_main.c
==============================================================================
--- trunk/coregrind/m_main.c (original)
+++ trunk/coregrind/m_main.c Fri Nov 11 14:15:45 2016
@@ -145,6 +145,12 @@
" --alignment=<number> set minimum alignment of heap allocations [%s]\n"
" --redzone-size=<number> set minimum size of redzones added before/after\n"
" heap blocks (in bytes). [%s]\n"
+" --xtree-memory=none|allocs|full profile heap memory in an xtree [none]\n"
+" and produces a report at the end of the execution\n"
+" none: no profiling, allocs: current allocated\n"
+" size/blocks, full: profile current and cumulative\n"
+" allocated size/blocks and freed size/blocks.\n"
+" --xtree-memory-file=<file> xtree memory report file [xtmemory.kcg.%%p]\n"
"\n"
" uncommon user options for all Valgrind tools:\n"
" --fullpath-after= (with nothing after the '=')\n"
Modified: trunk/coregrind/m_replacemalloc/replacemalloc_core.c
==============================================================================
--- trunk/coregrind/m_replacemalloc/replacemalloc_core.c (original)
+++ trunk/coregrind/m_replacemalloc/replacemalloc_core.c Fri Nov 11 14:15:45 2016
@@ -65,6 +65,16 @@
VG_MIN_MALLOC_SZB);
}
}
+ else if VG_XACT_CLO(arg, "--xtree-memory=none",
+ VG_(clo_xtree_memory), Vg_XTMemory_None) {}
+ else if VG_XACT_CLO(arg, "--xtree-memory=allocs",
+ VG_(clo_xtree_memory), Vg_XTMemory_Allocs) {}
+ else if VG_XACT_CLO(arg, "--xtree-memory=full",
+ VG_(clo_xtree_memory), Vg_XTMemory_Full) {}
+ else if VG_STR_CLO (arg, "--xtree-memory-file",
+ VG_(clo_xtree_memory_file)) {}
+ else if VG_BOOL_CLO(arg, "--xtree-compress-strings",
+ VG_(clo_xtree_compress_strings)) {}
else if VG_BOOL_CLO(arg, "--trace-malloc", VG_(clo_trace_malloc)) {}
else
Modified: trunk/none/tests/cmdline1.stdout.exp
==============================================================================
--- trunk/none/tests/cmdline1.stdout.exp (original)
+++ trunk/none/tests/cmdline1.stdout.exp Fri Nov 11 14:15:45 2016
@@ -58,6 +58,12 @@
--alignment=<number> set minimum alignment of heap allocations [not used by this tool]
--redzone-size=<number> set minimum size of redzones added before/after
heap blocks (in bytes). [not used by this tool]
+ --xtree-memory=none|allocs|full profile heap memory in an xtree [none]
+ and produces a report at the end of the execution
+ none: no profiling, allocs: current allocated
+ size/blocks, full: profile current and cumulative
+ allocated size/blocks and freed size/blocks.
+ --xtree-memory-file=<file> xtree memory report file [xtmemory.kcg.%p]
uncommon user options for all Valgrind tools:
--fullpath-after= (with nothing after the '=')
Modified: trunk/none/tests/cmdline2.stdout.exp
==============================================================================
--- trunk/none/tests/cmdline2.stdout.exp (original)
+++ trunk/none/tests/cmdline2.stdout.exp Fri Nov 11 14:15:45 2016
@@ -58,6 +58,12 @@
--alignment=<number> set minimum alignment of heap allocations [not used by this tool]
--redzone-size=<number> set minimum size of redzones added before/after
heap blocks (in bytes). [not used by this tool]
+ --xtree-memory=none|allocs|full profile heap memory in an xtree [none]
+ and produces a report at the end of the execution
+ none: no profiling, allocs: current allocated
+ size/blocks, full: profile current and cumulative
+ allocated size/blocks and freed size/blocks.
+ --xtree-memory-file=<file> xtree memory report file [xtmemory.kcg.%p]
uncommon user options for all Valgrind tools:
--fullpath-after= (with nothing after the '=')
|
|
From: <sv...@va...> - 2016-11-11 14:07:13
|
Author: philippe
Date: Fri Nov 11 14:07:03 2016
New Revision: 16123
Log:
Addition of the pub_tool_xtree.h and pub_tool_xtmemory.h modules, and of the --xtree-memory* options
This commit is the bulk of the new code.
There is however no functional impact yet : the new modules are not used by anybody.
Added:
trunk/coregrind/m_xtmemory.c
trunk/coregrind/m_xtree.c
trunk/coregrind/pub_core_xtmemory.h
trunk/coregrind/pub_core_xtree.h
trunk/include/pub_tool_xtmemory.h
trunk/include/pub_tool_xtree.h
Modified:
trunk/coregrind/Makefile.am
trunk/coregrind/m_options.c
trunk/coregrind/pub_core_options.h
trunk/include/Makefile.am
trunk/include/pub_tool_options.h
Modified: trunk/coregrind/Makefile.am
==============================================================================
--- trunk/coregrind/Makefile.am (original)
+++ trunk/coregrind/Makefile.am Fri Nov 11 14:07:03 2016
@@ -234,6 +234,8 @@
pub_core_vkiscnums_asm.h\
pub_core_wordfm.h \
pub_core_xarray.h \
+ pub_core_xtree.h \
+ pub_core_xtmemory.h \
m_aspacemgr/priv_aspacemgr.h \
m_debuginfo/priv_misc.h \
m_debuginfo/priv_storage.h \
@@ -332,6 +334,8 @@
m_vkiscnums.c \
m_wordfm.c \
m_xarray.c \
+ m_xtree.c \
+ m_xtmemory.c \
m_aspacehl.c \
m_aspacemgr/aspacemgr-common.c \
m_aspacemgr/aspacemgr-linux.c \
Modified: trunk/coregrind/m_options.c
==============================================================================
--- trunk/coregrind/m_options.c (original)
+++ trunk/coregrind/m_options.c Fri Nov 11 14:07:03 2016
@@ -112,6 +112,10 @@
// A value != -1 overrides the tool-specific value
// VG_(needs_malloc_replacement).tool_client_redzone_szB
Int VG_(clo_redzone_size) = -1;
+VgXTMemory VG_(clo_xtree_memory) = Vg_XTMemory_None;
+const HChar* VG_(clo_xtree_memory_file) = "xtmemory.kcg.%p";
+Bool VG_(clo_xtree_compress_strings) = True;
+
Int VG_(clo_dump_error) = 0;
Int VG_(clo_backtrace_size) = 12;
Int VG_(clo_merge_recursive_frames) = 0; // default value: no merge
Added: trunk/coregrind/m_xtmemory.c
==============================================================================
--- trunk/coregrind/m_xtmemory.c (added)
+++ trunk/coregrind/m_xtmemory.c Fri Nov 11 14:07:03 2016
@@ -0,0 +1,341 @@
+
+/*--------------------------------------------------------------------*/
+/*--- Support functions for xtree memory reports. m_xtmemory.c ---*/
+/*--------------------------------------------------------------------*/
+
+/*
+ This file is part of Valgrind, a dynamic binary instrumentation
+ framework.
+
+ Copyright (C) 2016-2016 Philippe Waroquiers
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License as
+ published by the Free Software Foundation; either version 2 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
+ 02111-1307, USA.
+
+ The GNU General Public License is contained in the file COPYING.
+*/
+
+#include "pub_core_libcassert.h"
+#include "pub_core_libcbase.h"
+#include "pub_core_libcprint.h"
+#include "pub_core_libcproc.h"
+#include "pub_core_mallocfree.h"
+#include "pub_core_options.h"
+#include "pub_core_xarray.h"
+#include "pub_core_xtree.h"
+#include "pub_core_xtmemory.h" /* self */
+
+static void VG_(XT_Allocs_init)(void* xt_allocs)
+{
+ VG_(memset) (xt_allocs, 0, sizeof(XT_Allocs));
+}
+static void VG_(XT_Allocs_add) (void* to, const void* xt_allocs)
+{
+ XT_Allocs* xto = to;
+ const XT_Allocs* xta = xt_allocs;
+
+ xto->nbytes += xta->nbytes;
+ xto->nblocks += xta->nblocks;
+}
+static void VG_(XT_Allocs_sub) (void* from, const void* xt_allocs)
+{
+ XT_Allocs* xfrom = from;
+ const XT_Allocs* xta = xt_allocs;
+
+ xfrom->nbytes -= xta->nbytes;
+ xfrom->nblocks -= xta->nblocks;
+}
+static const HChar* VG_(XT_Allocs_img) (const void* xt_allocs)
+{
+ static HChar buf[100];
+
+ const XT_Allocs* xta = xt_allocs;
+
+ if (xta->nbytes > 0 || xta->nblocks > 0) {
+ VG_(sprintf) (buf, "%lu %lu",
+ xta->nbytes, xta->nblocks);
+ return buf;
+ } else {
+ return NULL;
+ }
+}
+const HChar* XT_Allocs_events = "curB : currently allocated Bytes" ","
+ "curBk : currently allocated Blocks";
+
+/* Type and functions for full xtree memory profiling. */
+static XTree* full_xt;
+typedef
+ struct _XT_Full {
+ // Current nr of bytes/blocks allocated by this ec
+ SizeT cur_alloc_nbytes;
+ SizeT cur_alloc_nblocks;
+
+ // Total/cumulative nr of bytes/blocks allocated by this ec
+ ULong tot_alloc_nbytes;
+ ULong tot_alloc_nblocks;
+
+ // Total/cumulative nr of bytes/blocks freed by this ec
+ ULong tot_freed_nbytes;
+ ULong tot_freed_nblocks;
+ } XT_Full;
+/* Note: normally, an ec should never be used as both an alloc_ec and
+ a free_ec. This implies that we should never have a XT_Full that has
+ at the same time some alloc and some freed components > 0.
+ We however still will support this possibility, just in case very
+ strange ec are produced and/or given by the tool. */
+
+static void VG_(XT_Full_init)(void* xtfull)
+{
+ VG_(memset) (xtfull, 0, sizeof(XT_Full));
+}
+static void VG_(XT_Full_add) (void* to, const void* xtfull)
+{
+ XT_Full* xto = to;
+ const XT_Full* xtf = xtfull;
+
+ xto->cur_alloc_nbytes += xtf->cur_alloc_nbytes;
+ xto->cur_alloc_nblocks += xtf->cur_alloc_nblocks;
+ xto->tot_alloc_nbytes += xtf->tot_alloc_nbytes;
+ xto->tot_alloc_nblocks += xtf->tot_alloc_nblocks;
+ xto->tot_freed_nbytes += xtf->tot_freed_nbytes;
+ xto->tot_freed_nblocks += xtf->tot_freed_nblocks;
+}
+static void VG_(XT_Full_sub) (void* from, const void* xtfull)
+{
+ XT_Full* xfrom = from;
+ const XT_Full* xtf = xtfull;
+
+ xfrom->cur_alloc_nbytes -= xtf->cur_alloc_nbytes;
+ xfrom->cur_alloc_nblocks -= xtf->cur_alloc_nblocks;
+ xfrom->tot_alloc_nbytes -= xtf->tot_alloc_nbytes;
+ xfrom->tot_alloc_nblocks -= xtf->tot_alloc_nblocks;
+ xfrom->tot_freed_nbytes -= xtf->tot_freed_nbytes;
+ xfrom->tot_freed_nblocks -= xtf->tot_freed_nblocks;
+}
+static const HChar* VG_(XT_Full_img) (const void* xtfull)
+{
+ static HChar buf[300];
+
+ const XT_Full* xtf = xtfull;
+
+ if ( xtf->cur_alloc_nbytes > 0
+ || xtf->cur_alloc_nblocks > 0
+ || xtf->tot_alloc_nbytes > 0
+ || xtf->tot_alloc_nblocks > 0
+ || xtf->tot_freed_nbytes > 0
+ || xtf->tot_freed_nblocks > 0) {
+ VG_(sprintf) (buf,
+ "%lu %lu "
+ "%llu %llu "
+ "%llu %llu",
+ xtf->cur_alloc_nbytes, xtf->cur_alloc_nblocks,
+ xtf->tot_alloc_nbytes, xtf->tot_alloc_nblocks,
+ xtf->tot_freed_nbytes, xtf->tot_freed_nblocks);
+ return buf;
+ } else {
+ return NULL;
+ }
+}
+static const HChar* XT_Full_events =
+ "curB : currently allocated Bytes" ","
+ "curBk : currently allocated Blocks" ","
+ "totB : total allocated Bytes" ","
+ "totBk : total allocated Blocks" ","
+ "totFdB : total Freed Bytes" ","
+ "totFdBk : total Freed Blocks";
+void VG_(XTMemory_Full_init)(XT_filter_IPs_t filter_IPs_fn)
+{
+ full_xt = VG_(XT_create) (VG_(malloc),
+ "m_xtree.full_xt",
+ VG_(free),
+ sizeof(XT_Full),
+ VG_(XT_Full_init),
+ VG_(XT_Full_add),
+ VG_(XT_Full_sub),
+ filter_IPs_fn);
+}
+void VG_(XTMemory_Full_alloc)(SizeT szB,
+ ExeContext* ec_alloc)
+{
+ XT_Full xtf = {szB, 1, szB, 1, 0, 0};
+ VG_(XT_add_to_ec)(full_xt, ec_alloc, &xtf);
+}
+void VG_(XTMemory_Full_free)(SizeT szB,
+ ExeContext* ec_alloc,
+ ExeContext* ec_free)
+{
+ // substract from ec_alloc the freed memory.
+ XT_Full xtf_sub = {szB, 1, 0, 0, 0, 0};
+ VG_(XT_sub_from_ec)(full_xt, ec_alloc, &xtf_sub);
+
+ // add to ec_free the freed memory
+ XT_Full xtf_add = {0, 0, 0, 0, szB, 1};
+ VG_(XT_add_to_ec)(full_xt, ec_free, &xtf_add);
+}
+
+void VG_(XTMemory_Full_resize_in_place)(SizeT oldSzB, SizeT newSzB,
+ ExeContext* ec_alloc)
+{
+ if (oldSzB > newSzB) {
+ XT_Full xtf = {oldSzB - newSzB, 0, oldSzB - newSzB, 0, 0, 0};
+ VG_(XT_sub_from_ec)(full_xt, ec_alloc, &xtf);
+ } else {
+ XT_Full xtf = {newSzB - oldSzB, 0, newSzB - oldSzB, 0, 0, 0};
+ VG_(XT_add_to_ec)(full_xt, ec_alloc, &xtf);
+ }
+}
+
+// Indicates which event nr the report_value function must return.
+static UInt event_report_value_id;
+static ULong XT_Full_report_value(const void* xtfull)
+{
+ const XT_Full* xtf = xtfull;
+ switch (event_report_value_id) {
+ case 0: return (ULong) xtf->cur_alloc_nbytes;
+ case 1: return (ULong) xtf->cur_alloc_nblocks;
+ case 2: return xtf->tot_alloc_nbytes;
+ case 3: return xtf->tot_alloc_nblocks;
+ case 4: return xtf->tot_freed_nbytes;
+ case 5: return xtf->tot_freed_nblocks;
+ default: vg_assert(0);
+ }
+}
+static ULong XT_Allocs_report_value(const void* xt_allocs)
+{
+ const XT_Allocs* xta = xt_allocs;
+ switch (event_report_value_id) {
+ case 0: return (ULong) xta->nbytes;
+ case 1: return (ULong) xta->nblocks;
+ default: vg_assert(0);
+ }
+}
+
+static void produce_report(XTree* xt, const HChar* filename,
+ const HChar* events,
+ const HChar* (*img_value) (const void* value),
+ ULong (*report_value)(const void* value))
+{
+ /* The user can control the kind of report using filename extension. */
+ if (VG_(strstr)(filename, ".ms")) {
+ /* If needed, some harcoded value below could become parameters. */
+ MsFile* fp;
+ Massif_Header header = (Massif_Header) {
+ .snapshot_n = 0,
+ .time = VG_(read_millisecond_timer)(),
+ .sz_B = 0ul,
+ .extra_B = 0ul,
+ .stacks_B = 0ul,
+ .detailed = True,
+ .peak = False,
+ .top_node_desc = NULL,
+ .sig_threshold = 0.00000000000001
+ // Currently, we take a very small float value to not output
+ // the 0 values, but still output all the rest.
+ };
+
+ // Variables to parse events
+ HChar strtok_events[VG_(strlen)(events)+1];
+ HChar* e;
+ HChar* ssaveptr;
+
+ fp = VG_(XT_massif_open)(filename,
+ "xtree.produce_report",
+ NULL,
+ "ms");
+
+ event_report_value_id = 0;
+ VG_(strcpy)(strtok_events, events);
+ for (e = VG_(strtok_r) (strtok_events, ",", &ssaveptr);
+ e != NULL;
+ e = VG_(strtok_r) (NULL, ",", &ssaveptr)) {
+ header.top_node_desc = e;
+ VG_(XT_massif_print)(fp, xt, &header, report_value);
+ header.snapshot_n++;
+ event_report_value_id++;
+ }
+
+ VG_(XT_massif_close)(fp);
+ } else
+ VG_(XT_callgrind_print)(xt,
+ filename,
+ events,
+ img_value);
+}
+
+void VG_(XTMemory_report)
+ (const HChar* filename, Bool fini,
+ void (*next_block)(XT_Allocs* xta, ExeContext** ec_alloc),
+ XT_filter_IPs_t filter_IPs_fn)
+{
+ HChar* expanded_filename;
+
+ if (fini && VG_(clo_xtree_memory) == Vg_XTMemory_None)
+ return;
+
+ expanded_filename
+ = VG_(expand_file_name)("--xtree-memory-file",
+ (filename == NULL) ?
+ (fini ?
+ VG_(clo_xtree_memory_file) : "xtmemory.kcg")
+ : filename);
+
+ /* fini is False => even if user kept --xtree-memory=none, we
+ produce a report when explicitely requested e.g. via a monitor
+ command. */
+ switch (VG_(clo_xtree_memory)) {
+ case Vg_XTMemory_None:
+ case Vg_XTMemory_Allocs: {
+ XTree* xt;
+ XT_Allocs xta;
+ ExeContext* ec_alloc;
+
+ xt = VG_(XT_create) (VG_(malloc),
+ "VG_(XTMemory_report)",
+ VG_(free),
+ sizeof(XT_Allocs),
+ VG_(XT_Allocs_init),
+ VG_(XT_Allocs_add),
+ VG_(XT_Allocs_sub),
+ filter_IPs_fn);
+ (*next_block)(&xta, &ec_alloc);
+ while ( xta.nblocks > 0 ) {
+ VG_(XT_add_to_ec) (xt, ec_alloc, &xta);
+ (*next_block)(&xta, &ec_alloc);
+ }
+
+ produce_report(xt, expanded_filename,
+ XT_Allocs_events, VG_(XT_Allocs_img),
+ XT_Allocs_report_value);
+
+ VG_(XT_delete)(xt);
+ break;
+ }
+ case Vg_XTMemory_Full:
+ produce_report(full_xt, expanded_filename,
+ XT_Full_events, VG_(XT_Full_img),
+ XT_Full_report_value);
+ break;
+ default:
+ vg_assert(0);
+ }
+ if (VG_(clo_verbosity) >= 1 || !fini)
+ VG_(umsg)("xtree memory report: %s\n", expanded_filename);
+
+ VG_(free)(expanded_filename);
+}
+
+/*--------------------------------------------------------------------*/
+/*--- end m_xtree.c ---*/
+/*--------------------------------------------------------------------*/
Added: trunk/coregrind/m_xtree.c
==============================================================================
--- trunk/coregrind/m_xtree.c (added)
+++ trunk/coregrind/m_xtree.c Fri Nov 11 14:07:03 2016
@@ -0,0 +1,1013 @@
+
+/*--------------------------------------------------------------------*/
+/*--- An xtree, tree of stacktraces with data m_xtree.c ---*/
+/*--------------------------------------------------------------------*/
+
+/*
+ This file is part of Valgrind, a dynamic binary instrumentation
+ framework.
+
+ Copyright (C) 2016-2016 Philippe Waroquiers
+
+ This code generalises the XTree idea that was implemented in
+ the massif tool in Valgrind versions <= 3.12, which is
+ Copyright (C) 2005-2015 Nicholas Nethercote
+ nj...@va...
+
+ The XTree implementation in this file is however implemented completely
+ differently. Some code has been re-used for the production of the
+ massif file header (e.g. FP_cmd function).
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License as
+ published by the Free Software Foundation; either version 2 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
+ 02111-1307, USA.
+
+ The GNU General Public License is contained in the file COPYING.
+*/
+
+#include "pub_core_basics.h"
+#include "pub_core_debuglog.h"
+#include "pub_core_clientstate.h"
+#include "pub_core_stacktrace.h"
+#include "pub_core_execontext.h"
+#include "pub_core_libcbase.h"
+#include "pub_core_libcassert.h"
+#include "pub_core_libcfile.h"
+#include "pub_core_libcprint.h"
+#include "pub_core_libcproc.h"
+#include "pub_core_hashtable.h"
+#include "pub_core_mallocfree.h"
+#include "pub_core_options.h"
+#include "pub_core_debuginfo.h"
+#include "pub_core_deduppoolalloc.h"
+#include "pub_core_xtree.h" /* self */
+
+#define DMSG(level, ...) (level <= VG_(debugLog_getLevel)() ? \
+ VG_(dmsg)(__VA_ARGS__) \
+ : 0)
+
+/* Defines the relevant part of an ec. This is shared between an xt
+ and its snapshots (see XT_shared XArray of xec). */
+typedef
+ struct _xec {
+ ExeContext* ec;
+ UShort top; // The first ips of ec to take into account.
+ UShort n_ips_sel; // The nr of ips from top to take into account.
+ }
+ xec;
+
+/* XT_shared maintains the information shared between an XT and all
+ its snapshots. */
+typedef
+ struct _XT_shared {
+ UWord nrRef; /* nr of XTrees referencing this shared memory. */
+
+ void* (*alloc_fn)( const HChar*, SizeT ); /* alloc fn (nofail) */
+ const HChar* cc; /* cost centre for alloc */
+ void (*free_fn)( void* ); /* free fn */
+
+ /* The data associated to each ec is stored in 2 arrays:
+ an xec array, shared between an xt and all its snapshots.
+ a data array, private to each XTree.
+ For an ec with an ECU ecu, d4ecu2xecu[ecu/4] gives the offset in
+ xec and data arrays where the ec information is located (this
+ indirection is used to avoid huge xec and data arrays, in
+ case an XTree contains data only for a small number of ec.
+ The offset in the xec and data array is used as xtree ec unique
+ id i.e. an xecu. */
+
+ UInt d4ecu2xecu_sz; /* size of d4ecu2xecu (in nr of elements). */
+ UInt* d4ecu2xecu;
+
+ /* ec information common to an xt and its snapshots. */
+ XArray* xec; /* XArray of xec, indexed by xecu (== d4ecu2xecu[ecu/4]). */
+
+ /* XArray of xecu, sorted by StackTrace ips[top..top+n_ips_sel-1].
+ See ips_order_cmp. */
+ XArray* ips_order_xecu;
+ } XT_shared;
+
+/* NO_OFFSET indicates in d4ecu2xecu there is no data (yet) for this ec
+ (with the index ecu/4). */
+#define NO_OFFSET 0xffffffff
+
+static XT_shared* new_XT_shared (void* (*alloc_fn)(const HChar*, SizeT),
+ const HChar* cc,
+ void (*free_fn)(void*))
+{
+ XT_shared* shared;
+
+ vg_assert(alloc_fn);
+ vg_assert(cc);
+ vg_assert(free_fn);
+ shared = alloc_fn(cc, sizeof(*shared));
+ shared->nrRef = 0;
+ shared->alloc_fn = alloc_fn;
+ shared->cc = cc;
+ shared->free_fn = free_fn;
+
+ shared->d4ecu2xecu_sz = 0;
+ shared->d4ecu2xecu = NULL;
+ shared->xec = VG_(newXA)(alloc_fn, cc, free_fn, sizeof(xec));
+ shared->ips_order_xecu = NULL; // Allocated when needed.
+
+ return shared;
+}
+
+static void delete_XT_shared (XT_shared* shared)
+{
+ vg_assert(shared->nrRef == 0);
+ shared->free_fn(shared->d4ecu2xecu);
+ VG_(deleteXA)(shared->xec);
+ if (shared->ips_order_xecu != NULL)
+ VG_(deleteXA)(shared->ips_order_xecu);
+ shared->free_fn(shared);
+}
+
+/* Compare 2 entries in ips_order_xecu by StackTrace elements. Note
+ that a not existing ips is considered smaller than any other
+ address. */
+static XArray* xec_data_for_sort; // Needed to translate an xecu into an xec
+static Int ips_order_cmp(const void* vleft, const void* vright)
+{
+ const Xecu left_xecu = *(const Xecu*)vleft;
+ const Xecu right_xecu = *(const Xecu*)vright;
+ const xec* left = VG_(indexXA)(xec_data_for_sort, left_xecu);
+ const xec* right = VG_(indexXA)(xec_data_for_sort, right_xecu);
+ const StackTrace left_ips = VG_(get_ExeContext_StackTrace)(left->ec)
+ + left->top;
+ const StackTrace right_ips = VG_(get_ExeContext_StackTrace)(right->ec)
+ + right->top;
+ UInt i;
+
+ const UInt c_n_ips_sel = left->n_ips_sel < right->n_ips_sel
+ ? left->n_ips_sel : right->n_ips_sel;
+
+ // First see if we have a difference on the common nr of ips selected
+ for (i = 0; i < c_n_ips_sel; i++) {
+ if (left_ips[i] == right_ips[i]) continue;
+ if (left_ips[i] < right_ips[i]) return -1;
+ return 1;
+ }
+ // Common part is equal => compare lengths.
+ if (left->n_ips_sel < right->n_ips_sel) return -1;
+ if (left->n_ips_sel > right->n_ips_sel) return 1;
+ return 0;
+}
+
+// If needed, build or refresh shared->ips_order_xecu
+static void ensure_ips_order_xecu_valid(XT_shared* shared)
+{
+ UInt i;
+ UInt n_xecu;
+
+ if (shared->ips_order_xecu == NULL) {
+ shared->ips_order_xecu = VG_(newXA)(shared->alloc_fn, shared->cc,
+ shared->free_fn, sizeof(Xecu));
+ VG_(hintSizeXA)(shared->ips_order_xecu, VG_(sizeXA)(shared->xec));
+ VG_(setCmpFnXA)(shared->ips_order_xecu, ips_order_cmp);
+ }
+
+ if (VG_(sizeXA)(shared->xec) == VG_(sizeXA)(shared->ips_order_xecu))
+ return;
+
+ n_xecu = VG_(sizeXA)(shared->xec);
+ for (i = VG_(sizeXA)(shared->ips_order_xecu); i < n_xecu; i++)
+ VG_(addToXA)(shared->ips_order_xecu, &i);
+
+ xec_data_for_sort = shared->xec;
+ VG_(sortXA)(shared->ips_order_xecu);
+}
+
+static void addRef_XT_shared (XT_shared* shared)
+{
+ shared->nrRef++;
+}
+
+static UWord release_XT_shared(XT_shared* shared)
+{
+ UWord nrRef;
+
+ vg_assert(shared->nrRef > 0);
+ nrRef = --shared->nrRef;
+ if (nrRef == 0)
+ delete_XT_shared(shared);
+ return nrRef;
+}
+
+
+struct _XTree {
+ void* (*alloc_fn)( const HChar*, SizeT ); /* alloc fn (nofail) */
+ const HChar* cc; /* cost centre for alloc */
+ void (*free_fn)( void* ); /* free fn */
+ Word dataSzB; /* data size in bytes */
+ XT_init_data_t init_data_fn;
+ XT_add_data_t add_data_fn;
+ XT_sub_data_t sub_data_fn;
+ XT_filter_IPs_t filter_IPs_fn;
+
+ XT_shared* shared;
+
+ HChar* tmp_data; /* temporary buffer, to insert new elements. */
+ XArray* data; /* of elements of size dataSzB */
+};
+
+
+XTree* VG_(XT_create) ( void*(*alloc_fn)(const HChar*, SizeT),
+ const HChar* cc,
+ void(*free_fn) (void*),
+ Word dataSzB,
+ XT_init_data_t init_data_fn,
+ XT_add_data_t add_data_fn,
+ XT_sub_data_t sub_data_fn,
+ XT_filter_IPs_t filter_IPs_fn)
+{
+ XTree* xt;
+
+ /* check user-supplied info .. */
+ vg_assert(alloc_fn);
+ vg_assert(free_fn);
+ vg_assert(dataSzB >= 0);
+ vg_assert(init_data_fn);
+ vg_assert(add_data_fn);
+ vg_assert(sub_data_fn);
+
+ xt = alloc_fn(cc, sizeof(struct _XTree) );
+ xt->alloc_fn = alloc_fn;
+ xt->cc = cc;
+ xt->free_fn = free_fn;
+ xt->dataSzB = dataSzB;
+ xt->init_data_fn = init_data_fn;
+ xt->add_data_fn = add_data_fn;
+ xt->sub_data_fn = sub_data_fn;
+ xt->filter_IPs_fn = filter_IPs_fn;
+
+ xt->shared = new_XT_shared(alloc_fn, cc, free_fn);
+ addRef_XT_shared(xt->shared);
+ xt->tmp_data = alloc_fn(cc, xt->dataSzB);
+ xt->data = VG_(newXA)(alloc_fn, cc, free_fn, dataSzB);
+
+ return xt;
+}
+
+XTree* VG_(XT_snapshot)(XTree* xt)
+{
+ XTree* nxt;
+
+ vg_assert(xt);
+
+ nxt = xt->alloc_fn(xt->cc, sizeof(struct _XTree) );
+
+ *nxt = *xt;
+ addRef_XT_shared (nxt->shared);
+ nxt->tmp_data = nxt->alloc_fn(nxt->cc, nxt->dataSzB);
+ nxt->data = VG_(cloneXA)(nxt->cc, xt->data);
+
+ return nxt;
+}
+
+void VG_(XT_delete) ( XTree* xt )
+{
+ vg_assert(xt);
+
+ release_XT_shared(xt->shared);
+ xt->free_fn(xt->tmp_data);
+ VG_(deleteXA)(xt->data);
+ xt->free_fn(xt);
+}
+
+static Xecu find_or_insert (XTree* xt, ExeContext* ec)
+{
+
+ const UInt d4ecu = VG_(get_ECU_from_ExeContext)(ec) / 4;
+ XT_shared* shared = xt->shared;
+
+ /* First grow the d4ecu2xecu array if needed. */
+ if (d4ecu >= shared->d4ecu2xecu_sz) {
+ UInt old_sz = shared->d4ecu2xecu_sz;
+ UInt new_sz = (3 * d4ecu) / 2;
+
+ if (new_sz < 1000)
+ new_sz = 1000;
+ shared->d4ecu2xecu = VG_(realloc)(xt->cc, shared->d4ecu2xecu,
+ new_sz * sizeof(UInt));
+ shared->d4ecu2xecu_sz = new_sz;
+ for (UInt i = old_sz; i < new_sz; i++)
+ shared->d4ecu2xecu[i] = NO_OFFSET;
+ }
+
+ if (shared->d4ecu2xecu[d4ecu] == NO_OFFSET) {
+ xec xe;
+
+ xe.ec = ec;
+ if (xt->filter_IPs_fn == NULL) {
+ xe.top = 0;
+ xe.n_ips_sel = (UShort)VG_(get_ExeContext_n_ips)(xe.ec);
+ } else {
+ UInt top;
+ UInt n_ips_sel = VG_(get_ExeContext_n_ips)(xe.ec);
+ xt->filter_IPs_fn(VG_(get_ExeContext_StackTrace)(xe.ec), n_ips_sel,
+ &top, &n_ips_sel);
+ xe.top = (UShort)top;
+ xe.n_ips_sel = (UShort)n_ips_sel;
+ }
+ xt->init_data_fn(xt->tmp_data);
+ VG_(addToXA)(shared->xec, &xe);
+ shared->d4ecu2xecu[d4ecu] = (UInt)VG_(addToXA)(xt->data, xt->tmp_data);
+ }
+
+ return shared->d4ecu2xecu[d4ecu];
+}
+
+Xecu VG_(XT_add_to_ec) (XTree* xt, ExeContext* ec, const void* value)
+{
+ Xecu xecu = find_or_insert(xt, ec);
+ void* data = VG_(indexXA)(xt->data, xecu);
+
+ xt->add_data_fn(data, value);
+ return xecu;
+}
+
+Xecu VG_(XT_sub_from_ec) (XTree* xt, ExeContext* ec, const void* value)
+{
+ Xecu xecu = find_or_insert(xt, ec);
+ void* data = VG_(indexXA)(xt->data, xecu);
+
+ xt->sub_data_fn(data, value);
+ return xecu;
+}
+
+void VG_(XT_add_to_xecu) (XTree* xt, Xecu xecu, const void* value)
+{
+ void* data = VG_(indexXA)(xt->data, xecu);
+ xt->add_data_fn(data, value);
+}
+
+void VG_(XT_sub_from_xecu) (XTree* xt, Xecu xecu, const void* value)
+{
+ void* data = VG_(indexXA)(xt->data, xecu);
+ xt->sub_data_fn(data, value);
+}
+
+UInt VG_(XT_n_ips_sel) (XTree* xt, Xecu xecu)
+{
+ xec* xe = (xec*)VG_(indexXA)(xt->shared->xec, xecu);
+ return (UInt)xe->n_ips_sel;
+}
+
+ExeContext* VG_(XT_get_ec_from_xecu) (XTree* xt, Xecu xecu)
+{
+ xec* xe = (xec*)VG_(indexXA)(xt->shared->xec, xecu);
+ return xe->ec;
+}
+
+static VgFile* xt_open (const HChar* outfilename)
+{
+ VgFile* fp;
+
+ fp = VG_(fopen)(outfilename, VKI_O_CREAT|VKI_O_WRONLY|VKI_O_TRUNC,
+ VKI_S_IRUSR|VKI_S_IWUSR);
+ if (fp == NULL) {
+ VG_(message)(Vg_UserMsg,
+ "Error: can not open xtree output file `%s'\n",
+ outfilename);
+ }
+ return fp;
+}
+
+#define FP(format, args...) ({ VG_(fprintf)(fp, format, ##args); })
+
+// Print "cmd:" line.
+static void FP_cmd(VgFile* fp)
+{
+ UInt i;
+
+ FP("cmd: ");
+ FP("%s", VG_(args_the_exename));
+ for (i = 0; i < VG_(sizeXA)(VG_(args_for_client)); i++) {
+ HChar* arg = * (HChar**) VG_(indexXA)(VG_(args_for_client), i);
+ FP(" %s", arg);
+ }
+ FP("\n");
+}
+
+/* ----------- Callgrind output ------------------------------------------- */
+
+/* Output a callgrind file element in compressed or not compressed format,
+ according to VG_(clo_xtree_compress_strings). */
+static void FP_pos_str(VgFile* fp, const HChar* name, UInt pos,
+ const HChar* value, Bool value_new)
+{
+ if (!VG_(clo_xtree_compress_strings))
+ FP("%s=%s\n", name, value);
+ else if (value_new)
+ FP("%s=(%d) %s\n", name, pos, value);
+ else
+ FP("%s=(%d)\n", name, pos);
+}
+
+void VG_(XT_callgrind_print)
+ (XTree* xt,
+ const HChar* outfilename,
+ const HChar* events,
+ const HChar* (*img_value)(const void* value))
+{
+ UInt n_xecu;
+ XT_shared* shared = xt->shared;
+ VgFile* fp = xt_open(outfilename);
+ DedupPoolAlloc* fnname_ddpa;
+ DedupPoolAlloc* filename_ddpa;
+
+ if (fp == NULL)
+ return;
+
+ fnname_ddpa = VG_(newDedupPA)(16000, 1, xt->alloc_fn,
+ "XT_callgrind_print.fn", xt->free_fn);
+ filename_ddpa = VG_(newDedupPA)(16000, 1, xt->alloc_fn,
+ "XT_callgrind_print.fl", xt->free_fn);
+
+ FP("version: 1\n");
+ FP("creator: xtree-1\n");
+ FP("pid: %d\n", VG_(getpid)());
+ FP_cmd(fp);
+
+ /* Currently, we only need/support line positions. */
+ FP("\npositions:%s\n", " line");
+
+ /* Produce one "event:" line for each event, and the "events:" line. */
+ {
+ HChar strtok_events[VG_(strlen)(events)+1];
+ HChar* e;
+ HChar* ssaveptr;
+ HChar* p;
+
+ VG_(strcpy)(strtok_events, events);
+ for (e = VG_(strtok_r) (strtok_events, ",", &ssaveptr);
+ e != NULL;
+ e = VG_(strtok_r) (NULL, ",", &ssaveptr))
+ FP("event: %s\n", e);
+ FP("events:");
+ VG_(strcpy)(strtok_events, events);
+ for (e = VG_(strtok_r) (strtok_events, ",", &ssaveptr);
+ e != NULL;
+ e = VG_(strtok_r) (NULL, ",", &ssaveptr)) {
+ p = e;
+ while (*p) {
+ if (*p == ':')
+ *p = 0;
+ p++;
+ }
+ FP(" %s", e);
+ }
+ FP("\n");
+ }
+ xt->init_data_fn(xt->tmp_data); // to compute totals
+
+ n_xecu = VG_(sizeXA)(xt->data);
+ vg_assert (n_xecu <= VG_(sizeXA)(shared->xec));
+ for (Xecu xecu = 0; xecu < n_xecu; xecu++) {
+ xec* xe = (xec*)VG_(indexXA)(shared->xec, xecu);
+ if (xe->n_ips_sel == 0)
+ continue;
+
+ const HChar* img = img_value(VG_(indexXA)(xt->data, xecu));
+
+#define CALLED_FLF(n) \
+ if ((n) < 0 \
+ || !VG_(get_filename_linenum)(ips[(n)], \
+ &called_filename, \
+ NULL, \
+ &called_linenum)) { \
+ called_filename = "file ???"; \
+ called_linenum = 0; \
+ } \
+ if ((n) < 0 \
+ || !VG_(get_fnname)(ips[(n)], &called_fnname)) { \
+ called_fnname = "???"; \
+ } \
+ called_filename_nr = VG_(allocStrDedupPA)(filename_ddpa, \
+ called_filename, \
+ &called_filename_new); \
+ called_fnname_nr = VG_(allocStrDedupPA)(fnname_ddpa, \
+ called_fnname, \
+ &called_fnname_new);
+
+ /* Instead of unknown fnname ???, CALLED_FLF could use instead:
+ VG_(sprintf)(unknown_fn, "%p", (void*)ips[(n)]);
+ but that creates a lot of (useless) nodes at least for
+ valgrind self-hosting. */
+
+ if (img) {
+ const HChar* called_filename;
+ UInt called_filename_nr;
+ Bool called_filename_new;
+ const HChar* called_fnname;
+ UInt called_fnname_nr;
+ Bool called_fnname_new;
+ UInt called_linenum;
+ UInt prev_linenum;
+
+ const Addr* ips = VG_(get_ExeContext_StackTrace)(xe->ec) + xe->top;
+ Int ips_idx = xe->n_ips_sel - 1;
+
+ if (0) {
+ VG_(printf)("entry img %s\n", img);
+ VG_(pp_ExeContext)(xe->ec);
+ VG_(printf)("\n");
+ }
+ xt->add_data_fn(xt->tmp_data, VG_(indexXA)(xt->data, xecu));
+ CALLED_FLF(ips_idx);
+ for (;
+ ips_idx >= 0;
+ ips_idx--) {
+ FP_pos_str(fp, "fl", called_filename_nr,
+ called_filename, called_filename_new);
+ FP_pos_str(fp, "fn", called_fnname_nr,
+ called_fnname, called_fnname_new);
+ if (ips_idx == 0)
+ FP("%d %s\n", called_linenum, img);
+ else
+ FP("%d\n", called_linenum); //no self cost.
+ prev_linenum = called_linenum;
+ CALLED_FLF (ips_idx-1);
+ if (ips_idx >= 1) {
+ FP_pos_str(fp, "cfi", called_filename_nr,
+ called_filename, called_filename_new);
+ FP_pos_str(fp, "cfn", called_fnname_nr,
+ called_fnname, called_fnname_new);
+ called_filename_new = False;
+ called_fnname_new = False;
+ /* Giving a call count of 0 allows kcachegrind to hide the calls
+ column. A call count of 1 means kcachegrind would show in the
+ calls column the nr of stacktrace containing this arc, which
+ is very confusing. So, the less bad is to give a 0 call
+ count. */
+ FP("calls=0 %d\n", called_linenum);
+ FP("%d %s\n", prev_linenum, img);
+ }
+ }
+ FP("\n");
+ }
+ }
+ /* callgrind format is not really fully supporting (yet?) execution trees:
+ in an execution tree, self and inclusive costs are identical, and
+ cannot be added together.
+ If no totals: line is given, callgrind_annotate calculates the addition
+ of all costs, and so gives a wrong totals.
+ Giving a totals: line solves this, but the user must give the option
+ --inclusive=yes (kind of hack) to have all the functions given
+ in the output file. */
+ FP("totals: %s\n", img_value(xt->tmp_data));
+ VG_(fclose)(fp);
+ VG_(deleteDedupPA)(fnname_ddpa);
+ VG_(deleteDedupPA)(filename_ddpa);
+}
+
+
+/* ----------- Massif output ---------------------------------------------- */
+
+/* For Massif output, some functions from the execontext are not output, a.o.
+ the allocation functions at the top of the stack and the functions below
+ main. So, the StackTrace of the execontexts in the xtree must be filtered.
+ The functions below main.
+ Ms_Ec defines the subset of the stacktrace relevant for the report. */
+typedef
+ struct {
+ StackTrace ips; // ips and n_ips provides the subset of the xtree ec
+ UInt n_ips; // to use for a massif report.
+
+ SizeT report_value; // The value to report for this stack trace.
+ } Ms_Ec;
+
+/* Ms_Group defines (at a certain depth) a group of ec context that
+ have the same IPs at the given depth, and have the same 'parent'.
+ total is the sum of the values of all group elements.
+ A Ms_Group can also represent a set of ec contexts that do not
+ have the same IP, but that have each a total which is below the
+ significant size. Such a group has a NULL ms_ec, a zero group_io.
+ n_ec is the nr of insignificant ec that have been collected inside this
+ insignificant group, and total is the sum of all non significant ec
+ at the given depth. */
+typedef
+ struct {
+ Ms_Ec* ms_ec; // The group consists in ms_ec[0 .. n_ec-1]
+ Addr group_ip;
+ UInt n_ec;
+ SizeT total;
+ } Ms_Group;
+
+/* Compare 2 groups by total, to have bigger total first. */
+static Int ms_group_revcmp_total(const void* vleft, const void* vright)
+{
+ const Ms_Group* left = (const Ms_Group*)vleft;
+ const Ms_Group* right = (const Ms_Group*)vright;
+
+ // First reverse compare total
+ if (left->total > right->total) return -1;
+ if (left->total < right->total) return 1;
+
+ /* Equal size => compare IPs.
+ This (somewhat?) helps to have deterministic test results.
+ If this would change between platforms, then we should compare
+ function names/filename/linenr */
+ if (left->group_ip < right->group_ip) return -1;
+ if (left->group_ip > right->group_ip) return 1;
+ return 0;
+}
+
+/* Scan the addresses in ms_ec at the given depth.
+ On return,
+ *groups points to an array of Ms_Group sorted by total.
+ *n_groups is the nr of groups
+ The caller is responsible to free the allocated group array. */
+static void ms_make_groups (UInt depth, Ms_Ec* ms_ec, UInt n_ec, SizeT sig_sz,
+ UInt* n_groups, Ms_Group** groups)
+{
+ UInt i, g;
+ Addr cur_group_ip = 0;
+
+ *n_groups = 0;
+
+ /* Handle special case somewhat more efficiently */
+ if (n_ec == 0) {
+ *groups = NULL;
+ return;
+ }
+
+ /* Compute how many groups we have. */
+ for (i = 0; i < n_ec; i++) {
+ if (ms_ec[i].n_ips > depth
+ && (*n_groups == 0 || cur_group_ip != ms_ec[i].ips[depth])) {
+ (*n_groups)++;
+ cur_group_ip = ms_ec[i].ips[depth];
+ }
+ }
+
+ /* make the group array. */
+ *groups = VG_(malloc)("ms_make_groups", *n_groups * sizeof(Ms_Group));
+ i = 0;
+ for (g = 0; g < *n_groups; g++) {
+ while (ms_ec[i].n_ips <= depth)
+ i++;
+ cur_group_ip = ms_ec[i].ips[depth];
+ (*groups)[g].group_ip = cur_group_ip;
+ (*groups)[g].ms_ec = &ms_ec[i];
+ (*groups)[g].n_ec = 1;
+ (*groups)[g].total = ms_ec[i].report_value;
+ i++;
+ while (i < n_ec
+ && ms_ec[i].n_ips > depth
+ && cur_group_ip == ms_ec[i].ips[depth]) {
+ (*groups)[g].total += ms_ec[i].report_value;
+ i++;
+ (*groups)[g].n_ec++;
+ }
+ }
+
+ /* Search for insignificant groups, collect them all together
+ in the first insignificant group, and compact the group array. */
+ {
+ UInt insig1; // Position of first insignificant group.
+ UInt n_insig = 0; // Nr of insignificant groups found.
+
+ for (g = 0; g < *n_groups; g++) {
+ if ((*groups)[g].total < sig_sz) {
+ if (n_insig == 0) {
+ // First insig group => transform it into the special group
+ (*groups)[g].ms_ec = NULL;
+ (*groups)[g].group_ip = 0;
+ (*groups)[g].n_ec = 0;
+ // start the sum of insig total as total
+ insig1 = g;
+ } else {
+ // Add this insig group total into insig1 first group
+ (*groups)[insig1].total += (*groups)[g].total;
+ }
+ n_insig++;
+ } else {
+ if (n_insig > 1)
+ (*groups)[g - n_insig + 1] = (*groups)[g];
+ }
+ }
+ if (n_insig > 0) {
+ (*groups)[insig1].n_ec = n_insig;
+ *n_groups -= n_insig - 1;
+ }
+ DMSG(1, "depth %u n_groups %u n_insig %u\n", depth, *n_groups, n_insig);
+ }
+
+ /* Sort on total size, bigger size first. */
+ VG_(ssort) (*groups, *n_groups, sizeof(Ms_Group), ms_group_revcmp_total);
+}
+
+static void ms_output_group (VgFile* fp, UInt depth, Ms_Group* group,
+ SizeT sig_sz, double sig_pct_threshold)
+{
+ UInt i;
+ Ms_Group* groups;
+ UInt n_groups;
+
+ // If this is an insignificant group, handle it specially
+ if (group->ms_ec == NULL) {
+ const HChar* s = ( 1 == group->n_ec? "," : "s, all" );
+ vg_assert (group->group_ip == 0);
+ FP("%*sn0: %lu in %d place%s below massif's threshold (%.2f%%)\n",
+ depth+1, "", group->total, group->n_ec, s, sig_pct_threshold);
+ return;
+ }
+
+ // Normal group => output the group and its subgroups.
+ ms_make_groups (depth+1, group->ms_ec, group->n_ec, sig_sz,
+ &n_groups, &groups);
+
+ FP("%*s" "n%u: %ld %s\n",
+ depth + 1, "",
+ n_groups,
+ group->total,
+ VG_(describe_IP)(group->ms_ec->ips[depth] - 1, NULL));
+ /* XTREE??? Massif original code removes 1 to get the IP description. I am
+ wondering if this is not something that predates revision r8818,
+ which introduced a -1 in the stack unwind (see m_stacktrace.c)
+ Kept for the moment to allow exact comparison with massif output, but
+ probably we should remove this, as we very probably end up 2 bytes before
+ the RA Return Address. */
+
+ /* Output sub groups of this group. */
+ for (i = 0; i < n_groups; i++)
+ ms_output_group (fp, depth+1, &groups[i], sig_sz, sig_pct_threshold);
+
+ VG_(free)(groups);
+}
+
+/* Allocate and build an array of Ms_Ec sorted by addresses in the
+ Ms_Ec StackTrace. */
+static void prepare_ms_ec (XTree* xt,
+ ULong (*report_value)(const void* value),
+ ULong* top_total, Ms_Ec** vms_ec, UInt* vn_ec)
+{
+ XT_shared* shared = xt->shared;
+ const UInt n_xecu = VG_(sizeXA)(shared->xec);
+ const UInt n_data_xecu = VG_(sizeXA)(xt->data);
+ Ms_Ec* ms_ec = VG_(malloc)("XT_massif_print.ms_ec", n_xecu * sizeof(Ms_Ec));
+ UInt n_xecu_sel = 0; // Nr of xecu that are selected for output.
+
+ vg_assert (n_data_xecu <= n_xecu);
+
+ // Ensure we have in shared->ips_order_xecu our xecu sorted by StackTrace.
+ ensure_ips_order_xecu_valid(shared);
+
+ *top_total = 0;
+ DMSG(1, "iteration %u\n", n_xecu);
+ for (UInt i = 0; i < n_xecu; i++) {
+ Xecu xecu = *(Xecu*)VG_(indexXA)(shared->ips_order_xecu, i);
+ xec* xe = (xec*)VG_(indexXA)(shared->xec, xecu);
+
+ if (xecu >= n_data_xecu)
+ continue; // No data for this xecu in xt->data.
+ ms_ec[n_xecu_sel].n_ips = xe->n_ips_sel;
+ if (ms_ec[n_xecu_sel].n_ips == 0)
+ continue;
+
+ ms_ec[n_xecu_sel].ips = VG_(get_ExeContext_StackTrace)(xe->ec) + xe->top;
+ ms_ec[n_xecu_sel].report_value
+ = (*report_value)(VG_(indexXA)(xt->data, xecu));
+ *top_total += ms_ec[n_xecu_sel].report_value;
+
+ n_xecu_sel++;
+ }
+ vg_assert(n_xecu_sel <= n_xecu);
+
+ *vms_ec = ms_ec;
+ *vn_ec = n_xecu_sel;
+}
+
+MsFile* VG_(XT_massif_open)
+ (const HChar* outfilename,
+ const HChar* desc,
+ const XArray* desc_args,
+ const HChar* time_unit)
+{
+ UInt i;
+ VgFile* fp = xt_open(outfilename);
+
+ if (fp == NULL)
+ return NULL; // xt_open reported the error.
+
+ /* ------ file header ------------------------------- */
+ FP("desc:");
+ if (desc)
+ FP(" %s", desc);
+ i = 0;
+ if (desc_args) {
+ for (i = 0; i < VG_(sizeXA)(desc_args); i++) {
+ HChar* arg = *(HChar**)VG_(indexXA)(desc_args, i);
+ FP(" %s", arg);
+ }
+ }
+ if (0 == i && desc == NULL) FP(" (none)");
+ FP("\n");
+
+ FP_cmd(fp);
+
+ FP("time_unit: %s\n", time_unit);
+
+ return fp;
+}
+
+void VG_(XT_massif_close)(MsFile* fp)
+{
+ if (fp == NULL)
+ return; // Error should have been reported by VG_(XT_massif_open)
+
+ VG_(fclose)(fp);
+}
+
+void VG_(XT_massif_print)
+ (MsFile* fp,
+ XTree* xt,
+ const Massif_Header* header,
+ ULong (*report_value)(const void* value))
+{
+ UInt i;
+
+ if (fp == NULL)
+ return; // Normally VG_(XT_massif_open) already reported an error.
+
+ /* Compute/prepare Snapshot totals/data/... */
+ ULong top_total;
+
+ /* Following variables only used for detailed snapshot. */
+ UInt n_ec = 0;
+ Ms_Ec* ms_ec = NULL;
+ const HChar* kind =
+ header->detailed ? (header->peak ? "peak" : "detailed") : "empty";
+
+ DMSG(1, "XT_massif_print %s\n", kind);
+ if (header->detailed) {
+ /* Prepare the Ms_Ec sorted array of stacktraces and the groups
+ at level 0. */
+ prepare_ms_ec(xt, report_value, &top_total, &ms_ec, &n_ec);
+ DMSG(1, "XT_print_massif ms_ec n_ec %u\n", n_ec);
+ } else if (xt == NULL) {
+ /* Non detailed, no xt => use the sz provided in the header. */
+ top_total = header->sz_B;
+ } else {
+ /* For non detailed snapshot, compute total directly from the xec. */
+ const XT_shared* shared = xt->shared;
+ const UInt n_xecu = VG_(sizeXA)(xt->data);
+ top_total = 0;
+
+ for (UInt xecu = 0; xecu < n_xecu; xecu++) {
+ xec* xe = (xec*)VG_(indexXA)(shared->xec, xecu);
+ if (xe->n_ips_sel == 0)
+ continue;
+ top_total += (*report_value)(VG_(indexXA)(xt->data, xecu));
+ }
+ }
+
+ /* ------ snapshot header --------------------------- */
+ FP("#-----------\n");
+ FP("snapshot=%d\n", header->snapshot_n);
+ FP("#-----------\n");
+ FP("time=%lld\n", header->time);
+
+ FP("mem_heap_B=%llu\n", top_total); // without extra_B and without stacks_B
+ FP("mem_heap_extra_B=%llu\n", header->extra_B);
+ FP("mem_stacks_B=%llu\n", header->stacks_B);
+ FP("heap_tree=%s\n", kind);
+
+ /* ------ detailed snapshot data ----------------------------- */
+ if (header->detailed) {
+ UInt n_groups;
+ Ms_Group* groups;
+
+ ULong sig_sz;
+ // Work out how big a child must be to be significant. If the current
+ // top_total is zero, then we set it to 1, which means everything will be
+ // judged insignificant -- this is sensible, as there's no point showing
+ // any detail for this case. Unless they used threshold=0, in which
+ // case we show them everything because that's what they asked for.
+ //
+ // Nb: We do this once now, rather than once per child, because if we do
+ // that the cost of all the divisions adds up to something significant.
+ if (0 == top_total && 0 != header->sig_threshold)
+ sig_sz = 1;
+ else
+ sig_sz = ((top_total + header->extra_B + header->stacks_B)
+ * header->sig_threshold) / 100;
+
+ /* Produce the groups at depth 0 */
+ DMSG(1, "XT_massif_print producing depth 0 groups\n");
+ ms_make_groups (0, ms_ec, n_ec, sig_sz, &n_groups, &groups);
+
+ /* Output the top node. */
+ FP("n%u: %llu %s\n", n_groups, top_total, header->top_node_desc);
+
+ /* Output depth 0 groups. */
+ DMSG(1, "XT_massif_print outputing %u depth 0 groups\n", n_groups);
+ for (i = 0; i < n_groups; i++)
+ ms_output_group (fp, 0, &groups[i], sig_sz, header->sig_threshold);
+
+ VG_(free)(groups);
+ VG_(free)(ms_ec);
+ }
+}
+
+Int VG_(XT_offset_main_or_below_main)(Addr* ips, Int n_ips)
+{
+ /* Search for main or below main function.
+ To limit the nr of ips to examine, we maintain the deepest
+ offset where main was found, and we first search main
+ from there.
+ If no main is found, we will then do a search for main or
+ below main function till the top. */
+ static Int deepest_main = 0;
+ Vg_FnNameKind kind = Vg_FnNameNormal;
+ Int mbm = n_ips - 1; // Position of deepest main or below main.
+ Vg_FnNameKind mbmkind = Vg_FnNameNormal;
+ Int i;
+
+ for (i = n_ips - 1 - deepest_main;
+ i < n_ips;
+ i++) {
+ mbmkind = VG_(get_fnname_kind_from_IP)(ips[i]);
+ if (mbmkind != Vg_FnNameNormal) {
+ mbm = i;
+ break;
+ }
+ }
+
+ /* Search for main or below main function till top. */
+ for (i = mbm - 1;
+ i >= 0 && mbmkind != Vg_FnNameMain;
+ i--) {
+ kind = VG_(get_fnname_kind_from_IP)(ips[i]);
+ if (kind != Vg_FnNameNormal) {
+ mbm = i;
+ mbmkind = kind;
+ }
+ }
+ if (Vg_FnNameMain == mbmkind || Vg_FnNameBelowMain == mbmkind) {
+ if (mbmkind == Vg_FnNameMain && (n_ips - 1 - mbm) > deepest_main)
+ deepest_main = n_ips - 1 - mbm;
+ return mbm;
+ } else
+ return n_ips-1;
+}
+
+void VG_(XT_filter_1top_and_maybe_below_main)
+ (Addr* ips, Int n_ips,
+ UInt* top, UInt* n_ips_sel)
+{
+ Int mbm;
+
+ *n_ips_sel = n_ips;
+ if (n_ips == 0) {
+ *top = 0;
+ return;
+ }
+
+ /* Filter top function. */
+ *top = 1;
+
+ if (VG_(clo_show_below_main))
+ mbm = n_ips - 1;
+ else
+ mbm = VG_(XT_offset_main_or_below_main)(ips, n_ips);
+
+ *n_ips_sel = mbm - *top + 1;
+}
+
+void VG_(XT_filter_maybe_below_main)
+ (Addr* ips, Int n_ips,
+ UInt* top, UInt* n_ips_sel)
+{
+ Int mbm;
+
+ *n_ips_sel = n_ips;
+ *top = 0;
+ if (n_ips == 0)
+ return;
+
+ if (VG_(clo_show_below_main))
+ mbm = n_ips - 1;
+ else
+ mbm = VG_(XT_offset_main_or_below_main)(ips, n_ips);
+
+ *n_ips_sel = mbm - *top + 1;
+}
+
+/*--------------------------------------------------------------------*/
+/*--- end m_xtree.c ---*/
+/*--------------------------------------------------------------------*/
Modified: trunk/coregrind/pub_core_options.h
==============================================================================
--- trunk/coregrind/pub_core_options.h (original)
+++ trunk/coregrind/pub_core_options.h Fri Nov 11 14:07:03 2016
@@ -207,7 +207,6 @@
extern Int VG_(clo_core_redzone_size);
// VG_(clo_redzone_size) has default value -1, indicating to keep
// the tool provided value.
-extern Int VG_(clo_redzone_size);
/* DEBUG: display gory details for the k'th most popular error.
default: Infinity. */
extern Int VG_(clo_dump_error);
Added: trunk/coregrind/pub_core_xtmemory.h
==============================================================================
--- trunk/coregrind/pub_core_xtmemory.h (added)
+++ trunk/coregrind/pub_core_xtmemory.h Fri Nov 11 14:07:03 2016
@@ -0,0 +1,42 @@
+
+/*--------------------------------------------------------------------*/
+/*--- Support functions for xtree memory reports. pub_tool_xtmemory.h ---*/
+/*--------------------------------------------------------------------*/
+
+/*
+ This file is part of Valgrind, a dynamic binary instrumentation
+ framework.
+
+ Copyright (C) 2016-2016 Philippe Waroquiers
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License as
+ published by the Free Software Foundation; either version 2 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
+ 02111-1307, USA.
+
+ The GNU General Public License is contained in the file COPYING.
+*/
+
+#ifndef __PUB_CORE_XTMEMORY_H
+#define __PUB_CORE_XTMEMORY_H
+
+// No core-only exports; everything in this module is visible to both
+// the core and tools.
+
+#include "pub_tool_xtmemory.h"
+
+#endif // __PUB_CORE_XTMEMORY_H
+
+/*-----------------------------------------------------------------------*/
+/*--- end pub_core_xtmemory.h ---*/
+/*-----------------------------------------------------------------------*/
Added: trunk/coregrind/pub_core_xtree.h
==============================================================================
--- trunk/coregrind/pub_core_xtree.h (added)
+++ trunk/coregrind/pub_core_xtree.h Fri Nov 11 14:07:03 2016
@@ -0,0 +1,42 @@
+
+/*--------------------------------------------------------------------*/
+/*--- An xtree, tree of stacktraces with data pub_core_xtree.h ---*/
+/*--------------------------------------------------------------------*/
+
+/*
+ This file is part of Valgrind, a dynamic binary instrumentation
+ framework.
+
+ Copyright (C) 2015-2015 Philippe Waroquiers
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License as
+ published by the Free Software Foundation; either version 2 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
+ 02111-1307, USA.
+
+ The GNU General Public License is contained in the file COPYING.
+*/
+
+#ifndef __PUB_CORE_XTREE_H
+#define __PUB_CORE_XTREE_H
+
+// No core-only exports; everything in this module is visible to both
+// the core and tools.
+
+#include "pub_tool_xtree.h"
+
+#endif // __PUB_CORE_XTREE_H
+
+/*--------------------------------------------------------------------*/
+/*--- end pub_core_xtree.h ---*/
+/*--------------------------------------------------------------------*/
Modified: trunk/include/Makefile.am
==============================================================================
--- trunk/include/Makefile.am (original)
+++ trunk/include/Makefile.am Fri Nov 11 14:07:03 2016
@@ -43,6 +43,8 @@
pub_tool_vkiscnums_asm.h \
pub_tool_wordfm.h \
pub_tool_xarray.h \
+ pub_tool_xtree.h \
+ pub_tool_xtmemory.h \
valgrind.h \
vki/vki-linux.h \
vki/vki-darwin.h \
Modified: trunk/include/pub_tool_options.h
==============================================================================
--- trunk/include/pub_tool_options.h (original)
+++ trunk/include/pub_tool_options.h Fri Nov 11 14:07:03 2016
@@ -223,6 +223,26 @@
extern VexControl VG_(clo_vex_control);
extern VexRegisterUpdates VG_(clo_px_file_backed);
+extern Int VG_(clo_redzone_size);
+
+typedef
+ enum {
+ Vg_XTMemory_None, // Do not do any xtree memory profiling.
+ Vg_XTMemory_Allocs, // Currently allocated size xtree memory profiling
+ Vg_XTMemory_Full, // Full profiling : Current allocated size, total
+ // allocated size, nr of blocks, total freed size, ...
+ }
+ VgXTMemory;
+// Tools that replace malloc can optionally implement memory profiling
+// following the value of VG_(clo_xtree_profile_memory) to produce a report
+// at the end of execution.
+extern VgXTMemory VG_(clo_xtree_memory);
+/* Holds the filename to use for xtree memory profiling output, before expansion
+ of %p and %q templates. */
+extern const HChar* VG_(clo_xtree_memory_file);
+/* Compress strings in xtree dumps. */
+extern Bool VG_(clo_xtree_compress_strings);
+
/* Number of parents of a backtrace. Default: 12 */
extern Int VG_(clo_backtrace_size);
Added: trunk/include/pub_tool_xtmemory.h
==============================================================================
--- trunk/include/pub_tool_xtmemory.h (added)
+++ trunk/include/pub_tool_xtmemory.h Fri Nov 11 14:07:03 2016
@@ -0,0 +1,86 @@
+
+/*-----------------------------------------------------------------------*/
+/*--- Support functions for xtree memory reports. pub_tool_xtmemory.h ---*/
+/*-----------------------------------------------------------------------*/
+
+/*
+ This file is part of Valgrind, a dynamic binary instrumentation
+ framework.
+
+ Copyright (C) 2016-2016 Philippe Waroquiers
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License as
+ published by the Free Software Foundation; either version 2 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
+ 02111-1307, USA.
+
+ The GNU General Public License is contained in the file COPYING.
+*/
+
+#ifndef __PUB_TOOL_XTMEMORY_H
+#define __PUB_TOOL_XTMEMORY_H
+
+/* Type to profile allocated size and nr of blocks, typically used for
+ --xtree-memory=allocs. */
+typedef
+ struct _XT_Allocs {
+ SizeT nbytes;
+ SizeT nblocks;
+ } XT_Allocs;
+
+/* Support functions to produce a full xtree memory profiling. */
+/* tool must call VG_(XTMemory_Full_init) to ini full xtree memory profiling. */
+extern void VG_(XTMemory_Full_init) (XT_filter_IPs_t filter_IPs_Fn);
+/* Then each time a certain nr of blocks are allocated or freed, the below
+ functions must be called. The arguments are:
+ szB: nr of bytes for the allocated/freed block(s)
+ ec_alloc : ExeContext of the allocation (original allocation for
+ free and resize_in_place).
+ ec_free : ExeContext of the free.
+ The tool is responsible to properly provide the ExeContext for
+ the allocation and free. For VG_(XTMemory_Full_free), ec_alloc
+ must be the one that was used for the allocation of the just released
+ block. */
+extern void VG_(XTMemory_Full_alloc)(SizeT szB,
+ ExeContext* ec_alloc);
+extern void VG_(XTMemory_Full_free)(SizeT szB,
+ ExeContext* ec_alloc,
+ ExeContext* ec_free);
+extern void VG_(XTMemory_Full_resize_in_place)(SizeT oldSzB, SizeT newSzB,
+ ExeContext* ec_alloc);
+
+/* Handle the production of a xtree memory report, either during run (fini False
+ e.g. via a gdb monitor command), or at the end of execution (fini True).
+
+ VG_(XTMemory_report) behaviour depends on the value of the command line
+ options --xtree-memory=none|allocs|full and --xtree-memory-file=<filename> :
+ If --xtree-memory=full, the report will be produced from the data
+ provided via the calls to void VG_(XTMemory_Full_*).
+ Otherwise, for --xtree-memory=allocs or for --xtree-memory=none (if fini
+ is False), next_block is used to get the data for the report:
+ next_block is called repetitively to get information about all allocated
+ blocks, till xta->nblocks is 0.
+ If filename is NULL, --xtree-memory-file is used to produce the name.
+ filter_IPs_fn : used for --xtree-memory=allocs/none filtering (see
+ VG_(XT_create) and XT_filter_IPs_t typdef for more information). */
+extern void VG_(XTMemory_report)
+ (const HChar* filename, Bool fini,
+ void (*next_block)(XT_Allocs* xta, ExeContext** ec_alloc),
+ XT_filter_IPs_t filter_IPs_fn);
+
+#endif // __PUB_TOOL_XTMEMORY_H
+
+
+/*-----------------------------------------------------------------------*/
+/*--- end pub_tool_xtmemory.h ---*/
+/*-----------------------------------------------------------------------*/
Added: trunk/include/pub_tool_xtree.h
==============================================================================
--- trunk/include/pub_tool_xtree.h (added)
+++ trunk/include/pub_tool_xtree.h Fri Nov 11 14:07:03 2016
@@ -0,0 +1,248 @@
+
+/*--------------------------------------------------------------------*/
+/*--- An xtree, tree of stacktraces with data pub_tool_xtree.h ---*/
+/*--------------------------------------------------------------------*/
+
+/*
+ This file is part of Valgrind, a dynamic binary instrumentation
+ framework.
+
+ Copyright (C) 2015-2016 Philippe Waroquiers
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License as
+ published by the Free Software Foundation; either version 2 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
+ 02111-1307, USA.
+
+ The GNU General Public License is contained in the file COPYING.
+*/
+
+#ifndef __PUB_TOOL_XTREE_H
+#define __PUB_TOOL_XTREE_H
+
+#include "pub_tool_basics.h"
+#include "pub_tool_execontext.h"
+
+//--------------------------------------------------------------------
+// PURPOSE: an XTree is conceptually a set of stacktraces organ...
[truncated message content] |
|
From: <sv...@va...> - 2016-11-11 13:55:14
|
Author: philippe
Date: Fri Nov 11 13:55:07 2016
New Revision: 16122
Log:
Add VG_(strIsMemberXA) in pub_tool_xarray.h
This allows to search more efficiently a string in an xarray of string.
(support work for xtree)
Modified:
trunk/coregrind/m_xarray.c
trunk/include/pub_tool_xarray.h
Modified: trunk/coregrind/m_xarray.c
==============================================================================
--- trunk/coregrind/m_xarray.c (original)
+++ trunk/coregrind/m_xarray.c Fri Nov 11 13:55:07 2016
@@ -373,6 +373,16 @@
va_end(vargs);
}
+Bool VG_(strIsMemberXA)(const XArray* xa, const HChar* str )
+{
+ Word i;
+ HChar** members = (HChar**)xa->arr;
+
+ for (i = 0; i < xa->usedsizeE; i++)
+ if (VG_(strcmp)(str, members[i]) == 0)
+ return True;
+ return False;
+}
/*--------------------------------------------------------------------*/
/*--- end m_xarray.c ---*/
Modified: trunk/include/pub_tool_xarray.h
==============================================================================
--- trunk/include/pub_tool_xarray.h (original)
+++ trunk/include/pub_tool_xarray.h Fri Nov 11 13:55:07 2016
@@ -166,6 +166,8 @@
extern void VG_(xaprintf)( XArray* dst, const HChar* format, ... )
PRINTF_CHECK(2, 3);
+/* Convenience function: linear search in an XArray of HChar*. */
+extern Bool VG_(strIsMemberXA)(const XArray* xa, const HChar* str );
#endif // __PUB_TOOL_XARRAY_H
/*--------------------------------------------------------------------*/
|
|
From: <sv...@va...> - 2016-11-11 13:38:25
|
Author: philippe
Date: Fri Nov 11 13:38:18 2016
New Revision: 16121
Log:
Implement a cache 'address -> symbol name' in m_debuginfo.c
Support work for xtree: as xtree implementation makes a high
nr of calls to get_sym_name, this cache improves the performance
as usually, stacktraces are repeatitively querying the same
addresses.
The cache follows the same principle as the cfsi_m_cache.
In particular, cache is cleared together with the cfsi_m cache.
Modified:
trunk/coregrind/m_debuginfo/debuginfo.c
Modified: trunk/coregrind/m_debuginfo/debuginfo.c
==============================================================================
--- trunk/coregrind/m_debuginfo/debuginfo.c (original)
+++ trunk/coregrind/m_debuginfo/debuginfo.c Fri Nov 11 13:38:18 2016
@@ -105,8 +105,7 @@
/*--- fwdses ---*/
/*------------------------------------------------------------*/
-static UInt debuginfo_generation = 0;
-static void cfsi_m_cache__invalidate ( void );
+static void caches__invalidate (void);
/*------------------------------------------------------------*/
@@ -586,8 +585,8 @@
less independently. */
vg_assert(debugInfo_list == NULL);
- /* flush the CFI fast query cache. */
- cfsi_m_cache__invalidate();
+ /* flush the debug info caches. */
+ caches__invalidate();
}
@@ -757,8 +756,8 @@
TRACE_SYMTAB("\n------ Canonicalising the "
"acquired info ------\n");
- /* invalidate the CFI unwind cache. */
- cfsi_m_cache__invalidate();
+ /* invalidate the debug info caches. */
+ caches__invalidate();
/* prepare read data for use */
ML_(canonicaliseTables)( di );
/* Check invariants listed in
@@ -1082,7 +1081,7 @@
if (0) VG_(printf)("DISCARD %#lx %#lx\n", a, a+len);
anyFound = discard_syms_in_range(a, len);
if (anyFound)
- cfsi_m_cache__invalidate();
+ caches__invalidate();
}
@@ -1099,7 +1098,7 @@
if (0 && !exe_ok) {
Bool anyFound = discard_syms_in_range(a, len);
if (anyFound)
- cfsi_m_cache__invalidate();
+ caches__invalidate();
}
}
@@ -1394,9 +1393,9 @@
if (VG_(clo_verbosity) > 0)
VG_(message)(Vg_UserMsg, "LOAD_PDB_DEBUGINFO: pdbname: %s\n", pdbname);
- /* play safe; always invalidate the CFI cache. I don't know if
+ /* play safe; always invalidate the debug info caches. I don't know if
this is necessary, but anyway .. */
- cfsi_m_cache__invalidate();
+ caches__invalidate();
/* dump old info for this range, if any */
discard_syms_in_range( avma_obj, total_size );
@@ -1728,6 +1727,35 @@
*pdi = NULL;
}
+/* Caching of queries to symbol names. */
+// Prime number, giving about 6Kbytes cache on 32 bits,
+// 12Kbytes cache on 64 bits.
+#define N_SYM_NAME_CACHE 509
+
+typedef
+ struct { Addr sym_avma; const HChar* sym_name; PtrdiffT offset; }
+ Sym_Name_CacheEnt;
+/* Sym_Name_CacheEnt associates a queried address to the sym name found.
+ By nature, if a sym name was found, it means the searched address
+ stored in the cache is an avma (see e.g. search_all_symtabs).
+ Note however that the caller is responsibe to work with 'avma'
+ addresses e.g. when calling VG_(get_fnname) : m_debuginfo.c has
+ no way to differentiate an 'svma a' from an 'avma a'. It is however
+ unlikely that svma would percolate outside of this module. */
+
+static Sym_Name_CacheEnt sym_name_cache[N_SYM_NAME_CACHE];
+
+static const HChar* no_sym_name = "<<<noname>>>";
+/* We need a special marker for the address 0 : a not used entry has
+ a zero sym_avma. So, if ever the 0 address is really queried, we need
+ to be able to detect there is no sym name for this address.
+ If on some platforms, 0 is associated to a symbol, the cache would
+ work properly. */
+
+static void sym_name_cache__invalidate ( void ) {
+ VG_(memset)(&sym_name_cache, 0, sizeof(sym_name_cache));
+ sym_name_cache[0].sym_name = no_sym_name;
+}
/* The whole point of this whole big deal: map a code address to a
plausible symbol name. Returns False if no idea; otherwise True.
@@ -1752,34 +1780,46 @@
Bool match_anywhere_in_sym, Bool show_offset,
Bool findText, /*OUT*/PtrdiffT* offsetP )
{
- DebugInfo* di;
- Word sno;
- PtrdiffT offset;
+ UWord hash = a % N_SYM_NAME_CACHE;
+ Sym_Name_CacheEnt* se = &sym_name_cache[hash];
+
+ if (UNLIKELY(se->sym_avma != a)) {
+ DebugInfo* di;
+ Word sno;
+
+ search_all_symtabs ( a, &di, &sno, match_anywhere_in_sym, findText );
+ se->sym_avma = a;
+ if (di == NULL || a == 0)
+ se->sym_name = no_sym_name;
+ else {
+ vg_assert(di->symtab[sno].pri_name);
+ se->sym_name = di->symtab[sno].pri_name;
+ se->offset = a - di->symtab[sno].avmas.main;
+ }
+ }
- search_all_symtabs ( a, &di, &sno, match_anywhere_in_sym, findText );
- if (di == NULL) {
+ if (se->sym_name == no_sym_name) {
*buf = "";
return False;
}
- vg_assert(di->symtab[sno].pri_name);
VG_(demangle) ( do_cxx_demangling, do_z_demangling,
- di->symtab[sno].pri_name, buf );
+ se->sym_name, buf );
/* Do the below-main hack */
// To reduce the endless nuisance of multiple different names
// for "the frame below main()" screwing up the testsuite, change all
// known incarnations of said into a single name, "(below main)", if
// --show-below-main=yes.
- if ( do_below_main_renaming && ! VG_(clo_show_below_main) &&
- Vg_FnNameBelowMain == VG_(get_fnname_kind)(*buf) )
+ if ( do_below_main_renaming && ! VG_(clo_show_below_main)
+ && Vg_FnNameBelowMain == VG_(get_fnname_kind)(*buf) )
{
*buf = "(below main)";
}
- offset = a - di->symtab[sno].avmas.main;
- if (offsetP) *offsetP = offset;
- if (show_offset && offset != 0) {
+ if (offsetP) *offsetP = se->offset;
+
+ if (show_offset && se->offset != 0) {
static HChar *bufwo; // buf with offset
static SizeT bufwo_szB;
SizeT need, len;
@@ -1793,8 +1833,8 @@
VG_(strcpy)(bufwo, *buf);
VG_(sprintf)(bufwo + len, "%c%ld",
- offset < 0 ? '-' : '+',
- offset < 0 ? -offset : offset);
+ se->offset < 0 ? '-' : '+',
+ se->offset < 0 ? -se->offset : se->offset);
*buf = bufwo;
}
@@ -2654,12 +2694,6 @@
static void cfsi_m_cache__invalidate ( void ) {
VG_(memset)(&cfsi_m_cache, 0, sizeof(cfsi_m_cache));
- debuginfo_generation++;
-}
-
-UInt VG_(debuginfo_generation) (void)
-{
- return debuginfo_generation;
}
static inline CFSI_m_CacheEnt* cfsi_m_cache__find ( Addr ip )
@@ -4440,6 +4474,19 @@
}
+static UInt debuginfo_generation = 0;
+
+UInt VG_(debuginfo_generation) (void)
+{
+ return debuginfo_generation;
+}
+
+static void caches__invalidate ( void ) {
+ cfsi_m_cache__invalidate();
+ sym_name_cache__invalidate();
+ debuginfo_generation++;
+}
+
/*--------------------------------------------------------------------*/
/*--- end ---*/
/*--------------------------------------------------------------------*/
|
|
From: <sv...@va...> - 2016-11-11 13:30:21
|
Author: philippe
Date: Fri Nov 11 13:30:08 2016
New Revision: 16120
Log:
Support pool of unique string in pub_tool_deduppoolalloc.h
This is support code for the xtree implementation.
Modified:
trunk/coregrind/m_deduppoolalloc.c
trunk/include/pub_tool_deduppoolalloc.h
Modified: trunk/coregrind/m_deduppoolalloc.c
==============================================================================
--- trunk/coregrind/m_deduppoolalloc.c (original)
+++ trunk/coregrind/m_deduppoolalloc.c Fri Nov 11 13:30:08 2016
@@ -41,6 +41,7 @@
struct _DedupPoolAlloc {
SizeT poolSzB; /* Minimum size of a pool. */
SizeT fixedSzb; /* If using VG_(allocFixedEltDedupPA), size of elements */
+ Bool strPA; /* True if this is a string dedup pool */
SizeT eltAlign;
void* (*alloc_fn)(const HChar*, SizeT); /* pool allocator */
const HChar* cc; /* pool allocator's cost centre */
@@ -75,7 +76,8 @@
struct _ht_node {
struct _ht_node *next; // Read/Write by hashtable (pub_tool_hashtable.h)
UWord key; // Read by hashtable (pub_tool_hashtable.h)
- SizeT eltSzB;
+ SizeT eltSzBorStrNr; // for a normal pool, elt size
+ // for a string pool, the unique str number
const void *elt;
}
ht_node;
@@ -97,6 +99,7 @@
VG_(memset)(ddpa, 0, sizeof(*ddpa));
ddpa->poolSzB = poolSzB;
ddpa->fixedSzb = 0;
+ ddpa->strPA = False;
ddpa->eltAlign = eltAlign;
ddpa->alloc_fn = alloc_fn;
ddpa->cc = cc;
@@ -189,14 +192,24 @@
/* As this function is called by hashtable, that has already checked
for key equality, it is likely that it is the 'good' element.
So, we handle the equal case first. */
- if (hnode1->eltSzB == hnode2->eltSzB)
- return VG_(memcmp) (hnode1->elt, hnode2->elt, hnode1->eltSzB);
- else if (hnode1->eltSzB < hnode2->eltSzB)
+ if (hnode1->eltSzBorStrNr == hnode2->eltSzBorStrNr)
+ return VG_(memcmp) (hnode1->elt, hnode2->elt, hnode1->eltSzBorStrNr);
+ else if (hnode1->eltSzBorStrNr < hnode2->eltSzBorStrNr)
return -1;
else
return 1;
}
+/* String compare function for 'gen' hash table.
+ Similarly to cmp_pool_elt, no need to compare the key. */
+static Word cmp_pool_str (const void* node1, const void* node2 )
+{
+ const ht_node* hnode1 = node1;
+ const ht_node* hnode2 = node2;
+
+ return VG_(strcmp)(hnode1->elt, hnode2->elt);
+}
+
/* Print some stats. */
static void print_stats (DedupPoolAlloc *ddpa)
{
@@ -209,7 +222,10 @@
VG_(sizeXA)(ddpa->pools),
ddpa->curpool ?
(long int) (ddpa->curpool_limit - ddpa->curpool_free + 1) : 0);
- VG_(HT_print_stats) (ddpa->ht_elements, cmp_pool_elt);
+ if (ddpa->strPA)
+ VG_(HT_print_stats) (ddpa->ht_elements, cmp_pool_str);
+ else
+ VG_(HT_print_stats) (ddpa->ht_elements, cmp_pool_elt);
}
/* Dummy free, as the ht elements are allocated in a pool, and
@@ -247,8 +263,8 @@
return h;
}
-const void* VG_(allocEltDedupPA) (DedupPoolAlloc *ddpa, SizeT eltSzB,
- const void *elt)
+static ht_node* allocEltDedupPA (DedupPoolAlloc *ddpa, SizeT eltSzB,
+ const void *elt)
{
ht_node ht_elt;
void* elt_ins;
@@ -260,12 +276,16 @@
ht_elt.key = sdbm_hash (elt, eltSzB);
- ht_elt.eltSzB = eltSzB;
ht_elt.elt = elt;
- ht_ins = VG_(HT_gen_lookup) (ddpa->ht_elements, &ht_elt, cmp_pool_elt);
+ if (ddpa->strPA)
+ ht_ins = VG_(HT_gen_lookup) (ddpa->ht_elements, &ht_elt, cmp_pool_str);
+ else {
+ ht_elt.eltSzBorStrNr = eltSzB;
+ ht_ins = VG_(HT_gen_lookup) (ddpa->ht_elements, &ht_elt, cmp_pool_elt);
+ }
if (ht_ins)
- return ht_ins->elt;
+ return ht_ins;
/* Not found -> we need to allocate a new element from the pool
and insert it in the hash table of inserted elements. */
@@ -291,10 +311,37 @@
VG_(memcpy)(elt_ins, elt, eltSzB);
ht_ins = VG_(allocEltPA) (ddpa->ht_node_pa);
ht_ins->key = ht_elt.key;
- ht_ins->eltSzB = eltSzB;
+ if (ddpa->strPA)
+ ht_ins->eltSzBorStrNr = VG_(HT_count_nodes)(ddpa->ht_elements) + 1;
+ else
+ ht_ins->eltSzBorStrNr = eltSzB;
ht_ins->elt = elt_ins;
VG_(HT_add_node)(ddpa->ht_elements, ht_ins);
- return elt_ins;
+ return ht_ins;
+}
+
+const void* VG_(allocEltDedupPA) (DedupPoolAlloc *ddpa, SizeT eltSzB,
+ const void *elt)
+{
+ return allocEltDedupPA(ddpa, eltSzB, elt)->elt;
+}
+
+UInt VG_(allocStrDedupPA) (DedupPoolAlloc *ddpa,
+ const HChar* str,
+ Bool* newStr)
+{
+ if (!ddpa->strPA) {
+ // First insertion in this ddpa
+ vg_assert (ddpa->nr_alloc_calls == 0);
+ vg_assert (ddpa->fixedSzb == 0);
+ ddpa->strPA = True;
+ }
+
+ const UInt nr_str = VG_(HT_count_nodes)(ddpa->ht_elements);
+ const ht_node* ht_ins = allocEltDedupPA(ddpa, VG_(strlen)(str)+1, str);
+
+ *newStr = nr_str < VG_(HT_count_nodes)(ddpa->ht_elements);
+ return ht_ins->eltSzBorStrNr;
}
static __inline__
@@ -311,6 +358,7 @@
{
if (ddpa->fixedSzb == 0) {
// First insertion in this ddpa
+ vg_assert (!ddpa->strPA);
vg_assert (ddpa->nr_alloc_calls == 0);
vg_assert (eltSzB > 0);
ddpa->fixedSzb = eltSzB;
Modified: trunk/include/pub_tool_deduppoolalloc.h
==============================================================================
--- trunk/include/pub_tool_deduppoolalloc.h (original)
+++ trunk/include/pub_tool_deduppoolalloc.h Fri Nov 11 13:30:08 2016
@@ -44,8 +44,8 @@
// individually.
// Once allocated, an element must not be modified anymore.
//
-// Elements can be inserted in the pool using VG_(allocEltDedupPA)
-// or using VG_(allocFixedEltDedupPA).
+// Elements can be inserted in the pool using VG_(allocEltDedupPA),
+// VG_(allocFixedEltDedupPA) or VG_(allocStrDedupPA).
//
// Use VG_(allocFixedEltDedupPA) to allocate elements that are all of
// the same size and that you want to identify with a (small) number:
@@ -65,6 +65,10 @@
// The address of an element allocated with VG_(allocEltDedupPA) does
// not change, even if new elements are inserted in the pool.
//
+// Use VG_(allocStrDedupPA) to create a pool of strings (in other words, a
+// dictionnary of strings). Similarly to VG_(allocFixedEltDedupPA), strings
+// inserted in a dedup pool can be identified by an element number.
+//
// In the same pool, you can only use one of the allocate element functions.
//
// A dedup pool allocator has significantly less memory overhead than
@@ -93,29 +97,42 @@
const HChar* cc,
void (*free_fn)(void*) );
-/* Allocates a new element from ddpa with eltSzB bytes to store elt.
+/* Allocates or retrieve element from ddpa with eltSzB bytes to store elt.
This function never returns NULL.
If ddpa already contains an element equal to elt, then the address of
the already existing element is returned.
Equality between elements is done by comparing all bytes.
So, if void *elt points to a struct, be sure to initialise all components
and the holes between components. */
-extern const void* VG_(allocEltDedupPA) (DedupPoolAlloc *ddpa,
- SizeT eltSzB, const void *elt);
+extern const void* VG_(allocEltDedupPA) (DedupPoolAlloc* ddpa,
+ SizeT eltSzB, const void* elt);
-/* Allocates a new (fixed size) element from ddpa. Returns the
- unique number identifying this element. This function never returns NULL.
+/* Allocates or retrieve a (fixed size) element from ddpa. Returns the
+ unique number identifying this element.
Similarly to VG_(allocEltDedupPA), this will return the unique number
of an already existing identical element to elt. */
-extern UInt VG_(allocFixedEltDedupPA) (DedupPoolAlloc *ddpa,
- SizeT eltSzB, const void *elt);
+extern UInt VG_(allocFixedEltDedupPA) (DedupPoolAlloc* ddpa,
+ SizeT eltSzB, const void* elt);
/* Translate an element number to its address. Note that the address
corresponding to eltNr can change if new elements are inserted
in the pool. */
-extern void* VG_(indexEltNumber) (DedupPoolAlloc *ddpa,
+extern void* VG_(indexEltNumber) (DedupPoolAlloc* ddpa,
UInt eltNr);
+/* Allocates or retrieve a string element from ddpa. Returns the
+ unique number identifying this string.
+ newStr is set to True if the str is a newly inserted string, False
+ if the str was already present in the pool.
+ Similarly to VG_(allocEltDedupPA), this will return the unique number
+ of an already existing identical string. */
+extern UInt VG_(allocStrDedupPA) (DedupPoolAlloc *ddpa,
+ const HChar* str,
+ Bool* newStr);
+/* Note: Implementing a function to return the string value from its strNr
+ implies some overhead, so will be done only if/when needed. */
+
+
/* The Dedup Pool Allocator must maintain a data structure to avoid
duplicates as long as new elements can be allocated from the pool.
Once no new elements will be allocated, this dedup data structure
@@ -123,14 +140,14 @@
it is an error to call VG_(allocEltDedupPA) or VG_(allocFixedEltDedupPA).
If shrink_block is not NULL, the last pool will be shrunk using
shrink_block. */
-extern void VG_(freezeDedupPA) (DedupPoolAlloc *ddpa,
+extern void VG_(freezeDedupPA) (DedupPoolAlloc* ddpa,
void (*shrink_block)(void*, SizeT));
/* How many (unique) elements are there in this ddpa now? */
-extern UInt VG_(sizeDedupPA) (DedupPoolAlloc *ddpa);
+extern UInt VG_(sizeDedupPA) (DedupPoolAlloc* ddpa);
/* Free all memory associated with a DedupPoolAlloc. */
-extern void VG_(deleteDedupPA) ( DedupPoolAlloc *ddpa);
+extern void VG_(deleteDedupPA) ( DedupPoolAlloc* ddpa);
#endif // __PUB_TOOL_DEDUPPOOLALLOC_
|
|
From: Philippe W. <phi...@sk...> - 2016-11-11 10:04:48
|
On Thu, 2016-11-10 at 18:17 +0100, Petar Jovanovic wrote: > On Sat, Sep 24, 2016 at 2:04 PM, Philippe Waroquiers > <phi...@sk...> wrote: > > Mark/Ivo, > > > > I am (now?) seeing random failures of helgrind|drd/tests/bar_bad* > > (also now seeing failures in nightly builds). > > > > I have encountered such failures on amd64/debian 8, and on ppc64/gcc110. > > > Hi Philippe, > > Can you check if the following patch helps with those failures? > > https://bugsfiles.kde.org/attachment.cgi?id=102144 > Hello Petar, I tried but this does not (fully?) solve the problem. Based on a few trials, I think the effect is: * helgrind test seems to succeed (more?) systematically, but takes a lot longer (around one minute, burning a lot of CPU, against about 10 seconds unpatched). * drd tests seems to fail (more?) systematically Thanks for looking at this, looks like this test is a nightmare (Mark already spent significant time to try to make it deterministic and work on all/most distros). Philippe |
|
From: Andrew C. M. <and...@gm...> - 2016-11-10 18:27:33
|
OK, up and running with 3.12. Thanks again. Andrew On Thu, Nov 10, 2016 at 11:03 AM, Andrew C. Morrow < and...@gm...> wrote: > >> >> > However, we have now encountered the following error: >> > [..] >> > Are the socket syscalls somehow not implemented in valgrind for ppc64le? >> >> That's such basic functionality that it merely confirms my suspicion >> that you're not running the latest (3.12.0). >> >> J >> >> > Indeed we are not. Looks like Ubuntu 16.04 bundles 3.11.0: > > $ valgrind --version > valgrind-3.11.0 > > I'll get 3.12 built from source and give it a try. > > Thanks for your help, > Andrew > > > |
|
From: Petar J. <mip...@gm...> - 2016-11-10 17:17:17
|
On Sat, Sep 24, 2016 at 2:04 PM, Philippe Waroquiers <phi...@sk...> wrote: > Mark/Ivo, > > I am (now?) seeing random failures of helgrind|drd/tests/bar_bad* > (also now seeing failures in nightly builds). > > I have encountered such failures on amd64/debian 8, and on ppc64/gcc110. > Hi Philippe, Can you check if the following patch helps with those failures? https://bugsfiles.kde.org/attachment.cgi?id=102144 Regards, Petar |
|
From: Andrew C. M. <and...@gm...> - 2016-11-10 16:03:38
|
> > > > > However, we have now encountered the following error: > > [..] > > Are the socket syscalls somehow not implemented in valgrind for ppc64le? > > That's such basic functionality that it merely confirms my suspicion > that you're not running the latest (3.12.0). > > J > > Indeed we are not. Looks like Ubuntu 16.04 bundles 3.11.0: $ valgrind --version valgrind-3.11.0 I'll get 3.12 built from source and give it a try. Thanks for your help, Andrew |
|
From: Julian S. <js...@ac...> - 2016-11-10 15:01:56
|
On 10/11/16 15:34, Andrew C. Morrow wrote:
Your message was timely; I was just about to reply to and it showed up
seconds before.
> [mismatched alloc/free messages]
> is --show-mismatched-frees=no, which will work fine for us as a workaround
> now, but would be not generally a good idea since it could mask real user
> errors. In any event, we have a workaround for this.
I've seen this a lot running Firefox on Valgrind. My impression is that it
is due to what you could call "differential inlining". Imagine that we have
this:
operator new (size_t size) { ... return malloc(size) ... }
operator delete ( void* p) { ... free(p) ... }
If both |new| and |delete| are inlined, then Memcheck only ever sees calls
to |malloc| and |free|, and there's no problem (and no mismatch checking).
If neither are inlined, then there's no problem and everything works as
originally designed. The problem occurs when one but not the other
is inlined. Then Memcheck observes, for example, allocations from
|new| (which is not inlined) being returned to |free| (because |delete|
got inlined).
I don't have a good fix for this, so I always just run with
--show-mismatched-frees=no.
> For the other issue about the __lll_lock_elision error, that was
> encountered running valgrind on a ppc64le POWER8 machine running Ubuntu
> 16.04. The compiler was the system GCC, version 5.4.0. I've just tried
> again today without the --track-origins=yes flag, and I can confirm that
> I'm able to get the process running under valgrind if I omit the
> --track-origins=yes flag. This is unfortunate too, because if valgrind does
> report an error, I've found that almost always you need --track-origins=yes
> to really sort it out.
Fixing this is a 1-liner if we can figure out what is at the given offset
of the ppc64le guest state. Which we could do if we can see the instruction
it's failing at. Ah, so it's failing at the basic block that starts here:
==19834== at 0x4F3AC14: __lll_lock_elision (elision-lock.c:60)
Can you objdump -d libpthread (I think it's that) and show the block
that follows? We're probably looking for an instruction that pokes
some very obscure system status register or something like that; we're
not looking for vanilla integer arithmetic or anything ordinary.
But first -- before you do *any* of that -- can you try with V 3.12.0
(or make sure you're using it)? I say that because Carl fixed a exactly
that stuff earlier this year.
r15895 | carll | 2016-06-27 17:50:29 +0200 (Mon, 27 Jun 2016) | 3 lines
Add the HW register support for missing registers in
get_otrack_shadow_offset_wrk(). The registers are: NRADDR, NRADDR_GPR2,
(REDIR_STACK, TFHAR, TEXASR, TEXASRU, TFIAR, PPR, PSPB.
> However, we have now encountered the following error:
> [..]
> Are the socket syscalls somehow not implemented in valgrind for ppc64le?
That's such basic functionality that it merely confirms my suspicion
that you're not running the latest (3.12.0).
J
|
|
From: Andrew C. M. <and...@gm...> - 2016-11-10 14:35:04
|
On Thu, Nov 10, 2016 at 2:03 AM, Paul Floyd <pa...@fr...> wrote:
>
> On 9 Nov 2016, at 23:25, Carl E. Love wrote:
>
> > Julian, Valgrind developers:
> >
> > We have some people on the compiler team trying to help out on a
> > customer issue. The customer tried to use valgrind memcheck and got
> > several errors that we would like help understanding. In the first
> > experiment, they get messages about mismatched free() / delete /
> > delete[]. They had valgrind ignore those issues and then see erorrs on
> > "get_otrack_shadow_offset_wrk".
> >
> > Here is the valgrind output from the experiments from the bugzilla
> > report:
> >
> > valgrind --soname-synonyms=somalloc=NONE --track-origins=yes
> --leak-check=no ./mongos
> > ==17387== Memcheck, a memory error detector
> > ==17387== Copyright (C) 2002-2015, and GNU GPL'd, by Julian
> Seward et al.
> > ==17387== Using Valgrind-3.11.0 and LibVEX; rerun with -h for
> copyright info
> > ==17387== Command: ./mongos
> > ==17387==
> > ==17387== Mismatched free() / delete / delete []
>
> Hi Carl
>
> I'd like to add a little to what Philippe wrote for this case. There are
> two kind-of 'false positive' situations that I've encountered. Firstly, as
> already mentioned, if operator new/operator delete is inlined. In
> particular I see this in class overloads of the operators, which tend to be
> inlined more often e.g.,
>
> class MyClass
> {
> public:
> void* operator new(std::size_t count); // out of line
> void operator delete(void* ptr) { myFree(ptr); } // inline
> };
>
> The second situation that I've encountered is if you use an overload that
> uses a non-standard signature, Valgrind won't see it as a new or delete but
> will probably still see the underlying call to malloc/free and so will flag
> a mismatched free/delete.
>
> Looking at your callstack you seem to be using a
>
> std::unordered_map<std::string, std::pair<std::string, mongo::
> InitializerDependencyGraph::NodeData>
>
> and the error occurs during rehashing. So I doubt that it is a
> non-standard overload.
>
> Which compiler and standard library are you using?
>
> A+
> Paul
>
>
>
Hi -
As it happens, I'm both a lurker on this list from some long ago dev work
on valgrind, and I'm the customer referenced in the original email, so I
can answer questions directly.
We run our code under ASAN, which does detect mismatched free/delete, and
we are fairly certain that we don't have any instances of that problem. So
this is almost certainly due to the inlining issue as described above, and
all of the stacks do look like they reach into the std library headers.
That is sort of unfortunate, because it appears to mean that if you are
using a modern libstdc++ and an aggressive optimization level, you are
going to get a ton of false positives that would need to be suppressed
since they aren't under the control of the developer. Writing suppressions
for std library headers is going to be tricky and fragile. The alternative
is --show-mismatched-frees=no, which will work fine for us as a workaround
now, but would be not generally a good idea since it could mask real user
errors. In any event, we have a workaround for this.
For the other issue about the __lll_lock_elision error, that was
encountered running valgrind on a ppc64le POWER8 machine running Ubuntu
16.04. The compiler was the system GCC, version 5.4.0. I've just tried
again today without the --track-origins=yes flag, and I can confirm that
I'm able to get the process running under valgrind if I omit the
--track-origins=yes flag. This is unfortunate too, because if valgrind does
report an error, I've found that almost always you need --track-origins=yes
to really sort it out.
However, we have now encountered the following error:
[js_test:fsm_all_sharded_replication] 2016-11-10T14:24:04.496+0000 s40019|
--17827-- WARNING: unhandled ppc64le-linux syscall: 326
[js_test:fsm_all_sharded_replication] 2016-11-10T14:24:04.496+0000 s40019|
--17827-- You may be able to write your own handler.
[js_test:fsm_all_sharded_replication] 2016-11-10T14:24:04.496+0000 s40019|
--17827-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
[js_test:fsm_all_sharded_replication] 2016-11-10T14:24:04.497+0000 s40019|
--17827-- Nevertheless we consider this a bug. Please report
[js_test:fsm_all_sharded_replication] 2016-11-10T14:24:04.497+0000 s40019|
--17827-- it at http://valgrind.org/support/bug_reports.html.
[js_test:fsm_all_sharded_replication] 2016-11-10T14:24:04.509+0000 s40019|
2016-11-10T14:24:04.509+0000 I - [mongosMain] Assertion:
15863:listen(): invalid socket? Function not implemented src/mon
go/util/net/listen.cpp 184
Looking at listen.cpp:
182 SOCKET sock = ::socket(me.getType(), SOCK_STREAM, 0);
183 ScopeGuard socketGuard = MakeGuard(&closesocket, sock);
184 massert(15863,
185 str::stream() << "listen(): invalid socket? " <<
errnoWithDescription(),
186 sock >= 0);
The process ends up terminating after this because it can't open its
listening socket.
Are the socket syscalls somehow not implemented in valgrind for ppc64le?
Thanks,
Andrew
|
|
From: Paul F. <pa...@fr...> - 2016-11-10 07:03:38
|
On 9 Nov 2016, at 23:25, Carl E. Love wrote:
> Julian, Valgrind developers:
>
> We have some people on the compiler team trying to help out on a
> customer issue. The customer tried to use valgrind memcheck and got
> several errors that we would like help understanding. In the first
> experiment, they get messages about mismatched free() / delete /
> delete[]. They had valgrind ignore those issues and then see erorrs on
> "get_otrack_shadow_offset_wrk".
>
> Here is the valgrind output from the experiments from the bugzilla
> report:
>
> valgrind --soname-synonyms=somalloc=NONE --track-origins=yes --leak-check=no ./mongos
> ==17387== Memcheck, a memory error detector
> ==17387== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
> ==17387== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
> ==17387== Command: ./mongos
> ==17387==
> ==17387== Mismatched free() / delete / delete []
Hi Carl
I'd like to add a little to what Philippe wrote for this case. There are two kind-of 'false positive' situations that I've encountered. Firstly, as already mentioned, if operator new/operator delete is inlined. In particular I see this in class overloads of the operators, which tend to be inlined more often e.g.,
class MyClass
{
public:
void* operator new(std::size_t count); // out of line
void operator delete(void* ptr) { myFree(ptr); } // inline
};
The second situation that I've encountered is if you use an overload that uses a non-standard signature, Valgrind won't see it as a new or delete but will probably still see the underlying call to malloc/free and so will flag a mismatched free/delete.
Looking at your callstack you seem to be using a
std::unordered_map<std::string, std::pair<std::string, mongo::InitializerDependencyGraph::NodeData>
and the error occurs during rehashing. So I doubt that it is a non-standard overload.
Which compiler and standard library are you using?
A+
Paul
|
|
From: Philippe W. <phi...@sk...> - 2016-11-09 23:02:42
|
On Wed, 2016-11-09 at 14:25 -0800, Carl E. Love wrote: > $ valgrind --show-mismatched-frees=no > --soname-synonyms=somalloc=NONE --track-origins=yes > --leak-check=no ./mongos > ==19834== Memcheck, a memory error detector > ==19834== Copyright (C) 2002-2015, and GNU GPL'd, by Julian > Seward et al. > ==19834== Using Valgrind-3.11.0 and LibVEX; rerun with -h for > copyright info > ==19834== Command: ./mongos > ==19834== > MC_(get_otrack_shadow_offset)(ppc64)(off=1688,sz=8) > > Memcheck: mc_machine.c:329 (get_otrack_shadow_offset_wrk): the > 'impossible' happened. BTW, waiting for a solution in Valgrind; you should be able to bypass this problem by not using --track-origins Philippe |
|
From: Philippe W. <phi...@sk...> - 2016-11-09 22:46:39
|
On Wed, 2016-11-09 at 14:25 -0800, Carl E. Love wrote:
> ==17387== Mismatched free() / delete / delete []
> ==17387== at 0x4895888: free (in /usr/lib/valgrind/vgpreload_memcheck-ppc64le-linux.so)
> ==17387== by 0x59514F: deallocate (new_allocator.h:110)
> ==17387== by 0x59514F: deallocate (alloc_traits.h:517)
....
> ==17387== Address 0x5151fb0 is 0 bytes inside a block of size 16 alloc'd
> ==17387== at 0x48951D4: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-ppc64le-linux.so)
> ==17387== by 0x59328F: allocate (new_allocator.h:104)
> ==17387== by 0x59328F: allocate (alloc_traits.h:491)
....
I assume that the code is correct, and does not call 'free' on some
memory allocated with 'new[]'.
Some false positive have been seen already with such 'Mismatched ....'
due to compiler optimisation: e.g. if the deallocator at
new_allocator.h:110
is calling delete[] (I assume that is what is needed), the compiler
optimiser/inliner/.... might in fact remove the call to delete[] and
directly call the underlying free (if delete[] is implemented on top of
free).
For sure, some inlining was activated as we see several stack entries
that have the same IP (e.g. new_allocator.h:110 and alloc_traits.h:517).
>
> So, that is a puzzle. However, I can instruct valgrind to ignore that. But it still fails to start, now with something more odd:
>
> $ valgrind --show-mismatched-frees=no --soname-synonyms=somalloc=NONE --track-origins=yes --leak-check=no ./mongos
> ==19834== Memcheck, a memory error detector
> ==19834== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
> ==19834== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
> ==19834== Command: ./mongos
> ==19834==
> MC_(get_otrack_shadow_offset)(ppc64)(off=1688,sz=8)
>
> Memcheck: mc_machine.c:329 (get_otrack_shadow_offset_wrk): the 'impossible' happened.
Humph, for this, I do not have much knowledge, so here are some wild
guesses:
Maybe there are some new registers that have been added recently,
and the shadow offsets for this (or these) new registers is not correct.
>From the stacktrace below containing __lll_lock_elision, I assume this
is using transactions to implement some locking technique, and origin
tracking for this transaction register is not ok ?
It looks like there is a request for shadow offset of 8 bytes at offset
1688, but this is a 4 bytes register ?
So, I guess you need to avoid having origin tracking for the transaction
register, or hyou need to add something in mc_machine.c to cover this
register ?
>
> host stacktrace:
> ==19834== at 0x3808D9B8: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
> ==19834== by 0x3808DB5F: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
> ==19834== by 0x3808DCDB: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
> ==19834== by 0x38078CE3: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
> ==19834== by 0x38076FAB: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
> ==19834== by 0x380BAA2B: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
> ==19834== by 0x381B9BB7: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
> ==19834== by 0x380BE19F: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
> ==19834== by 0x3810D04F: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
> ==19834== by 0x3810FFEF: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
> ==19834== by 0x3812BB97: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
>
> sched status:
> running_tid=1
>
> Thread 1: status = VgTs_Runnable (lwpid 19834)
> ==19834== at 0x4F3AC14: __lll_lock_elision (elision-lock.c:60)
> ==19834== by 0x4F2BBC7: pthread_mutex_lock (pthread_mutex_lock.c:92)
> ==19834== by 0x602753: mongo::DBConnectionPool::DBConnectionPool() (connpool.cpp:196)
> ==19834== by 0x5319EB: __static_initialization_and_destruction_0 (global_conn_pool.cpp:35)
> ==19834== by 0x5319EB: _GLOBAL__sub_I__ZN5mongo14globalConnPoolE (global_conn_pool.cpp:39)
> ==19834== by 0x137FED3: __libc_csu_init (in /home/acm/opt/src/mongo/mongos)
> ==19834== by 0x4F830A7: generic_start_main.isra.0 (libc-start.c:247)
> ==19834== by 0x4F83337: (below main) (libc-start.c:116)
>
> You should be able to see the complete discussion of the issue at
> https://bugs.launchpad.net/ubuntu/+source/gcc-5/+bug/1640518
>
> If you have any insight on the errors or suggestions on how to proceed
> with Valgrind, that would be appreciated.
>
> Carl Love
>
>
> ------------------------------------------------------------------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today. http://sdm.link/xeonphi
> _______________________________________________
> Valgrind-developers mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-developers
|
|
From: Carl E. L. <ce...@us...> - 2016-11-09 22:25:49
|
Julian, Valgrind developers:
We have some people on the compiler team trying to help out on a
customer issue. The customer tried to use valgrind memcheck and got
several errors that we would like help understanding. In the first
experiment, they get messages about mismatched free() / delete /
delete[]. They had valgrind ignore those issues and then see erorrs on
"get_otrack_shadow_offset_wrk".
Here is the valgrind output from the experiments from the bugzilla
report:
valgrind --soname-synonyms=somalloc=NONE --track-origins=yes --leak-check=no ./mongos
==17387== Memcheck, a memory error detector
==17387== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==17387== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==17387== Command: ./mongos
==17387==
==17387== Mismatched free() / delete / delete []
==17387== at 0x4895888: free (in /usr/lib/valgrind/vgpreload_memcheck-ppc64le-linux.so)
==17387== by 0x59514F: deallocate (new_allocator.h:110)
==17387== by 0x59514F: deallocate (alloc_traits.h:517)
==17387== by 0x59514F: _M_deallocate_buckets (hashtable_policy.h:2010)
==17387== by 0x59514F: _M_deallocate_buckets (hashtable.h:356)
==17387== by 0x59514F: _M_deallocate_buckets (hashtable.h:361)
==17387== by 0x59514F: _M_rehash_aux (hashtable.h:1999)
==17387== by 0x59514F: std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_rehash(unsigned long, unsigned long const&) (hashtable.h:1953)
==17387== by 0x595253: std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_insert_unique_node(unsigned long, unsigned long, std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData>, true>*) (hashtable.h:1600)
==17387== by 0x5954D3: std::__detail::_Map_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (hashtable_policy.h:600)
==17387== by 0x593693: operator[] (unordered_map.h:668)
==17387== by 0x593693: mongo::InitializerDependencyGraph::addInitializer(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<mongo::Status (mongo::InitializerContext*)> const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) (initializer_dependency_graph.cpp:58)
==17387== by 0x591057: mongo::GlobalInitializerRegisterer::GlobalInitializerRegisterer(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<mongo::Status (mongo::InitializerContext*)> const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) (global_initializer_registerer.cpp:44)
==17387== by 0x52D46F: __static_initialization_and_destruction_0(int, int) [clone .constprop.34] (mongos_options_init.cpp:39)
==17387== by 0x137FED3: __libc_csu_init (in /home/acm/opt/src/mongo/mongos)
==17387== by 0x4F830A7: generic_start_main.isra.0 (libc-start.c:247)
==17387== by 0x4F83337: (below main) (libc-start.c:116)
==17387== Address 0x5151fb0 is 0 bytes inside a block of size 16 alloc'd
==17387== at 0x48951D4: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-ppc64le-linux.so)
==17387== by 0x59328F: allocate (new_allocator.h:104)
==17387== by 0x59328F: allocate (alloc_traits.h:491)
==17387== by 0x59328F: std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, true> > >::_M_allocate_buckets(unsigned long) [clone .isra.108] (hashtable_policy.h:1996)
==17387== by 0x595093: _M_allocate_buckets (hashtable.h:347)
==17387== by 0x595093: _M_rehash_aux (hashtable.h:1974)
==17387== by 0x595093: std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_rehash(unsigned long, unsigned long const&) (hashtable.h:1953)
==17387== by 0x595253: std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_insert_unique_node(unsigned long, unsigned long, std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData>, true>*) (hashtable.h:1600)
==17387== by 0x5954D3: std::__detail::_Map_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, mongo::InitializerDependencyGraph::NodeData> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (hashtable_policy.h:600)
==17387== by 0x59356B: operator[] (unordered_map.h:668)
==17387== by 0x59356B: mongo::InitializerDependencyGraph::addInitializer(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<mongo::Status (mongo::InitializerContext*)> const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) (initializer_dependency_graph.cpp:46)
==17387== by 0x591057: mongo::GlobalInitializerRegisterer::GlobalInitializerRegisterer(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<mongo::Status (mongo::InitializerContext*)> const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) (global_initializer_registerer.cpp:44)
==17387== by 0x52D46F: __static_initialization_and_destruction_0(int, int) [clone .constprop.34] (mongos_options_init.cpp:39)
==17387== by 0x137FED3: __libc_csu_init (in /home/acm/opt/src/mongo/mongos)
==17387== by 0x4F830A7: generic_start_main.isra.0 (libc-start.c:247)
==17387== by 0x4F83337: (below main) (libc-start.c:116)
So, that is a puzzle. However, I can instruct valgrind to ignore that. But it still fails to start, now with something more odd:
$ valgrind --show-mismatched-frees=no --soname-synonyms=somalloc=NONE --track-origins=yes --leak-check=no ./mongos
==19834== Memcheck, a memory error detector
==19834== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==19834== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==19834== Command: ./mongos
==19834==
MC_(get_otrack_shadow_offset)(ppc64)(off=1688,sz=8)
Memcheck: mc_machine.c:329 (get_otrack_shadow_offset_wrk): the 'impossible' happened.
host stacktrace:
==19834== at 0x3808D9B8: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
==19834== by 0x3808DB5F: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
==19834== by 0x3808DCDB: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
==19834== by 0x38078CE3: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
==19834== by 0x38076FAB: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
==19834== by 0x380BAA2B: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
==19834== by 0x381B9BB7: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
==19834== by 0x380BE19F: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
==19834== by 0x3810D04F: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
==19834== by 0x3810FFEF: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
==19834== by 0x3812BB97: ??? (in /usr/lib/valgrind/memcheck-ppc64le-linux)
sched status:
running_tid=1
Thread 1: status = VgTs_Runnable (lwpid 19834)
==19834== at 0x4F3AC14: __lll_lock_elision (elision-lock.c:60)
==19834== by 0x4F2BBC7: pthread_mutex_lock (pthread_mutex_lock.c:92)
==19834== by 0x602753: mongo::DBConnectionPool::DBConnectionPool() (connpool.cpp:196)
==19834== by 0x5319EB: __static_initialization_and_destruction_0 (global_conn_pool.cpp:35)
==19834== by 0x5319EB: _GLOBAL__sub_I__ZN5mongo14globalConnPoolE (global_conn_pool.cpp:39)
==19834== by 0x137FED3: __libc_csu_init (in /home/acm/opt/src/mongo/mongos)
==19834== by 0x4F830A7: generic_start_main.isra.0 (libc-start.c:247)
==19834== by 0x4F83337: (below main) (libc-start.c:116)
You should be able to see the complete discussion of the issue at
https://bugs.launchpad.net/ubuntu/+source/gcc-5/+bug/1640518
If you have any insight on the errors or suggestions on how to proceed
with Valgrind, that would be appreciated.
Carl Love
|
|
From: <sv...@va...> - 2016-11-07 20:01:54
|
Author: carll
Date: Mon Nov 7 20:01:47 2016
New Revision: 16119
Log:
Fix xxsel parsing error
VEX commit 3284 listed Bugzilla 148000 which is incorrect.
Tweeked comments so the file VEX/priv/guest_ppc_toIR.c can be committed again.
VEX commit 3285.
Bugzilla is 372185.
Modified:
trunk/NEWS
Modified: trunk/NEWS
==============================================================================
--- trunk/NEWS (original)
+++ trunk/NEWS Mon Nov 7 20:01:47 2016
@@ -274,6 +274,8 @@
369468 Remove quadratic metapool algorithm using VG_(HT_remove_at_Iter)
370265 ISA 3.0 HW cap stuff needs updating
371128 BCD add and subtract instructions on Power BE in 32-bit mode do not work
+372195 Power PC, xxsel instruction is not always recognized
+
n-i-bz Fix incorrect (or infinite loop) unwind on RHEL7 x86 and amd64
n-i-bz massif --pages-as-heap=yes does not report peak caused by mmap+munmap
n-i-bz false positive leaks due to aspacemgr merging heap & non heap segments
|