You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(122) |
Nov
(152) |
Dec
(69) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(6) |
Feb
(25) |
Mar
(73) |
Apr
(82) |
May
(24) |
Jun
(25) |
Jul
(10) |
Aug
(11) |
Sep
(10) |
Oct
(54) |
Nov
(203) |
Dec
(182) |
| 2004 |
Jan
(307) |
Feb
(305) |
Mar
(430) |
Apr
(312) |
May
(187) |
Jun
(342) |
Jul
(487) |
Aug
(637) |
Sep
(336) |
Oct
(373) |
Nov
(441) |
Dec
(210) |
| 2005 |
Jan
(385) |
Feb
(480) |
Mar
(636) |
Apr
(544) |
May
(679) |
Jun
(625) |
Jul
(810) |
Aug
(838) |
Sep
(634) |
Oct
(521) |
Nov
(965) |
Dec
(543) |
| 2006 |
Jan
(494) |
Feb
(431) |
Mar
(546) |
Apr
(411) |
May
(406) |
Jun
(322) |
Jul
(256) |
Aug
(401) |
Sep
(345) |
Oct
(542) |
Nov
(308) |
Dec
(481) |
| 2007 |
Jan
(427) |
Feb
(326) |
Mar
(367) |
Apr
(255) |
May
(244) |
Jun
(204) |
Jul
(223) |
Aug
(231) |
Sep
(354) |
Oct
(374) |
Nov
(497) |
Dec
(362) |
| 2008 |
Jan
(322) |
Feb
(482) |
Mar
(658) |
Apr
(422) |
May
(476) |
Jun
(396) |
Jul
(455) |
Aug
(267) |
Sep
(280) |
Oct
(253) |
Nov
(232) |
Dec
(304) |
| 2009 |
Jan
(486) |
Feb
(470) |
Mar
(458) |
Apr
(423) |
May
(696) |
Jun
(461) |
Jul
(551) |
Aug
(575) |
Sep
(134) |
Oct
(110) |
Nov
(157) |
Dec
(102) |
| 2010 |
Jan
(226) |
Feb
(86) |
Mar
(147) |
Apr
(117) |
May
(107) |
Jun
(203) |
Jul
(193) |
Aug
(238) |
Sep
(300) |
Oct
(246) |
Nov
(23) |
Dec
(75) |
| 2011 |
Jan
(133) |
Feb
(195) |
Mar
(315) |
Apr
(200) |
May
(267) |
Jun
(293) |
Jul
(353) |
Aug
(237) |
Sep
(278) |
Oct
(611) |
Nov
(274) |
Dec
(260) |
| 2012 |
Jan
(303) |
Feb
(391) |
Mar
(417) |
Apr
(441) |
May
(488) |
Jun
(655) |
Jul
(590) |
Aug
(610) |
Sep
(526) |
Oct
(478) |
Nov
(359) |
Dec
(372) |
| 2013 |
Jan
(467) |
Feb
(226) |
Mar
(391) |
Apr
(281) |
May
(299) |
Jun
(252) |
Jul
(311) |
Aug
(352) |
Sep
(481) |
Oct
(571) |
Nov
(222) |
Dec
(231) |
| 2014 |
Jan
(185) |
Feb
(329) |
Mar
(245) |
Apr
(238) |
May
(281) |
Jun
(399) |
Jul
(382) |
Aug
(500) |
Sep
(579) |
Oct
(435) |
Nov
(487) |
Dec
(256) |
| 2015 |
Jan
(338) |
Feb
(357) |
Mar
(330) |
Apr
(294) |
May
(191) |
Jun
(108) |
Jul
(142) |
Aug
(261) |
Sep
(190) |
Oct
(54) |
Nov
(83) |
Dec
(22) |
| 2016 |
Jan
(49) |
Feb
(89) |
Mar
(33) |
Apr
(50) |
May
(27) |
Jun
(34) |
Jul
(53) |
Aug
(53) |
Sep
(98) |
Oct
(206) |
Nov
(93) |
Dec
(53) |
| 2017 |
Jan
(65) |
Feb
(82) |
Mar
(102) |
Apr
(86) |
May
(187) |
Jun
(67) |
Jul
(23) |
Aug
(93) |
Sep
(65) |
Oct
(45) |
Nov
(35) |
Dec
(17) |
| 2018 |
Jan
(26) |
Feb
(35) |
Mar
(38) |
Apr
(32) |
May
(8) |
Jun
(43) |
Jul
(27) |
Aug
(30) |
Sep
(43) |
Oct
(42) |
Nov
(38) |
Dec
(67) |
| 2019 |
Jan
(32) |
Feb
(37) |
Mar
(53) |
Apr
(64) |
May
(49) |
Jun
(18) |
Jul
(14) |
Aug
(53) |
Sep
(25) |
Oct
(30) |
Nov
(49) |
Dec
(31) |
| 2020 |
Jan
(87) |
Feb
(45) |
Mar
(37) |
Apr
(51) |
May
(99) |
Jun
(36) |
Jul
(11) |
Aug
(14) |
Sep
(20) |
Oct
(24) |
Nov
(40) |
Dec
(23) |
| 2021 |
Jan
(14) |
Feb
(53) |
Mar
(85) |
Apr
(15) |
May
(19) |
Jun
(3) |
Jul
(14) |
Aug
(1) |
Sep
(57) |
Oct
(73) |
Nov
(56) |
Dec
(22) |
| 2022 |
Jan
(3) |
Feb
(22) |
Mar
(6) |
Apr
(55) |
May
(46) |
Jun
(39) |
Jul
(15) |
Aug
(9) |
Sep
(11) |
Oct
(34) |
Nov
(20) |
Dec
(36) |
| 2023 |
Jan
(79) |
Feb
(41) |
Mar
(99) |
Apr
(169) |
May
(48) |
Jun
(16) |
Jul
(16) |
Aug
(57) |
Sep
(19) |
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
|
|
|
1
(3) |
2
|
3
|
4
|
|
5
(13) |
6
(2) |
7
(5) |
8
(4) |
9
(3) |
10
(4) |
11
(4) |
|
12
(7) |
13
|
14
(1) |
15
|
16
|
17
(2) |
18
|
|
19
|
20
|
21
(3) |
22
(8) |
23
(7) |
24
(5) |
25
(4) |
|
26
(6) |
27
|
28
(9) |
29
|
30
(4) |
31
(5) |
|
|
From: Nicholas N. <nj...@so...> - 2023-03-28 22:18:02
|
https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=2d97182f0dec988fd43cba3b6ac79641284b23af commit 2d97182f0dec988fd43cba3b6ac79641284b23af Author: Nicholas Nethercote <n.n...@gm...> Date: Wed Mar 29 09:15:56 2023 +1100 cg_annotate: use `<unspecified>` for an unspecified filename. Users shouldn't ever see this, but it's useful to distinguish this malformed data file case from the missing symbol case (which is still shown as `???`). Diff: --- cachegrind/cg_annotate.in | 4 ++-- cachegrind/tests/ann2.cgout | 2 +- cachegrind/tests/ann2.post.exp | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/cachegrind/cg_annotate.in b/cachegrind/cg_annotate.in index 2ebd93911b..d5f1e27bd2 100755 --- a/cachegrind/cg_annotate.in +++ b/cachegrind/cg_annotate.in @@ -371,8 +371,8 @@ def read_cgout_file() -> tuple[str, str, Events, DictFlfnCc, DictFlDictLineCc, C elif line.startswith("fl="): curr_fl = line[3:-1] - # A `fn=` line should follow, overwriting the "???". - curr_flfn = Flfn((curr_fl, "???")) + # A `fn=` line should follow, overwriting the function name. + curr_flfn = Flfn((curr_fl, "<unspecified>")) elif m := re.match(r"summary:\s+(.*)", line): try: diff --git a/cachegrind/tests/ann2.cgout b/cachegrind/tests/ann2.cgout index 08ddddc317..2bb2bf8eb8 100644 --- a/cachegrind/tests/ann2.cgout +++ b/cachegrind/tests/ann2.cgout @@ -82,7 +82,7 @@ fn=f1 # File found in ann2-aux/, via -I. fl=ann2-via-I.rs -fn=f1 +# No `fn=` line, so the function name falls back to `<unspecified>`. 1 1000 500 0 # File below the threshold. (It also doesn't exist, but that doesn't matter. We diff --git a/cachegrind/tests/ann2.post.exp b/cachegrind/tests/ann2.post.exp index 0e85dcb556..6fcbe12f62 100644 --- a/cachegrind/tests/ann2.post.exp +++ b/cachegrind/tests/ann2.post.exp @@ -30,7 +30,7 @@ A SomeCount VeryLongEventName file:function 15,000 (15.0%) 600 (0.6%) 0 ann2-basic.rs:f1 9,000 (9.0%) 6,000 (6.0%) 0 ann2-could-not-be-found.rs:f1 2,000 (2.0%) 100 (0.1%) 0 ann2-basic.rs:f2 - 1,000 (1.0%) 500 (0.5%) 0 ann2-via-I.rs:f1 + 1,000 (1.0%) 500 (0.5%) 0 ann2-via-I.rs:<unspecified> 1,000 (1.0%) 300 (0.3%) -1,000 (n/a) ann2-past-the-end.rs:f1 -1,000 (-1.0%) 0 0 ann2-negatives.rs:neg3 -1,000 (-1.0%) 0 0 ann2-negatives.rs:neg2 |
|
From: Nicholas N. <n.n...@gm...> - 2023-03-28 22:04:03
|
Hi,
I recently rewrote `cg_annotate`, `cg_diff`, and `cg_merge` in Python. The
old versions were written in Perl, Perl, and C, respectively. The new
versions are much nicer and easier to modify, and I have various ideas for
improving `cg_annotate`. This email is about one of those ideas.
A typical way to invoke `cg_annotate` is like this:
> cg_annotate cachegrind.out.12345
This implies `--auto=yes`, which requests line-by-line "auto-annotation" of
source files. I.e. `cg_annotate` will automatically annotate all files in
the profile that meet the significance threshold.
It's also possible to do something like this:
> cg_annotate --auto=no cachegrind.out.12345 a.c b.c
Which instead requests "user annotation" of the files `a.c` and `b.c`.
My thesis is that auto-annotation suffices in practice for all reasonable
use cases, and that user annotation is unnecessary and can be removed.
When I first wrote `cg_annotate` in 2002, only user annotation was
implemented. Shortly after, I added the `--auto={yes,no}` option. Since
then I've never used user annotation, and I suspect nobody else has either.
User annotation is ok when dealing with tiny programs, but as soon as you
are profiling a program with more than a handful of source files it becomes
impractical.
The only possible use cases I can think of for user annotation are as
follows.
- If you want to see a particular file(s) annotated but you don't want
to see any others, then you can use user annotation in combination with
`--auto=no`. But it's trivial to search through the output for the
particular file, so this doesn't seem important.
- If the path to a file is somehow really messed up in the debug info,
it might be possible that auto-annotation would fail to find it, but user
annotation could find it, possibly in combination with `-I`. But this seems
unlikely. Some basic testing shows that gcc, clang and rustc all default to
using full paths in debug info. gcc supports `-fdebug-prefix-map` but that
seems to mostly be used for changing full paths to relative paths, which
will still work fine.
Removing user annotation would (a) simplify the code and docs, and (b)
enable the possibility of moving the merge functionality from `cg_merge`
into `cg_annotate`, by allowing the user to specify multiple cachegrind.out
files as input.
So: is anybody using user annotation? Does anybody see any problems with
this proposal?
Thanks.
Nick
|
|
From: Nicholas N. <nj...@so...> - 2023-03-28 06:23:26
|
https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=6436be0a3ff05e72c097dab960347726c32c8890 commit 6436be0a3ff05e72c097dab960347726c32c8890 Author: Nicholas Nethercote <n.n...@gm...> Date: Tue Mar 28 17:22:42 2023 +1100 cg_annotate.in: fix a small bug in the printing of past-the-end lines. Diff: --- cachegrind/cg_annotate.in | 27 ++++++++++++++------------- cachegrind/tests/ann2.post.exp | 2 +- 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/cachegrind/cg_annotate.in b/cachegrind/cg_annotate.in index 240e069ccb..2ebd93911b 100755 --- a/cachegrind/cg_annotate.in +++ b/cachegrind/cg_annotate.in @@ -686,18 +686,17 @@ def print_annotated_src_file( pairs.append((lo, hi)) i += 1 - # Annotate chosen lines, tracking total annotated counts. - line_num = 0 - if pairs: + def print_lines(pairs: list[tuple[int, int]]) -> None: + line_num = 0 while pairs: + src_line = "" (lo, hi) = pairs.pop(0) while line_num < lo - 1: - tmp = src_file.readline() + src_line = src_file.readline() line_num += 1 - if not tmp: - break # EOF + if not src_line: + return # EOF - src_line = "" # Print line number, unless start of file. if lo != 1: print("-- line", lo, "-" * 40) @@ -706,7 +705,7 @@ def print_annotated_src_file( src_line = src_file.readline() line_num += 1 if not src_line: - break + return # EOF if line_nums and line_num == line_nums[0]: printer.print_cc(dict_line_cc[line_num], src_line[:-1]) annotated_ccs.line_nums_known_cc += dict_line_cc[line_num] @@ -714,14 +713,16 @@ def print_annotated_src_file( else: printer.print_missing_cc(src_line[:-1]) - # Print line number, unless EOF. - if src_line: - print("-- line", hi, "-" * 40) - else: - break + # Print line number. + print("-- line", hi, "-" * 40) + + # Annotate chosen lines, tracking total annotated counts. + if pairs: + print_lines(pairs) # If there was info on lines past the end of the file, warn. if line_nums: + print() for line_num in line_nums: printer.print_cc(dict_line_cc[line_num], f"<bogus line {line_num}>") annotated_ccs.line_nums_known_cc += dict_line_cc[line_num] diff --git a/cachegrind/tests/ann2.post.exp b/cachegrind/tests/ann2.post.exp index 315e13474d..0e85dcb556 100644 --- a/cachegrind/tests/ann2.post.exp +++ b/cachegrind/tests/ann2.post.exp @@ -126,7 +126,7 @@ A SomeCount VeryLongEventName . . . two . . . three -- line 3 ---------------------------------------- --- line 18 ---------------------------------------- + 300 (0.3%) 100 (0.1%) 0 <bogus line 20> 300 (0.3%) 100 (0.1%) 0 <bogus line 21> 200 (0.2%) 0 -1,000 (n/a) <bogus line 22> |
|
From: Nicholas N. <nj...@so...> - 2023-03-28 05:30:41
|
https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=ca44cbbd7b909c083a55291df2fd0d13db9db845 commit ca44cbbd7b909c083a55291df2fd0d13db9db845 Author: Nicholas Nethercote <n.n...@gm...> Date: Tue Mar 28 16:13:27 2023 +1100 Fix some problems in `cachegrind/tests/Makefile.am`. Diff: --- cachegrind/tests/Makefile.am | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/cachegrind/tests/Makefile.am b/cachegrind/tests/Makefile.am index 3aa85c9990..8e8deb363b 100644 --- a/cachegrind/tests/Makefile.am +++ b/cachegrind/tests/Makefile.am @@ -15,11 +15,15 @@ dist_noinst_SCRIPTS = filter_stderr filter_cachesim_discards EXTRA_DIST = \ ann-diff1.post.exp ann-diff1.stderr.exp ann-diff1.vgtest \ ann-diff2a.cgout ann-diff2b.cgout \ - ann-merge1.post.exp ann-merge1.stderr.exp ann-merge1.vgtest + ann-merge1.post.exp ann-merge1.stderr.exp ann-merge1.vgtest \ ann-merge1a.cgout ann-merge1b.cgout \ + ann-merge-x.rs ann-merge-y.rs \ ann1a.post.exp ann1a.stderr.exp ann1a.vgtest ann1.cgout \ ann1b.post.exp ann1b.stderr.exp ann1b.vgtest ann1b.cgout \ ann2.post.exp ann2.stderr.exp ann2.vgtest ann2.cgout \ + ann2-basic.rs ann2-more-recent-than-cgout.rs \ + ann2-negatives.rs ann2-past-the-end.rs \ + ann2-unmentioned.rs ann2-aux/ann2-via-I.rs \ chdir.vgtest chdir.stderr.exp \ clreq.vgtest clreq.stderr.exp \ dlclose.vgtest dlclose.stderr.exp dlclose.stdout.exp \ |
|
From: Nicholas N. <nj...@so...> - 2023-03-28 04:46:05
|
https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=e95328b12c9151919fd327ba2558ff4c0e9eb124 commit e95328b12c9151919fd327ba2558ff4c0e9eb124 Author: Nicholas Nethercote <n.n...@gm...> Date: Tue Mar 28 15:44:34 2023 +1100 Simpler and more consistent `cachegrind/tests/*.vgtest` files. - Always use `python3`, never `python`. - Avoid unnecessary `../cachegrind/` in paths. Diff: --- cachegrind/tests/ann-diff1.vgtest | 2 +- cachegrind/tests/ann-diff2.vgtest | 2 +- cachegrind/tests/ann-merge1.vgtest | 2 +- cachegrind/tests/ann1a.vgtest | 2 +- cachegrind/tests/ann1b.vgtest | 2 +- cachegrind/tests/ann2.vgtest | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/cachegrind/tests/ann-diff1.vgtest b/cachegrind/tests/ann-diff1.vgtest index b737b713e3..e379401876 100644 --- a/cachegrind/tests/ann-diff1.vgtest +++ b/cachegrind/tests/ann-diff1.vgtest @@ -2,5 +2,5 @@ # the post-processing of the `ann{1,1b}.cgout` test files. prog: ../../tests/true vgopts: --cachegrind-out-file=cachegrind.out -post: python ../../cachegrind/cg_diff --mod-funcname="s/main/MAIN/" ann1.cgout ann1b.cgout > ann-diff1.cgout && python ../../cachegrind/cg_annotate ann-diff1.cgout +post: python3 ../cg_diff --mod-funcname="s/main/MAIN/" ann1.cgout ann1b.cgout > ann-diff1.cgout && python3 ../cg_annotate ann-diff1.cgout cleanup: rm ann-diff1.cgout diff --git a/cachegrind/tests/ann-diff2.vgtest b/cachegrind/tests/ann-diff2.vgtest index 101cac07d1..7b395e4e48 100644 --- a/cachegrind/tests/ann-diff2.vgtest +++ b/cachegrind/tests/ann-diff2.vgtest @@ -2,5 +2,5 @@ # the post-processing of the `ann-diff2{a,b}.cgout` test files. prog: ../../tests/true vgopts: --cachegrind-out-file=cachegrind.out -post: python ../../cachegrind/cg_diff --mod-filename="s/.*aux\//aux\//i" --mod-funcname="s/(f[a-z]*)[0-9]/\1N/g" ann-diff2a.cgout ann-diff2b.cgout > ann-diff2c.cgout && python ../../cachegrind/cg_annotate ann-diff2c.cgout +post: python3 ../cg_diff --mod-filename="s/.*aux\//aux\//i" --mod-funcname="s/(f[a-z]*)[0-9]/\1N/g" ann-diff2a.cgout ann-diff2b.cgout > ann-diff2c.cgout && python3 ../cg_annotate ann-diff2c.cgout cleanup: rm ann-diff2c.cgout diff --git a/cachegrind/tests/ann-merge1.vgtest b/cachegrind/tests/ann-merge1.vgtest index 5862828a3f..b0b0eedc96 100644 --- a/cachegrind/tests/ann-merge1.vgtest +++ b/cachegrind/tests/ann-merge1.vgtest @@ -2,6 +2,6 @@ # the post-processing of the `ann{1,1b}.cgout` test files. prog: ../../tests/true vgopts: --cachegrind-out-file=cachegrind.out -post: python ../../cachegrind/cg_merge ann-merge1a.cgout ann-merge1b.cgout > ann-merge1c.cgout && python ../../cachegrind/cg_annotate ann-merge1c.cgout +post: python3 ../cg_merge ann-merge1a.cgout ann-merge1b.cgout > ann-merge1c.cgout && python3 ../cg_annotate ann-merge1c.cgout cleanup: rm ann-merge1c.cgout diff --git a/cachegrind/tests/ann1a.vgtest b/cachegrind/tests/ann1a.vgtest index 1774f1b1c4..740522f6dd 100644 --- a/cachegrind/tests/ann1a.vgtest +++ b/cachegrind/tests/ann1a.vgtest @@ -2,5 +2,5 @@ # the post-processing of the `ann1.cgout` file. prog: ../../tests/true vgopts: --cachegrind-out-file=cachegrind.out -post: touch ann1.cgout && python3 ../../cachegrind/cg_annotate --show=Ir,I1mr,ILmr --show-percs=no ann1.cgout +post: touch ann1.cgout && python3 ../cg_annotate --show=Ir,I1mr,ILmr --show-percs=no ann1.cgout cleanup: rm cachegrind.out diff --git a/cachegrind/tests/ann1b.vgtest b/cachegrind/tests/ann1b.vgtest index 5eacaa8b10..2b51af9ce3 100644 --- a/cachegrind/tests/ann1b.vgtest +++ b/cachegrind/tests/ann1b.vgtest @@ -2,5 +2,5 @@ # the post-processing of the `ann1.cgout` file. prog: ../../tests/true vgopts: --cachegrind-out-file=cachegrind.out -post: touch ann1.cgout && python3 ../../cachegrind/cg_annotate --sort=Dr --show=Dw,Dr,Ir --auto=no ann1.cgout a.c +post: touch ann1.cgout && python3 ../cg_annotate --sort=Dr --show=Dw,Dr,Ir --auto=no ann1.cgout a.c cleanup: rm cachegrind.out diff --git a/cachegrind/tests/ann2.vgtest b/cachegrind/tests/ann2.vgtest index b2ec364c72..8a09e28cb4 100644 --- a/cachegrind/tests/ann2.vgtest +++ b/cachegrind/tests/ann2.vgtest @@ -8,6 +8,6 @@ vgopts: --cachegrind-out-file=cachegrind.out # The `sleep` is to ensure the mtime of the second touched file is greater than # the mtime of the first touched file. -post: touch ann2.cgout && sleep 0.1 && touch ann2-more-recent-than-cgout.rs && python3 ../../cachegrind/cg_annotate --context 2 --auto --show-percs=yes --threshold=0.5 -Iann2-no-such-dir --include ann2-no-such-dir-2 -I=ann2-aux ann2.cgout ann2-unmentioned.rs ann2-no-such-file.rs +post: touch ann2.cgout && sleep 0.1 && touch ann2-more-recent-than-cgout.rs && python3 ../cg_annotate --context 2 --auto --show-percs=yes --threshold=0.5 -Iann2-no-such-dir --include ann2-no-such-dir-2 -I=ann2-aux ann2.cgout ann2-unmentioned.rs ann2-no-such-file.rs cleanup: rm cachegrind.out |
|
From: Nicholas N. <nj...@so...> - 2023-03-28 04:37:28
|
https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=551874920f7f49fcf18b69e2a3f23d948db2c50b commit 551874920f7f49fcf18b69e2a3f23d948db2c50b Author: Nicholas Nethercote <n.n...@gm...> Date: Mon Mar 27 17:27:56 2023 +1100 Rewrite `cg_merge` in Python. It's currently written in C, but `cg_annotate` and `cg_diff` are written in Python. It's better to have them all in the same language. The good news is that the Python code is 4.5x shorter than the C code. The bad news is that the Python code is roughly 3x slower than the C code. But `cg_merge` isn't used that often, so I think it's a reasonable trade-off. Diff: --- cachegrind/Makefile.am | 27 +- cachegrind/cg_merge.c | 1580 -------------------------------- cachegrind/cg_merge.in | 339 +++++++ cachegrind/tests/Makefile.am | 2 + cachegrind/tests/ann-merge-x.rs | 5 + cachegrind/tests/ann-merge-y.rs | 6 + cachegrind/tests/ann-merge1.post.exp | 66 ++ cachegrind/tests/ann-merge1.stderr.exp | 17 + cachegrind/tests/ann-merge1.vgtest | 7 + cachegrind/tests/ann-merge1a.cgout | 19 + cachegrind/tests/ann-merge1b.cgout | 14 + configure.ac | 1 + 12 files changed, 482 insertions(+), 1601 deletions(-) diff --git a/cachegrind/Makefile.am b/cachegrind/Makefile.am index 34717ae541..cbaa90522b 100644 --- a/cachegrind/Makefile.am +++ b/cachegrind/Makefile.am @@ -10,32 +10,13 @@ EXTRA_DIST = \ # Headers, etc #---------------------------------------------------------------------------- -bin_SCRIPTS = cg_annotate cg_diff +bin_SCRIPTS = cg_annotate cg_diff cg_merge noinst_HEADERS = \ cg_arch.h \ cg_branchpred.c \ cg_sim.c -#---------------------------------------------------------------------------- -# cg_merge (built for the primary target only) -#---------------------------------------------------------------------------- - -bin_PROGRAMS = cg_merge - -cg_merge_SOURCES = cg_merge.c -cg_merge_CPPFLAGS = $(AM_CPPFLAGS_PRI) -cg_merge_CFLAGS = $(AM_CFLAGS_PRI) -cg_merge_CCASFLAGS = $(AM_CCASFLAGS_PRI) -cg_merge_LDFLAGS = $(AM_CFLAGS_PRI) -# If there is no secondary platform, and the platforms include x86-darwin, -# then the primary platform must be x86-darwin. Hence: -if ! VGCONF_HAVE_PLATFORM_SEC -if VGCONF_PLATFORMS_INCLUDE_X86_DARWIN -cg_merge_LDFLAGS += -Wl,-read_only_relocs -Wl,suppress -endif -endif - #---------------------------------------------------------------------------- # cachegrind-<platform> #---------------------------------------------------------------------------- @@ -101,4 +82,8 @@ pyann: pydiff: +../auxprogs/pybuild.sh cg_diff.in cg_diff -.PHONY: pyann pydiff +# "Build" `cg_merge`. The `+` avoids warnings about the jobserver. +pymerge: + +../auxprogs/pybuild.sh cg_merge.in cg_merge + +.PHONY: pyann pydiff pymerge diff --git a/cachegrind/cg_merge.c b/cachegrind/cg_merge.c deleted file mode 100644 index 4d13cb5d19..0000000000 --- a/cachegrind/cg_merge.c +++ /dev/null @@ -1,1580 +0,0 @@ - -/*--------------------------------------------------------------------*/ -/*--- A program that merges multiple cachegrind output files. ---*/ -/*--- cg_merge.c ---*/ -/*--------------------------------------------------------------------*/ - -/* - This file is part of Cachegrind, a Valgrind tool for cache - profiling programs. - - Copyright (C) 2002-2017 Nicholas Nethercote - nj...@va... - - AVL tree code derived from - ANSI C Library for maintenance of AVL Balanced Trees - (C) 2000 Daniel Nagy, Budapest University of Technology and Economics - Released under GNU General Public License (GPL) version 2 - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of the - License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, but - WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, see <http://www.gnu.org/licenses/>. - - The GNU General Public License is contained in the file COPYING. -*/ - -#include <stdio.h> -#include <stdlib.h> -#include <assert.h> -#include <string.h> -#include <ctype.h> - -typedef signed long Word; -typedef unsigned long UWord; -typedef unsigned char Bool; -#define True ((Bool)1) -#define False ((Bool)0) -typedef signed int Int; -typedef unsigned int UInt; -typedef unsigned long long int ULong; -typedef signed char Char; -typedef size_t SizeT; - - -//------------------------------------------------------------------// -//--- WordFM ---// -//--- Public interface ---// -//------------------------------------------------------------------// - -typedef struct _WordFM WordFM; /* opaque */ - -/* Initialise a WordFM */ -void initFM ( WordFM* t, - void* (*alloc_nofail)( SizeT ), - void (*dealloc)(void*), - Word (*kCmp)(Word,Word) ); - -/* Allocate and initialise a WordFM */ -WordFM* newFM( void* (*alloc_nofail)( SizeT ), - void (*dealloc)(void*), - Word (*kCmp)(Word,Word) ); - -/* Free up the FM. If kFin is non-NULL, it is applied to keys - before the FM is deleted; ditto with vFin for vals. */ -void deleteFM ( WordFM*, void(*kFin)(Word), void(*vFin)(Word) ); - -/* Add (k,v) to fm. If a binding for k already exists, it is updated - to map to this new v. In that case we should really return the - previous v so that caller can finalise it. Oh well. */ -void addToFM ( WordFM* fm, Word k, Word v ); - -// Delete key from fm, returning associated val if found -Bool delFromFM ( WordFM* fm, /*OUT*/Word* oldV, Word key ); - -// Look up in fm, assigning found val at spec'd address -Bool lookupFM ( WordFM* fm, /*OUT*/Word* valP, Word key ); - -Word sizeFM ( WordFM* fm ); - -// set up FM for iteration -void initIterFM ( WordFM* fm ); - -// get next key/val pair. Will assert if fm has been modified -// or looked up in since initIterFM was called. -Bool nextIterFM ( WordFM* fm, /*OUT*/Word* pKey, /*OUT*/Word* pVal ); - -// clear the I'm iterating flag -void doneIterFM ( WordFM* fm ); - -// Deep copy a FM. If dopyK is NULL, keys are copied verbatim. -// If non-null, dopyK is applied to each key to generate the -// version in the new copy. In that case, if the argument to dopyK -// is non-NULL but the result is NULL, it is assumed that dopyK -// could not allocate memory, in which case the copy is abandoned -// and NULL is returned. Ditto with dopyV for values. -WordFM* dopyFM ( WordFM* fm, Word(*dopyK)(Word), Word(*dopyV)(Word) ); - -//------------------------------------------------------------------// -//--- end WordFM ---// -//--- Public interface ---// -//------------------------------------------------------------------// - - -static const char* argv0 = "cg_merge"; - -/* Keep track of source filename/line no so as to be able to - print decent error messages. */ -typedef - struct { - FILE* fp; - UInt lno; - char* filename; - } - SOURCE; - -static void printSrcLoc ( SOURCE* s ) -{ - fprintf(stderr, "%s: near %s line %u\n", argv0, s->filename, s->lno-1); -} - -__attribute__((noreturn)) -static void mallocFail ( SOURCE* s, const char* who ) -{ - fprintf(stderr, "%s: out of memory in %s\n", argv0, who ); - printSrcLoc( s ); - exit(2); -} - -__attribute__((noreturn)) -static void parseError ( SOURCE* s, const char* msg ) -{ - fprintf(stderr, "%s: parse error: %s\n", argv0, msg ); - printSrcLoc( s ); - exit(1); -} - -__attribute__((noreturn)) -static void barf ( SOURCE* s, const char* msg ) -{ - fprintf(stderr, "%s: %s\n", argv0, msg ); - printSrcLoc( s ); - exit(1); -} - -// Read a line. Return the line read, or NULL if at EOF. -// The line is allocated dynamically but will be overwritten with -// every invocation. Caller must not free it. -static const char *readline ( SOURCE* s ) -{ - static char *line = NULL; - static size_t linesiz = 0; - - int ch, i = 0; - - while (1) { - ch = getc(s->fp); - if (ch != EOF) { - if (i + 1 >= linesiz) { - linesiz += 500; - line = realloc(line, linesiz * sizeof *line); - if (line == NULL) - mallocFail(s, "readline:"); - } - line[i++] = ch; - line[i] = 0; - if (ch == '\n') { - line[i-1] = 0; - s->lno++; - break; - } - } else { - if (ferror(s->fp)) { - perror(argv0); - barf(s, "I/O error while reading input file"); - } else { - // hit EOF - break; - } - } - } - return i == 0 ? NULL : line; -} - -static Bool streqn ( const char* s1, const char* s2, size_t n ) -{ - return 0 == strncmp(s1, s2, n); -} - -static Bool streq ( const char* s1, const char* s2 ) -{ - return 0 == strcmp(s1, s2 ); -} - - -//////////////////////////////////////////////////////////////// - -typedef - struct { - char* fi_name; - char* fn_name; - } - FileFn; - -typedef - struct { - Int n_counts; - ULong* counts; - } - Counts; - -typedef - struct { - // null-terminated vector of desc_lines - char** desc_lines; - - // Cmd line - char* cmd_line; - - // Events line - char* events_line; - Int n_events; - - // Summary line (copied from input) - char* summary_line; - - /* Outermost map is - WordFM FileFn* innerMap - where innerMap is WordFM line-number=UWord Counts */ - WordFM* outerMap; - - // Summary counts (computed whilst parsing) - // should match .summary_line - Counts* summary; - } - CacheProfFile; - -static FileFn* new_FileFn ( char* file_name, char* fn_name ) -{ - FileFn* ffn = malloc(sizeof(FileFn)); - if (ffn == NULL) - return NULL; - ffn->fi_name = file_name; - ffn->fn_name = fn_name; - return ffn; -} - -static void ddel_FileFn ( FileFn* ffn ) -{ - if (ffn->fi_name) - free(ffn->fi_name); - if (ffn->fn_name) - free(ffn->fn_name); - memset(ffn, 0, sizeof(FileFn)); - free(ffn); -} - -static FileFn* dopy_FileFn ( FileFn* ff ) -{ - char *fi2, *fn2; - fi2 = strdup(ff->fi_name); - if (fi2 == NULL) return NULL; - fn2 = strdup(ff->fn_name); - if (fn2 == NULL) { - free(fi2); - return NULL; - } - return new_FileFn( fi2, fn2 ); -} - -static Counts* new_Counts ( Int n_counts, /*COPIED*/ULong* counts ) -{ - Int i; - Counts* cts = malloc(sizeof(Counts)); - if (cts == NULL) - return NULL; - - assert(n_counts >= 0); - cts->counts = malloc(n_counts * sizeof(ULong)); - if (cts->counts == NULL) { - free(cts); - return NULL; - } - - cts->n_counts = n_counts; - for (i = 0; i < n_counts; i++) - cts->counts[i] = counts[i]; - - return cts; -} - -static Counts* new_Counts_Zeroed ( Int n_counts ) -{ - Int i; - Counts* cts = malloc(sizeof(Counts)); - if (cts == NULL) - return NULL; - - assert(n_counts >= 0); - cts->counts = malloc(n_counts * sizeof(ULong)); - if (cts->counts == NULL) { - free(cts); - return NULL; - } - - cts->n_counts = n_counts; - for (i = 0; i < n_counts; i++) - cts->counts[i] = 0; - - return cts; -} - -static void sdel_Counts ( Counts* cts ) -{ - memset(cts, 0, sizeof(Counts)); - free(cts); -} - -static void ddel_Counts ( Counts* cts ) -{ - if (cts->counts) - free(cts->counts); - memset(cts, 0, sizeof(Counts)); - free(cts); -} - -static Counts* dopy_Counts ( Counts* cts ) -{ - return new_Counts( cts->n_counts, cts->counts ); -} - -static -CacheProfFile* new_CacheProfFile ( char** desc_lines, - char* cmd_line, - char* events_line, - Int n_events, - char* summary_line, - WordFM* outerMap, - Counts* summary ) -{ - CacheProfFile* cpf = malloc(sizeof(CacheProfFile)); - if (cpf == NULL) - return NULL; - cpf->desc_lines = desc_lines; - cpf->cmd_line = cmd_line; - cpf->events_line = events_line; - cpf->n_events = n_events; - cpf->summary_line = summary_line; - cpf->outerMap = outerMap; - cpf->summary = summary; - return cpf; -} - -static WordFM* dopy_InnerMap ( WordFM* innerMap ) -{ - return dopyFM ( innerMap, NULL, - (Word(*)(Word))dopy_Counts ); -} - -static void ddel_InnerMap ( WordFM* innerMap ) -{ - deleteFM( innerMap, NULL, (void(*)(Word))ddel_Counts ); -} - -static void ddel_CacheProfFile ( CacheProfFile* cpf ) -{ - char** p; - if (cpf->desc_lines) { - for (p = cpf->desc_lines; *p; p++) - free(*p); - free(cpf->desc_lines); - } - if (cpf->cmd_line) - free(cpf->cmd_line); - if (cpf->events_line) - free(cpf->events_line); - if (cpf->summary_line) - free(cpf->summary_line); - if (cpf->outerMap) - deleteFM( cpf->outerMap, (void(*)(Word))ddel_FileFn, - (void(*)(Word))ddel_InnerMap ); - if (cpf->summary) - ddel_Counts(cpf->summary); - - memset(cpf, 0, sizeof(CacheProfFile)); - free(cpf); -} - -static void showCounts ( FILE* f, Counts* c ) -{ - Int i; - for (i = 0; i < c->n_counts; i++) { - fprintf(f, "%lld ", c->counts[i]); - } -} - -static void show_CacheProfFile ( FILE* f, CacheProfFile* cpf ) -{ - Int i; - char** d; - FileFn* topKey; - WordFM* topVal; - UWord subKey; - Counts* subVal; - - for (d = cpf->desc_lines; *d; d++) - fprintf(f, "%s\n", *d); - fprintf(f, "%s\n", cpf->cmd_line); - fprintf(f, "%s\n", cpf->events_line); - - initIterFM( cpf->outerMap ); - while (nextIterFM( cpf->outerMap, (Word*)(&topKey), (Word*)(&topVal) )) { - fprintf(f, "fl=%s\nfn=%s\n", - topKey->fi_name, topKey->fn_name ); - initIterFM( topVal ); - while (nextIterFM( topVal, (Word*)(&subKey), (Word*)(&subVal) )) { - fprintf(f, "%ld ", subKey ); - showCounts( f, subVal ); - fprintf(f, "\n"); - } - doneIterFM( topVal ); - } - doneIterFM( cpf->outerMap ); - - //fprintf(f, "%s\n", cpf->summary_line); - fprintf(f, "summary:"); - for (i = 0; i < cpf->summary->n_counts; i++) - fprintf(f, " %lld", cpf->summary->counts[i]); - fprintf(f, "\n"); -} - -//////////////////////////////////////////////////////////////// - -static Word cmp_FileFn ( Word s1, Word s2 ) -{ - FileFn* ff1 = (FileFn*)s1; - FileFn* ff2 = (FileFn*)s2; - Word r = strcmp(ff1->fi_name, ff2->fi_name); - if (r == 0) - r = strcmp(ff1->fn_name, ff2->fn_name); - return r; -} - -static Word cmp_unboxed_UWord ( Word s1, Word s2 ) -{ - UWord u1 = (UWord)s1; - UWord u2 = (UWord)s2; - if (u1 < u2) return -1; - if (u1 > u2) return 1; - return 0; -} - -//////////////////////////////////////////////////////////////// - -static Bool parse_ULong ( /*OUT*/ULong* res, /*INOUT*/const char** pptr) -{ - ULong u64; - const char* ptr = *pptr; - while (isspace(*ptr)) ptr++; - if (!isdigit(*ptr)) { - *pptr = ptr; - return False; /* end of string, or junk */ - } - u64 = 0; - while (isdigit(*ptr)) { - u64 = (u64 * 10) + (ULong)(*ptr - '0'); - ptr++; - } - *res = u64; - *pptr = ptr; - return True; -} - -// str is a line of integers, starting with a line number. Parse it, -// returning the first number in *lnno and the rest in a newly -// allocated Counts struct. If lnno is non-NULL, treat the first -// number as a line number and assign it to *lnno instead of -// incorporating it in the counts array. -static -Counts* splitUpCountsLine ( SOURCE* s, /*OUT*/UWord* lnno, const char* str ) -{ - Bool ok; - Counts* counts; - ULong *tmpC = NULL; - UInt n_tmpC = 0, tmpCsize = 0; - while (1) { - if (n_tmpC >= tmpCsize) { - tmpCsize += 50; - tmpC = realloc(tmpC, tmpCsize * sizeof *tmpC); - if (tmpC == NULL) - mallocFail(s, "splitUpCountsLine:"); - } - ok = parse_ULong( &tmpC[n_tmpC], &str ); - if (!ok) - break; - n_tmpC++; - } - if (*str != 0) - parseError(s, "garbage in counts line"); - if (lnno ? (n_tmpC < 2) : (n_tmpC < 1)) - parseError(s, "too few counts in count line"); - - if (lnno) { - *lnno = (UWord)tmpC[0]; - counts = new_Counts( n_tmpC-1, /*COPIED*/&tmpC[1] ); - } else { - counts = new_Counts( n_tmpC, /*COPIED*/&tmpC[0] ); - } - free(tmpC); - - return counts; -} - -static void addCounts ( SOURCE* s, /*OUT*/Counts* counts1, Counts* counts2 ) -{ - Int i; - if (counts1->n_counts != counts2->n_counts) - parseError(s, "addCounts: inconsistent number of counts"); - for (i = 0; i < counts1->n_counts; i++) - counts1->counts[i] += counts2->counts[i]; -} - -static Bool addCountsToMap ( SOURCE* s, - WordFM* counts_map, - UWord lnno, Counts* newCounts ) -{ - Counts* oldCounts; - // look up lnno in the map. If none present, add a binding - // lnno->counts. If present, add counts to the existing entry. - if (lookupFM( counts_map, (Word*)(&oldCounts), (Word)lnno )) { - // merge with existing binding - addCounts( s, oldCounts, newCounts ); - return True; - } else { - // create new binding - addToFM( counts_map, (Word)lnno, (Word)newCounts ); - return False; - } -} - -static -void handle_counts ( SOURCE* s, - CacheProfFile* cpf, - const char* fi, const char* fn, const char* newCountsStr ) -{ - WordFM* countsMap; - Bool freeNewCounts; - UWord lnno; - Counts* newCounts; - FileFn* topKey; - - if (0) printf("%s %s %s\n", fi, fn, newCountsStr ); - - // parse the numbers - newCounts = splitUpCountsLine( s, &lnno, newCountsStr ); - - // Did we get the right number? - if (newCounts->n_counts != cpf->n_events) - goto oom; - - // allocate the key - topKey = malloc(sizeof(FileFn)); - if (topKey) { - topKey->fi_name = strdup(fi); - topKey->fn_name = strdup(fn); - } - if (! (topKey && topKey->fi_name && topKey->fn_name)) - mallocFail(s, "handle_counts:"); - - // search for it - if (lookupFM( cpf->outerMap, (Word*)(&countsMap), (Word)topKey )) { - // found it. Merge in new counts - freeNewCounts = addCountsToMap( s, countsMap, lnno, newCounts ); - ddel_FileFn(topKey); - } else { - // not found in the top map. Create new entry - countsMap = newFM( malloc, free, cmp_unboxed_UWord ); - if (!countsMap) - goto oom; - addToFM( cpf->outerMap, (Word)topKey, (Word)countsMap ); - freeNewCounts = addCountsToMap( s, countsMap, lnno, newCounts ); - } - - // also add to running summary total - addCounts( s, cpf->summary, newCounts ); - - // if safe to do so, free up the count vector - if (freeNewCounts) - ddel_Counts(newCounts); - - return; - - oom: - parseError(s, "# counts doesn't match # events"); -} - - -/* Parse a complete file from the stream in 's'. If a parse error - happens, do not return; instead exit via parseError(). If an - out-of-memory condition happens, do not return; instead exit via - mallocError(). -*/ -static CacheProfFile* parse_CacheProfFile ( SOURCE* s ) -{ - Int i; - char** tmp_desclines = NULL; - unsigned tmp_desclines_size = 0; - char* p; - int n_tmp_desclines = 0; - CacheProfFile* cpf; - Counts* summaryRead; - char* curr_fn = strdup("???"); - char* curr_fl = strdup("???"); - const char* line; - - cpf = new_CacheProfFile( NULL, NULL, NULL, 0, NULL, NULL, NULL ); - if (cpf == NULL) - mallocFail(s, "parse_CacheProfFile(1)"); - - // Parse "desc:" lines - while (1) { - line = readline(s); - if (!line) - break; - if (!streqn(line, "desc: ", 6)) - break; - if (n_tmp_desclines >= tmp_desclines_size) { - tmp_desclines_size += 100; - tmp_desclines = realloc(tmp_desclines, - tmp_desclines_size * sizeof *tmp_desclines); - if (tmp_desclines == NULL) - mallocFail(s, "parse_CacheProfFile(1)"); - } - tmp_desclines[n_tmp_desclines++] = strdup(line); - } - - if (n_tmp_desclines == 0) - parseError(s, "parse_CacheProfFile: no DESC lines present"); - - cpf->desc_lines = malloc( (1+n_tmp_desclines) * sizeof(char*) ); - if (cpf->desc_lines == NULL) - mallocFail(s, "parse_CacheProfFile(2)"); - - cpf->desc_lines[n_tmp_desclines] = NULL; - for (i = 0; i < n_tmp_desclines; i++) - cpf->desc_lines[i] = tmp_desclines[i]; - - // Parse "cmd:" line - if (!streqn(line, "cmd: ", 5)) - parseError(s, "parse_CacheProfFile: no CMD line present"); - - cpf->cmd_line = strdup(line); - if (cpf->cmd_line == NULL) - mallocFail(s, "parse_CacheProfFile(3)"); - - // Parse "events:" line and figure out how many events there are - line = readline(s); - if (!line) - parseError(s, "parse_CacheProfFile: eof before EVENTS line"); - if (!streqn(line, "events: ", 8)) - parseError(s, "parse_CacheProfFile: no EVENTS line present"); - - // figure out how many events there are by counting the number - // of space-alphanum transitions in the events_line - cpf->events_line = strdup(line); - if (cpf->events_line == NULL) - mallocFail(s, "parse_CacheProfFile(3)"); - - cpf->n_events = 0; - assert(cpf->events_line[6] == ':'); - for (p = &cpf->events_line[6]; *p; p++) { - if (p[0] == ' ' && isalpha(p[1])) - cpf->n_events++; - } - - // create the running cross-check summary - cpf->summary = new_Counts_Zeroed( cpf->n_events ); - if (cpf->summary == NULL) - mallocFail(s, "parse_CacheProfFile(4)"); - - // create the outer map (file+fn name --> inner map) - cpf->outerMap = newFM ( malloc, free, cmp_FileFn ); - if (cpf->outerMap == NULL) - mallocFail(s, "parse_CacheProfFile(5)"); - - // process count lines - while (1) { - line = readline(s); - if (!line) - parseError(s, "parse_CacheProfFile: eof before SUMMARY line"); - - if (isdigit(line[0])) { - handle_counts(s, cpf, curr_fl, curr_fn, line); - continue; - } - else - if (streqn(line, "fn=", 3)) { - free(curr_fn); - curr_fn = strdup(line+3); - continue; - } - else - if (streqn(line, "fl=", 3)) { - free(curr_fl); - curr_fl = strdup(line+3); - continue; - } - else - if (streqn(line, "summary: ", 9)) { - break; - } - else - parseError(s, "parse_CacheProfFile: unexpected line in main data"); - } - - // finally, the "summary:" line - if (!streqn(line, "summary: ", 9)) - parseError(s, "parse_CacheProfFile: missing SUMMARY line"); - - cpf->summary_line = strdup(line); - if (cpf->summary_line == NULL) - mallocFail(s, "parse_CacheProfFile(6)"); - - // there should be nothing more - line = readline(s); - if (line) - parseError(s, "parse_CacheProfFile: " - "extraneous content after SUMMARY line"); - - // check the summary counts are as expected - summaryRead = splitUpCountsLine( s, NULL, &cpf->summary_line[8] ); - if (summaryRead == NULL) - mallocFail(s, "parse_CacheProfFile(7)"); - if (summaryRead->n_counts != cpf->n_events) - parseError(s, "parse_CacheProfFile: wrong # counts in SUMMARY line"); - for (i = 0; i < summaryRead->n_counts; i++) { - if (summaryRead->counts[i] != cpf->summary->counts[i]) { - parseError(s, "parse_CacheProfFile: " - "computed vs stated SUMMARY counts mismatch"); - } - } - free(summaryRead->counts); - sdel_Counts(summaryRead); - - // since the summary counts are OK, free up the summary_line text - // which contains the same info. - free(cpf->summary_line); - cpf->summary_line = NULL; - - free(tmp_desclines); - free(curr_fn); - free(curr_fl); - - // All looks OK - return cpf; -} - - -static void merge_CacheProfInfo ( SOURCE* s, - /*MOD*/CacheProfFile* dst, - CacheProfFile* src ) -{ - /* For each (filefn, innerMap) in src - if filefn not in dst - add binding dopy(filefn)->dopy(innerMap) in src - else - // merge src->innerMap with dst->innerMap - for each (lineno, counts) in src->innerMap - if lineno not in dst->innerMap - add binding lineno->dopy(counts) to dst->innerMap - else - add counts into dst->innerMap[lineno] - */ - /* Outer iterator: FileFn* -> WordFM* (inner iterator) - Inner iterator: UWord -> Counts* - */ - FileFn* soKey; - WordFM* soVal; - WordFM* doVal; - UWord siKey; - Counts* siVal; - Counts* diVal; - - /* First check mundane things: that the events: lines are - identical. */ - if (!streq( dst->events_line, src->events_line )) - barf(s, "\"events:\" line of most recent file does " - "not match those previously processed"); - - initIterFM( src->outerMap ); - - // for (filefn, innerMap) in src - while (nextIterFM( src->outerMap, (Word*)&soKey, (Word*)&soVal )) { - - // is filefn in dst? - if (! lookupFM( dst->outerMap, (Word*)&doVal, (Word)soKey )) { - - // no .. add dopy(filefn) -> dopy(innerMap) to src - FileFn* c_soKey = dopy_FileFn(soKey); - WordFM* c_soVal = dopy_InnerMap(soVal); - if ((!c_soKey) || (!c_soVal)) goto oom; - addToFM( dst->outerMap, (Word)c_soKey, (Word)c_soVal ); - - } else { - - // yes .. merge the two innermaps - initIterFM( soVal ); - - // for (lno, counts) in soVal (source inner map) - while (nextIterFM( soVal, (Word*)&siKey, (Word*)&siVal )) { - - // is lno in the corresponding dst inner map? - if (! lookupFM( doVal, (Word*)&diVal, siKey )) { - - // no .. add lineno->dopy(counts) to dst inner map - Counts* c_siVal = dopy_Counts( siVal ); - if (!c_siVal) goto oom; - addToFM( doVal, siKey, (Word)c_siVal ); - - } else { - - // yes .. merge counts into dst inner map val - addCounts( s, diVal, siVal ); - - } - } - - } - - } - - // add the summaries too - addCounts(s, dst->summary, src->summary ); - - return; - - oom: - mallocFail(s, "merge_CacheProfInfo"); -} - -static void usage ( void ) -{ - fprintf(stderr, "%s: Merges multiple cachegrind output files into one\n", - argv0); - fprintf(stderr, "%s: usage: %s [-o outfile] [files-to-merge]\n", - argv0, argv0); - exit(1); -} - -int main ( int argc, char** argv ) -{ - Int i; - SOURCE src; - CacheProfFile *cpf, *cpfTmp; - - FILE* outfile = NULL; - char* outfilename = NULL; - Int outfileix = 0; - - if (argv[0]) - argv0 = argv[0]; - - if (argc < 2) - usage(); - - for (i = 1; i < argc; i++) { - if (streq(argv[i], "-h") || streq(argv[i], "--help")) - usage(); - } - - /* Scan args, looking for '-o outfilename'. */ - for (i = 1; i < argc; i++) { - if (streq(argv[i], "-o")) { - if (i+1 < argc) { - outfilename = argv[i+1]; - outfileix = i; - break; - } else { - usage(); - } - } - } - - cpf = NULL; - - for (i = 1; i < argc; i++) { - - if (i == outfileix) { - /* Skip '-o' and whatever follows it */ - i += 1; - continue; - } - - fprintf(stderr, "%s: parsing %s\n", argv0, argv[i]); - src.lno = 1; - src.filename = argv[i]; - src.fp = fopen(src.filename, "r"); - if (!src.fp) { - perror(argv0); - barf(&src, "Cannot open input file"); - } - assert(src.fp); - cpfTmp = parse_CacheProfFile( &src ); - fclose(src.fp); - - /* If this isn't the first file, merge */ - if (cpf == NULL) { - /* this is the first file */ - cpf = cpfTmp; - } else { - /* not the first file; merge */ - fprintf(stderr, "%s: merging %s\n", argv0, argv[i]); - merge_CacheProfInfo( &src, cpf, cpfTmp ); - ddel_CacheProfFile( cpfTmp ); - } - - } - - /* Now create the output file. */ - - if (cpf) { - - fprintf(stderr, "%s: writing %s\n", - argv0, outfilename ? outfilename : "(stdout)" ); - - /* Write the output. */ - if (outfilename) { - outfile = fopen(outfilename, "w"); - if (!outfile) { - fprintf(stderr, "%s: can't create output file %s\n", - argv0, outfilename); - perror(argv0); - exit(1); - } - } else { - outfile = stdout; - } - - show_CacheProfFile( outfile, cpf ); - if (ferror(outfile)) { - fprintf(stderr, "%s: error writing output file %s\n", - argv0, outfilename ? outfilename : "(stdout)" ); - perror(argv0); - if (outfile != stdout) - fclose(outfile); - exit(1); - } - - fflush(outfile); - if (outfile != stdout) - fclose( outfile ); - - ddel_CacheProfFile( cpf ); - } - - return 0; -} - - -//------------------------------------------------------------------// -//--- WordFM ---// -//--- Implementation ---// -//------------------------------------------------------------------// - -/* ------------ Implementation ------------ */ - -/* One element of the AVL tree */ -typedef - struct _AvlNode { - Word key; - Word val; - struct _AvlNode* left; - struct _AvlNode* right; - Char balance; - } - AvlNode; - -typedef - struct { - Word w; - Bool b; - } - MaybeWord; - -#define WFM_STKMAX 32 // At most 2**32 entries can be iterated over - -struct _WordFM { - AvlNode* root; - void* (*alloc_nofail)( SizeT ); - void (*dealloc)(void*); - Word (*kCmp)(Word,Word); - AvlNode* nodeStack[WFM_STKMAX]; // Iterator node stack - Int numStack[WFM_STKMAX]; // Iterator num stack - Int stackTop; // Iterator stack pointer, one past end -}; - -/* forward */ -static Bool avl_removeroot_wrk(AvlNode** t, Word(*kCmp)(Word,Word)); - -/* Swing to the left. Warning: no balance maintenance. */ -static void avl_swl ( AvlNode** root ) -{ - AvlNode* a = *root; - AvlNode* b = a->right; - *root = b; - a->right = b->left; - b->left = a; -} - -/* Swing to the right. Warning: no balance maintenance. */ -static void avl_swr ( AvlNode** root ) -{ - AvlNode* a = *root; - AvlNode* b = a->left; - *root = b; - a->left = b->right; - b->right = a; -} - -/* Balance maintenance after especially nasty swings. */ -static void avl_nasty ( AvlNode* root ) -{ - switch (root->balance) { - case -1: - root->left->balance = 0; - root->right->balance = 1; - break; - case 1: - root->left->balance = -1; - root->right->balance = 0; - break; - case 0: - root->left->balance = 0; - root->right->balance = 0; - break; - default: - assert(0); - } - root->balance=0; -} - -/* Find size of a non-NULL tree. */ -static Word size_avl_nonNull ( AvlNode* nd ) -{ - return 1 + (nd->left ? size_avl_nonNull(nd->left) : 0) - + (nd->right ? size_avl_nonNull(nd->right) : 0); -} - -/* Insert element a into the AVL tree t. Returns True if the depth of - the tree has grown. If element with that key is already present, - just copy a->val to existing node, first returning old ->val field - of existing node in *oldV, so that the caller can finalize it - however it wants. -*/ -static -Bool avl_insert_wrk ( AvlNode** rootp, - /*OUT*/MaybeWord* oldV, - AvlNode* a, - Word (*kCmp)(Word,Word) ) -{ - Word cmpres; - - /* initialize */ - a->left = 0; - a->right = 0; - a->balance = 0; - oldV->b = False; - - /* insert into an empty tree? */ - if (!(*rootp)) { - (*rootp) = a; - return True; - } - - cmpres = kCmp( (*rootp)->key, a->key ); - - if (cmpres > 0) { - /* insert into the left subtree */ - if ((*rootp)->left) { - AvlNode* left_subtree = (*rootp)->left; - if (avl_insert_wrk(&left_subtree, oldV, a, kCmp)) { - switch ((*rootp)->balance--) { - case 1: return False; - case 0: return True; - case -1: break; - default: assert(0); - } - if ((*rootp)->left->balance < 0) { - avl_swr( rootp ); - (*rootp)->balance = 0; - (*rootp)->right->balance = 0; - } else { - avl_swl( &((*rootp)->left) ); - avl_swr( rootp ); - avl_nasty( *rootp ); - } - } else { - (*rootp)->left = left_subtree; - } - return False; - } else { - (*rootp)->left = a; - if ((*rootp)->balance--) - return False; - return True; - } - assert(0);/*NOTREACHED*/ - } - else - if (cmpres < 0) { - /* insert into the right subtree */ - if ((*rootp)->right) { - AvlNode* right_subtree = (*rootp)->right; - if (avl_insert_wrk(&right_subtree, oldV, a, kCmp)) { - switch((*rootp)->balance++){ - case -1: return False; - case 0: return True; - case 1: break; - default: assert(0); - } - if ((*rootp)->right->balance > 0) { - avl_swl( rootp ); - (*rootp)->balance = 0; - (*rootp)->left->balance = 0; - } else { - avl_swr( &((*rootp)->right) ); - avl_swl( rootp ); - avl_nasty( *rootp ); - } - } else { - (*rootp)->right = right_subtree; - } - return False; - } else { - (*rootp)->right = a; - if ((*rootp)->balance++) - return False; - return True; - } - assert(0);/*NOTREACHED*/ - } - else { - /* cmpres == 0, a duplicate - replace the val, but don't - incorporate the node in the tree */ - oldV->b = True; - oldV->w = (*rootp)->val; - (*rootp)->val = a->val; - return False; - } -} - -/* Remove an element a from the AVL tree t. a must be part of - the tree. Returns True if the depth of the tree has shrunk. -*/ -static -Bool avl_remove_wrk ( AvlNode** rootp, - AvlNode* a, - Word(*kCmp)(Word,Word) ) -{ - Bool ch; - Word cmpres = kCmp( (*rootp)->key, a->key ); - - if (cmpres > 0){ - /* remove from the left subtree */ - AvlNode* left_subtree = (*rootp)->left; - assert(left_subtree); - ch = avl_remove_wrk(&left_subtree, a, kCmp); - (*rootp)->left=left_subtree; - if (ch) { - switch ((*rootp)->balance++) { - case -1: return True; - case 0: return False; - case 1: break; - default: assert(0); - } - switch ((*rootp)->right->balance) { - case 0: - avl_swl( rootp ); - (*rootp)->balance = -1; - (*rootp)->left->balance = 1; - return False; - case 1: - avl_swl( rootp ); - (*rootp)->balance = 0; - (*rootp)->left->balance = 0; - return -1; - case -1: - break; - default: - assert(0); - } - avl_swr( &((*rootp)->right) ); - avl_swl( rootp ); - avl_nasty( *rootp ); - return True; - } - } - else - if (cmpres < 0) { - /* remove from the right subtree */ - AvlNode* right_subtree = (*rootp)->right; - assert(right_subtree); - ch = avl_remove_wrk(&right_subtree, a, kCmp); - (*rootp)->right = right_subtree; - if (ch) { - switch ((*rootp)->balance--) { - case 1: return True; - case 0: return False; - case -1: break; - default: assert(0); - } - switch ((*rootp)->left->balance) { - case 0: - avl_swr( rootp ); - (*rootp)->balance = 1; - (*rootp)->right->balance = -1; - return False; - case -1: - avl_swr( rootp ); - (*rootp)->balance = 0; - (*rootp)->right->balance = 0; - return True; - case 1: - break; - default: - assert(0); - } - avl_swl( &((*rootp)->left) ); - avl_swr( rootp ); - avl_nasty( *rootp ); - return True; - } - } - else { - assert(cmpres == 0); - assert((*rootp)==a); - return avl_removeroot_wrk(rootp, kCmp); - } - return 0; -} - -/* Remove the root of the AVL tree *rootp. - * Warning: dumps core if *rootp is empty - */ -static -Bool avl_removeroot_wrk ( AvlNode** rootp, - Word(*kCmp)(Word,Word) ) -{ - Bool ch; - AvlNode* a; - if (!(*rootp)->left) { - if (!(*rootp)->right) { - (*rootp) = 0; - return True; - } - (*rootp) = (*rootp)->right; - return True; - } - if (!(*rootp)->right) { - (*rootp) = (*rootp)->left; - return True; - } - if ((*rootp)->balance < 0) { - /* remove from the left subtree */ - a = (*rootp)->left; - while (a->right) a = a->right; - } else { - /* remove from the right subtree */ - a = (*rootp)->right; - while (a->left) a = a->left; - } - ch = avl_remove_wrk(rootp, a, kCmp); - a->left = (*rootp)->left; - a->right = (*rootp)->right; - a->balance = (*rootp)->balance; - (*rootp) = a; - if(a->balance == 0) return ch; - return False; -} - -static -AvlNode* avl_find_node ( AvlNode* t, Word k, Word(*kCmp)(Word,Word) ) -{ - Word cmpres; - while (True) { - if (t == NULL) return NULL; - cmpres = kCmp(t->key, k); - if (cmpres > 0) t = t->left; else - if (cmpres < 0) t = t->right; else - return t; - } -} - -// Clear the iterator stack. -static void stackClear(WordFM* fm) -{ - Int i; - assert(fm); - for (i = 0; i < WFM_STKMAX; i++) { - fm->nodeStack[i] = NULL; - fm->numStack[i] = 0; - } - fm->stackTop = 0; -} - -// Push onto the iterator stack. -static inline void stackPush(WordFM* fm, AvlNode* n, Int i) -{ - assert(fm->stackTop < WFM_STKMAX); - assert(1 <= i && i <= 3); - fm->nodeStack[fm->stackTop] = n; - fm-> numStack[fm->stackTop] = i; - fm->stackTop++; -} - -// Pop from the iterator stack. -static inline Bool stackPop(WordFM* fm, AvlNode** n, Int* i) -{ - assert(fm->stackTop <= WFM_STKMAX); - - if (fm->stackTop > 0) { - fm->stackTop--; - *n = fm->nodeStack[fm->stackTop]; - *i = fm-> numStack[fm->stackTop]; - assert(1 <= *i && *i <= 3); - fm->nodeStack[fm->stackTop] = NULL; - fm-> numStack[fm->stackTop] = 0; - return True; - } else { - return False; - } -} - -static -AvlNode* avl_dopy ( AvlNode* nd, - Word(*dopyK)(Word), - Word(*dopyV)(Word), - void*(alloc_nofail)(SizeT) ) -{ - AvlNode* nyu; - if (! nd) - return NULL; - nyu = alloc_nofail(sizeof(AvlNode)); - assert(nyu); - - nyu->left = nd->left; - nyu->right = nd->right; - nyu->balance = nd->balance; - - /* Copy key */ - if (dopyK) { - nyu->key = dopyK( nd->key ); - if (nd->key != 0 && nyu->key == 0) - return NULL; /* oom in key dcopy */ - } else { - /* copying assumedly unboxed keys */ - nyu->key = nd->key; - } - - /* Copy val */ - if (dopyV) { - nyu->val = dopyV( nd->val ); - if (nd->val != 0 && nyu->val == 0) - return NULL; /* oom in val dcopy */ - } else { - /* copying assumedly unboxed vals */ - nyu->val = nd->val; - } - - /* Copy subtrees */ - if (nyu->left) { - nyu->left = avl_dopy( nyu->left, dopyK, dopyV, alloc_nofail ); - if (! nyu->left) - return NULL; - } - if (nyu->right) { - nyu->right = avl_dopy( nyu->right, dopyK, dopyV, alloc_nofail ); - if (! nyu->right) - return NULL; - } - - return nyu; -} - -/* --- Public interface functions --- */ - -/* Initialise a WordFM. */ -void initFM ( WordFM* fm, - void* (*alloc_nofail)( SizeT ), - void (*dealloc)(void*), - Word (*kCmp)(Word,Word) ) -{ - fm->root = 0; - fm->kCmp = kCmp; - fm->alloc_nofail = alloc_nofail; - fm->dealloc = dealloc; - fm->stackTop = 0; -} - -/* Allocate and Initialise a WordFM. */ -WordFM* newFM( void* (*alloc_nofail)( SizeT ), - void (*dealloc)(void*), - Word (*kCmp)(Word,Word) ) -{ - WordFM* fm = alloc_nofail(sizeof(WordFM)); - assert(fm); - initFM(fm, alloc_nofail, dealloc, kCmp); - return fm; -} - -static void avl_free ( AvlNode* nd, - void(*kFin)(Word), - void(*vFin)(Word), - void(*dealloc)(void*) ) -{ - if (!nd) - return; - if (nd->left) - avl_free(nd->left, kFin, vFin, dealloc); - if (nd->right) - avl_free(nd->right, kFin, vFin, dealloc); - if (kFin) - kFin( nd->key ); - if (vFin) - vFin( nd->val ); - memset(nd, 0, sizeof(AvlNode)); - dealloc(nd); -} - -/* Free up the FM. If kFin is non-NULL, it is applied to keys - before the FM is deleted; ditto with vFin for vals. */ -void deleteFM ( WordFM* fm, void(*kFin)(Word), void(*vFin)(Word) ) -{ - void(*dealloc)(void*) = fm->dealloc; - avl_free( fm->root, kFin, vFin, dealloc ); - memset(fm, 0, sizeof(WordFM) ); - dealloc(fm); -} - -/* Add (k,v) to fm. */ -void addToFM ( WordFM* fm, Word k, Word v ) -{ - MaybeWord oldV; - AvlNode* node; - node = fm->alloc_nofail( sizeof(struct _AvlNode) ); - node->key = k; - node->val = v; - oldV.b = False; - oldV.w = 0; - avl_insert_wrk( &fm->root, &oldV, node, fm->kCmp ); - //if (oldV.b && fm->vFin) - // fm->vFin( oldV.w ); - if (oldV.b) - free(node); -} - -// Delete key from fm, returning associated val if found -Bool delFromFM ( WordFM* fm, /*OUT*/Word* oldV, Word key ) -{ - AvlNode* node = avl_find_node( fm->root, key, fm->kCmp ); - if (node) { - avl_remove_wrk( &fm->root, node, fm->kCmp ); - if (oldV) - *oldV = node->val; - fm->dealloc(node); - return True; - } else { - return False; - } -} - -// Look up in fm, assigning found val at spec'd address -Bool lookupFM ( WordFM* fm, /*OUT*/Word* valP, Word key ) -{ - AvlNode* node = avl_find_node( fm->root, key, fm->kCmp ); - if (node) { - if (valP) - *valP = node->val; - return True; - } else { - return False; - } -} - -Word sizeFM ( WordFM* fm ) -{ - // Hmm, this is a bad way to do this - return fm->root ? size_avl_nonNull( fm->root ) : 0; -} - -// set up FM for iteration -void initIterFM ( WordFM* fm ) -{ - assert(fm); - stackClear(fm); - if (fm->root) - stackPush(fm, fm->root, 1); -} - -// get next key/val pair. Will assert if fm has been modified -// or looked up in since initIterFM was called. -Bool nextIterFM ( WordFM* fm, /*OUT*/Word* pKey, /*OUT*/Word* pVal ) -{ - Int i = 0; - AvlNode* n = NULL; - - assert(fm); - - // This in-order traversal requires each node to be pushed and popped - // three times. These could be avoided by updating nodes in-situ on the - // top of the stack, but the push/pop cost is so small that it's worth - // keeping this loop in this simpler form. - while (stackPop(fm, &n, &i)) { - switch (i) { - case 1: - stackPush(fm, n, 2); - if (n->left) stackPush(fm, n->left, 1); - break; - case 2: - stackPush(fm, n, 3); - if (pKey) *pKey = n->key; - if (pVal) *pVal = n->val; - return True; - case 3: - if (n->right) stackPush(fm, n->right, 1); - break; - default: - assert(0); - } - } - - // Stack empty, iterator is exhausted, return NULL - return False; -} - -// clear the I'm iterating flag -void doneIterFM ( WordFM* fm ) -{ -} - -WordFM* dopyFM ( WordFM* fm, Word(*dopyK)(Word), Word(*dopyV)(Word) ) -{ - WordFM* nyu; - - /* can't clone the fm whilst iterating on it */ - assert(fm->stackTop == 0); - - nyu = fm->alloc_nofail( sizeof(WordFM) ); - assert(nyu); - - *nyu = *fm; - - fm->stackTop = 0; - memset(fm->nodeStack, 0, sizeof(fm->nodeStack)); - memset(fm->numStack, 0, sizeof(fm->numStack)); - - if (nyu->root) { - nyu->root = avl_dopy( nyu->root, dopyK, dopyV, fm->alloc_nofail ); - if (! nyu->root) - return NULL; - } - - return nyu; -} - -//------------------------------------------------------------------// -//--- end WordFM ---// -//--- Implementation ---// -//------------------------------------------------------------------// - -/*--------------------------------------------------------------------*/ -/*--- end cg_merge.c ---*/ -/*--------------------------------------------------------------------*/ diff --git a/cachegrind/cg_merge.in b/cachegrind/cg_merge.in new file mode 100755 index 0000000000..fca73439eb --- /dev/null +++ b/cachegrind/cg_merge.in @@ -0,0 +1,339 @@ +#! /usr/bin/env python3 +# pyright: strict + +# -------------------------------------------------------------------- +# --- Cachegrind's merger. cg_merge.in --- +# -------------------------------------------------------------------- + +# This file is part of Cachegrind, a Valgrind tool for cache +# profiling programs. +# +# Copyright (C) 2002-2023 Nicholas Nethercote +# nj...@va... +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, see <http://www.gnu.org/licenses/>. +# +# The GNU General Public License is contained in the file COPYING. + +""" +This script diffs Cachegrind output files. +""" + +# Use `make pymerge` to "build" this script every time it is changed. This runs +# the formatters, type-checkers, and linters on `cg_merge.in` and then +# generates `cg_merge`. +# +# This is a cut-down version of `cg_annotate.in`. + +from __future__ import annotations + +import re +import sys +from argparse import ArgumentParser, Namespace +from collections import defaultdict +from typing import DefaultDict, NoReturn, TextIO + + +class Args(Namespace): + """ + A typed wrapper for parsed args. + + None of these fields are modified after arg parsing finishes. + """ + + output: str + cgout_filename: list[str] + + @staticmethod + def parse() -> Args: + p = ArgumentParser(description="Merge multiple Cachegrind output files.") + + p.add_argument("--version", action="version", version="%(prog)s-@VERSION@") + + p.add_argument( + "-o", + dest="output", + type=str, + metavar="FILE", + help="output file (default: stdout)", + ) + + p.add_argument( + "cgout_filename", + nargs="+", + metavar="cachegrind-out-file", + help="file produced by Cachegrind", + ) + + return p.parse_args(namespace=Args()) + + +# Args are stored in a global for easy access. +args = Args.parse() + +# A single instance of this class is constructed, from `args` and the `events:` +# line in the cgout file. +class Events: + # The event names. + events: list[str] + + def __init__(self, text: str) -> None: + self.events = text.split() + self.num_events = len(self.events) + + def mk_cc(self, text: str) -> Cc: + """Raises a `ValueError` exception on syntax error.""" + # This is slightly faster than a list comprehension. + counts = list(map(int, text.split())) + + if len(counts) == self.num_events: + pass + elif len(counts) < self.num_events: + # Add zeroes at the end for any missing numbers. + counts.extend([0] * (self.num_events - len(counts))) + else: + raise ValueError + + return Cc(counts) + + def mk_empty_cc(self) -> Cc: + # This is much faster than a list comprehension. + return Cc([0] * self.num_events) + + +class Cc: + """ + This is a dumb container for counts. + + It doesn't know anything about events, i.e. what each count means. It can + do basic operations like `__iadd__` and `__eq__`, and anything more must be + done elsewhere. `Events.mk_cc` and `Events.mk_empty_cc` are used for + construction. + """ + + # Always the same length as `Events.events`. + counts: list[int] + + def __init__(self, counts: list[int]) -> None: + self.counts = counts + + def __repr__(self) -> str: + return str(self.counts) + + def __eq__(self, other: object) -> bool: + if not isinstance(other, Cc): + return NotImplemented + return self.counts == other.counts + + def __iadd__(self, other: Cc) -> Cc: + for i, other_count in enumerate(other.counts): + self.counts[i] += other_count + return self + + +# Per-line CCs, organised by filename, function name, and line number. +DictLineCc = DefaultDict[int, Cc] +DictFnDictLineCc = DefaultDict[str, DictLineCc] +DictFlDictFnDictLineCc = DefaultDict[str, DictFnDictLineCc] + + +def die(msg: str) -> NoReturn: + print("cg_merge: error:", msg, file=sys.stderr) + sys.exit(1) + + +def read_cgout_file( + cgout_filename: str, + is_first_file: bool, + cumul_dict_fl_dict_fn_dict_line_cc: DictFlDictFnDictLineCc, + cumul_summary_cc: Cc, +) -> tuple[list[str], str, Events]: + # The file format is described in Cachegrind's manual. + try: + cgout_file = open(cgout_filename, "r", encoding="utf-8") + except OSError as err: + die(f"{err}") + + with cgout_file: + cgout_line_num = 0 + + def parse_die(msg: str) -> NoReturn: + die(f"{cgout_file.name}:{cgout_line_num}: {msg}") + + def readline() -> str: + nonlocal cgout_line_num + cgout_line_num += 1 + return cgout_file.readline() + + # Read "desc:" lines. + desc: list[str] = [] + while line := readline(): + if m := re.match(r"desc:\s+(.*)", line): + desc.append(m.group(1)) + else: + break + + # Read "cmd:" line. (`line` is already set from the "desc:" loop.) + if m := re.match(r"cmd:\s+(.*)", line): + cmd = m.group(1) + else: + parse_die("missing a `command:` line") + + # Read "events:" line. + line = readline() + if m := re.match(r"events:\s+(.*)", line): + events = Events(m.group(1)) + else: + parse_die("missing an `events:` line") + + def mk_empty_dict_line_cc() -> DictLineCc: + return defaultdict(events.mk_empty_cc) + + def mk_empty_dict_fn_dict_line_cc() -> DictFnDictLineCc: + return defaultdict(mk_empty_dict_line_cc) + + summary_cc_present = False + + curr_fl = "" + curr_fn = "" + + # The `cumul_*` values are passed in by reference and are modified by + # this function. But they can't be properly initialized until the + # `events:` line of the first file is read and the number of events is + # known. So we initialize them in an invalid state, and then + # reinitialize them properly here, before their first use. + if is_first_file: + cumul_dict_fl_dict_fn_dict_line_cc.default_factory = ( + mk_empty_dict_fn_dict_line_cc + ) + cumul_summary_cc.counts = events.mk_empty_cc().counts + + # Compile the one hot regex. + count_pat = re.compile(r"(\d+)\s+(.*)") + + # Line matching is done in order of pattern frequency, for speed. + while True: + line = readline() + + if m := count_pat.match(line): + line_num = int(m.group(1)) + try: + cc = events.mk_cc(m.group(2)) + except ValueError: + parse_die("malformed or too many event counts") + + # Record this CC at the file/func/line level. + line_cc = cumul_dict_fl_dict_fn_dict_line_cc[curr_fl][curr_fn][line_num] + line_cc += cc + + elif line.startswith("fn="): + curr_fn = line[3:-1] + + elif line.startswith("fl="): + curr_fl = line[3:-1] + # A `fn=` line should follow, overwriting the "???". + curr_fn = "???" + + elif m := re.match(r"summary:\s+(.*)", line): + summary_cc_present = True + try: + cumul_summary_cc += events.mk_cc(m.group(1)) + except ValueError: + parse_die("too many event counts") + + elif line == "": + break # EOF + + elif line == "\n" or line.startswith("#"): + # Skip empty lines and comment lines. + pass + + else: + parse_die(f"malformed line: {line[:-1]}") + + # Check if summary line was present. + if not summary_cc_present: + parse_die("missing `summary:` line, aborting") + + # In `cg_annotate.in` and `cg_diff.in` we check that the file's summary CC + # matches the totals of the file's individual CCs, but not here. That's + # because in this script we don't collect the file's CCs in isolation, + # instead we just add them to the accumulated CCs, for speed. This makes it + # difficult to do the per-file checking. + + return (desc, cmd, events) + + +def main() -> None: + desc1: list[str] | None = None + cmd1 = None + events1 = None + + # Different places where we accumulate CC data. Initialized to invalid + # states prior to the number of events being known. + cumul_dict_fl_dict_fn_dict_line_cc: DictFlDictFnDictLineCc = defaultdict(None) + cumul_summary_cc: Cc = Cc([]) + + for n, filename in enumerate(args.cgout_filename): + is_first_file = n == 0 + (desc_n, cmd_n, events_n) = read_cgout_file( + filename, + is_first_file, + cumul_dict_fl_dict_fn_dict_line_cc, + cumul_summary_cc, + ) + # We reuse the description and command from the first file, like the + # the old C version of `cg_merge`. + if is_first_file: + desc1 = desc_n + cmd1 = cmd_n + events1 = events_n + else: + assert events1 + if events1.num_events != events_n.num_events: + die("events don't match") + + def write_output(f: TextIO) -> None: + # These assertions hold because the loop above executes at least twice. + assert desc1 + assert events1 + assert cumul_dict_fl_dict_fn_dict_line_cc is not None + assert cumul_summary_cc + + for desc_line in desc1: + print("desc:", desc_line, file=f) + print("cmd:", cmd1, file=f) + print("events:", *events1.events, sep=" ", file=f) + + for fl, dict_fn_dict_line_cc in cumul_dict_fl_dict_fn_dict_line_cc.items(): + print(f"fl={fl}", file=f) + for fn, dict_line_cc in dict_fn_dict_line_cc.items(): + print(f"fn={fn}", file=f) + for line, cc in dict_line_cc.items(): + print(line, *cc.counts, file=f) + + print("summary:", *cumul_summary_cc.counts, sep=" ", file=f) + + if args.output: + try: + with open(args.output, "w", encoding="utf-8") as f: + write_output(f) + except OSError as err: + die(f"{err}") + else: + write_output(sys.stdout) + + +if __name__ == "__main__": + main() diff --git a/cachegrind/tests/Makefile.am b/cachegrind/tests/Makefile.am index 16ac524b35..3aa85c9990 100644 --- a/cachegrind/tests/Makefile.am +++ b/cachegrind/tests/Makefile.am @@ -15,6 +15,8 @@ dist_noinst_SCRIPTS = filter_stderr filter_cachesim_discards EXTRA_DIST = \ ann-diff1.post.exp ann-diff1.stderr.exp ann-diff1.vgtest \ ann-diff2a.cgout ann-diff2b.cgout \ + ann-merge1.post.exp ann-merge1.stderr.exp ann-merge1.vgtest + ann-merge1a.cgout ann-merge1b.cgout \ ann1a.post.exp ann1a.stderr.exp ann1a.vgtest ann1.cgout \ ann1b.post.exp ann1b.stderr.exp ann1b.vgtest ann1b.cgout \ ann2.post.exp ann2.stderr.exp ann2.vgtest ann2.cgout \ diff --git a/cachegrind/tests/ann-merge-x.rs b/cachegrind/tests/ann-merge-x.rs new file mode 100644 index 0000000000..b2f931a673 --- /dev/null +++ b/cachegrind/tests/ann-merge-x.rs @@ -0,0 +1,5 @@ +one +two +three +four +five diff --git a/cachegrind/tests/ann-merge-y.rs b/cachegrind/tests/ann-merge-y.rs new file mode 100644 index 0000000000..b566061598 --- /dev/null +++ b/cachegrind/tests/ann-merge-y.rs @@ -0,0 +1,6 @@ +one +two +three +four +five +six diff --git a/cachegrind/tests/ann-merge1.post.exp b/cachegrind/tests/ann-merge1.post.exp new file mode 100644 index 0000000000..6e9a5f1a37 --- /dev/null +++ b/cachegrind/tests/ann-merge1.post.exp @@ -0,0 +1,66 @@ +-------------------------------------------------------------------------------- +-- Cachegrind profile +-------------------------------------------------------------------------------- +Description 1a +Description 1b +Command: Command 1 +Data file: ann-merge1c.cgout +Events recorded: A B C +Events shown: A B C +Event sort order: A B C +Threshold: 0.1 +Include dirs: +User annotated: +Auto-annotation: on + +-------------------------------------------------------------------------------- +-- Summary +-------------------------------------------------------------------------------- +A B C + +86 (100.0%) 113 (100.0%) 145 (100.0%) PROGRAM TOTALS + +-------------------------------------------------------------------------------- +-- Function summary +-------------------------------------------------------------------------------- +A B C file:function + +40 (46.5%) 80 (70.8%) 120 (82.8%) ann-merge-x.rs:x1 +20 (23.3%) 10 (8.8%) 5 (3.4%) ann-merge-x.rs:x3 +16 (18.6%) 18 (15.9%) 20 (13.8%) ann-m... [truncated message content] |
|
From: Nicholas N. <nj...@so...> - 2023-03-28 04:27:09
|
https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=29d1f00e205cf8362374c7e73a3e2f621faf73f3 commit 29d1f00e205cf8362374c7e73a3e2f621faf73f3 Author: Nicholas Nethercote <n.n...@gm...> Date: Mon Mar 27 10:47:27 2023 +1100 Rename `ann-diff` test as `ann-diff1`. To make room for another test. Diff: --- cachegrind/tests/Makefile.am | 2 +- cachegrind/tests/{ann-diff.post.exp => ann-diff1.post.exp} | 2 +- cachegrind/tests/{ann-diff.stderr.exp => ann-diff1.stderr.exp} | 0 cachegrind/tests/{ann-diff.vgtest => ann-diff1.vgtest} | 4 ++-- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/cachegrind/tests/Makefile.am b/cachegrind/tests/Makefile.am index cee1f29167..33baeeea69 100644 --- a/cachegrind/tests/Makefile.am +++ b/cachegrind/tests/Makefile.am @@ -13,7 +13,7 @@ dist_noinst_SCRIPTS = filter_stderr filter_cachesim_discards # Note that `test.c` and `a.c` are not compiled. # They just serve as input for cg_annotate in `ann1a` and `ann1b`. EXTRA_DIST = \ - ann-diff.post.exp ann-diff.stderr.exp ann-diff.vgtest \ + ann-diff1.post.exp ann-diff1.stderr.exp ann-diff1.vgtest \ ann1a.post.exp ann1a.stderr.exp ann1a.vgtest ann1.cgout \ ann1b.post.exp ann1b.stderr.exp ann1b.vgtest ann1b.cgout \ ann2.post.exp ann2.stderr.exp ann2.vgtest ann2.cgout \ diff --git a/cachegrind/tests/ann-diff.post.exp b/cachegrind/tests/ann-diff1.post.exp similarity index 98% rename from cachegrind/tests/ann-diff.post.exp rename to cachegrind/tests/ann-diff1.post.exp index 9f8f276ff1..2d7d61ac70 100644 --- a/cachegrind/tests/ann-diff.post.exp +++ b/cachegrind/tests/ann-diff1.post.exp @@ -3,7 +3,7 @@ -------------------------------------------------------------------------------- Files compared: ann1.cgout; ann1b.cgout Command: ./a.out; ./a.out -Data file: ann-diff.cgout +Data file: ann-diff1.cgout Events recorded: Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw Events shown: Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw Event sort order: Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw diff --git a/cachegrind/tests/ann-diff.stderr.exp b/cachegrind/tests/ann-diff1.stderr.exp similarity index 100% rename from cachegrind/tests/ann-diff.stderr.exp rename to cachegrind/tests/ann-diff1.stderr.exp diff --git a/cachegrind/tests/ann-diff.vgtest b/cachegrind/tests/ann-diff1.vgtest similarity index 74% rename from cachegrind/tests/ann-diff.vgtest rename to cachegrind/tests/ann-diff1.vgtest index e3dc0992e0..ce3e216c66 100644 --- a/cachegrind/tests/ann-diff.vgtest +++ b/cachegrind/tests/ann-diff1.vgtest @@ -2,5 +2,5 @@ # the post-processing of the `ann{1,1b}.cgout` test files. prog: ../../tests/true vgopts: --cachegrind-out-file=cachegrind.out -post: perl ../../cachegrind/cg_diff ann1.cgout ann1b.cgout > ann-diff.cgout && perl ../../cachegrind/cg_annotate ann-diff.cgout -cleanup: rm ann-diff.cgout +post: perl ../../cachegrind/cg_diff ann1.cgout ann1b.cgout > ann-diff1.cgout && perl ../../cachegrind/cg_annotate ann-diff1.cgout +cleanup: rm ann-diff1.cgout |
|
From: Nicholas N. <nj...@so...> - 2023-03-28 04:27:08
|
https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=8a75eecbad1da39fc29fa112bae4d9d547404656 commit 8a75eecbad1da39fc29fa112bae4d9d547404656 Author: Nicholas Nethercote <n.n...@gm...> Date: Fri Mar 24 23:50:49 2023 +1100 Rewrite `cg_diff` in Python. For all the same reasons I rewrote `cg_annotate` in Python. The commit also moves the Python "build" steps into `auxprogs/pybuild.sh`, for easy sharing. Finally, it very slightly tweaks the whitespace in the output of `cg_annotate`. Diff: --- auxprogs/pybuild.sh | 88 +++ cachegrind/Makefile.am | 34 +- cachegrind/cg_annotate.in | 67 +-- cachegrind/cg_diff.in | 657 +++++++++++----------- cachegrind/tests/Makefile.am | 1 + cachegrind/tests/ann-diff1.post.exp | 3 +- cachegrind/tests/ann-diff1.vgtest | 2 +- cachegrind/tests/ann-diff2-aux/ann-diff2-basic.rs | 10 + cachegrind/tests/ann-diff2.post.exp | 46 ++ cachegrind/tests/ann-diff2.stderr.exp | 17 + cachegrind/tests/ann-diff2.vgtest | 6 + cachegrind/tests/ann-diff2a.cgout | 9 + cachegrind/tests/ann-diff2b.cgout | 15 + 13 files changed, 561 insertions(+), 394 deletions(-) diff --git a/auxprogs/pybuild.sh b/auxprogs/pybuild.sh new file mode 100755 index 0000000000..432a768c51 --- /dev/null +++ b/auxprogs/pybuild.sh @@ -0,0 +1,88 @@ +#! /bin/sh + +# "Build" a given Python file `foo`: format it, type-check it, lint it, and +# generate the final file from the `foo.in` file. +# +# The following Python tools are used by this script. +# +# - Formatters: +# - `black`, for general formatting. This avoids the need for style checkers +# like `flake8`. Note that `black` allows a max line length of 88, which is +# a mild but common PEP-8 violation. +# - `isort`, for import sorting. +# +# - Type-checkers: +# - `mypy`. This is the most commonly used Python type checker. +# - `pyright`. This is another good type checker. +# - Sometimes they give different result. Both should be kept happy. +# +# - Linters: +# - `ruff`. Sometimes useful, and very fast to run. +# - `pylint`. Sometimes annoying, sometimes useful. The `pylintrc` +# modifies/disables the more annoying lints. +# - Sometimes they give different result. Both should be kept happy. +# +# The following tools are relevant, but not run by this script. +# +# - Profilers: +# - `cProfile` + `snakeviz`: Typically run with +# `python3 -m cProfile -o cg.prof cg_annotate $INPUT && snakeviz cg.prof`. +# - `scalene`. Typically run with `scalene ./cg_annotate $INPUT`. +# +# - Packager: +# - `cp` is used for distribution. This is possible because this program is a +# single file and only uses the Python Standard Library. This avoids the +# needs for any of the million different Python package management tools. +# +# All of the above tools can be installed with `pip3 install $NAME`, except +# `cProfile` which is built into Python. + +set -e + +# Currently targetting Python 3.9 (released in October 2020) and up. The tools +# use two different syntaxes for specifying the version number. +ver=3.9 +pyver=py39 + +infile=$1 +outfile=$2 +if [ -z "$outfile" ] ; then + exit 1 +fi + +echo "== black ==" +black $infile +echo + +echo "== isort ==" +isort $infile +echo + +echo "== mypy ==" +mypy --strict $infile --python-version $ver +echo + +# Strict mode for pyright is enabled by a `pyright: strict` comment inside each +# Python file. +# +# Note: `pyright` refuses to check any file without a `.py` extension, hence +# the copying to a temp file with a `.py` extension. +echo "== pyright ==" +tmpfile=`mktemp --tmpdir $infile.XXX.py` +echo "program output" >> $tmpfile +cp $infile $tmpfile +pyright --pythonversion $ver $tmpfile +rm $tmpfile +echo + +echo "== ruff ==" +ruff check --target-version $pyver $infile +echo + +echo "== pylint ==" +pylint --py-version $ver $infile + +echo "== config.status ==" +make $outfile +echo + diff --git a/cachegrind/Makefile.am b/cachegrind/Makefile.am index 8ea99ca529..34717ae541 100644 --- a/cachegrind/Makefile.am +++ b/cachegrind/Makefile.am @@ -93,28 +93,12 @@ endif # Miscellaneous #---------------------------------------------------------------------------- -# Run the formatters, type checkers, and linters on `cg_annotate.in`, then -# generate `cg_annotate`. -# -# Note: `pyright` refuses to check any file without a `.py` extension, hence -# the copying to `/tmp/tmp.py`. -ann: - @echo "== black ==" - @black cg_annotate.in - @echo - @echo "== isort ==" - @isort cg_annotate.in - @echo - @echo "== mypy ==" - @mypy --strict cg_annotate.in - @echo - @echo "== pyright ==" - @cp cg_annotate.in /tmp/tmp.py && pyright /tmp/tmp.py && rm /tmp/tmp.py - @echo - @echo "== ruff ==" - @ruff cg_annotate.in - @echo - @echo "== pylint ==" - @pylint cg_annotate.in - @echo "== config.status ==" - $(MAKE) cg_annotate +# "Build" `cg_annotate`. The `+` avoids warnings about the jobserver. +pyann: + +../auxprogs/pybuild.sh cg_annotate.in cg_annotate + +# "Build" `cg_diff`. The `+` avoids warnings about the jobserver. +pydiff: + +../auxprogs/pybuild.sh cg_diff.in cg_diff + +.PHONY: pyann pydiff diff --git a/cachegrind/cg_annotate.in b/cachegrind/cg_annotate.in index 43dce8f10e..240e069ccb 100755 --- a/cachegrind/cg_annotate.in +++ b/cachegrind/cg_annotate.in @@ -30,44 +30,9 @@ This script reads Cachegrind output files and produces human-readable reports. """ -# Use `make ann` to "build" this script every time it is changed. This runs the -# formatters, type-checkers, and linters on `cg_annotate.in` and then generates -# `cg_annotate`. -# -# Python versions: Currently this script targets Python 3.9 and later versions. -# Consequences of this: -# - No use of `TypeAlias` for explicit type aliases, which requires 3.10. -# -# The following Python tools are used. All can be installed with `pip3 install -# $NAME`, except `cProfile` which is built into Python. -# -# - Formatters: -# - `black`, for general formatting. This avoids the need for style checkers -# like `flake8`. Note that `black` allows a max line length of 88, which is -# a mild but common PEP-8 violation. -# - `isort`, for import sorting. -# -# - Type-checkers: -# - `mypy --strict`. This is the most commonly used Python type checker. -# - `pyright`. This is another good type checker. The `pyright: strict` -# comment above forces strict checking. -# - Sometimes one type-checker will complain about something the other does -# not. The goal is to keep both type checkers happy. -# -# - Linters: -# - `ruff`. Sometimes useful, and very fast to run. -# - `pylint`. Sometimes annoying, sometimes useful. The `pylintrc` -# modifies/disables the more annoying lints. -# -# - Profilers: -# - `cProfile` + `snakeviz`: Typically run with -# `python3 -m cProfile -o cg.prof cg_annotate $INPUT && snakeviz cg.prof`. -# - `scalene`. Typically run with `scalene ./cg_annotate $INPUT`. -# -# - Packager: -# - `cp` is used for distribution. This is possible because this program is a -# single file and only uses the Python Standard Library. This avoids the -# needs for any of the million different Python package management tools. +# Use `make pyann` to "build" this script with `auxprogs/pybuild.rs` every time +# it is changed. This runs the formatters, type-checkers, and linters on +# `cg_annotate.in` and then generates `cg_annotate`. from __future__ import annotations @@ -140,7 +105,7 @@ class Args(Namespace): help=f"(deprecated) same as --no-{name}", ) - p = ArgumentParser(description="Process Cachegrind output files.") + p = ArgumentParser(description="Process a Cachegrind output file.") p.add_argument("--version", action="version", version="%(prog)s-@VERSION@") @@ -317,11 +282,9 @@ class Cc: Flfn = NewType("Flfn", tuple[str, str]) # Per-function CCs. -# Note: not using `TypeAlias`. See "Python versions" comment above. DictFlfnCc = DefaultDict[Flfn, Cc] # Per-line CCs, organised by filename and line number. -# Note: not using `TypeAlias`. See "Python versions" comment above. DictLineCc = DefaultDict[int, Cc] DictFlDictLineCc = DefaultDict[str, DictLineCc] @@ -376,7 +339,7 @@ def read_cgout_file() -> tuple[str, str, Events, DictFlfnCc, DictFlDictLineCc, C curr_fl = "" curr_flfn = Flfn(("", "")) - # Three different places where we accumulate CC data. + # Different places where we accumulate CC data. dict_flfn_cc: DictFlfnCc = defaultdict(events.mk_empty_cc) dict_fl_dict_line_cc: DictFlDictLineCc = defaultdict(mk_empty_dict_line_cc) summary_cc = None @@ -479,8 +442,8 @@ class CcPrinter: min_cc.counts[i] = count # Find maximum width for each column. - self.count_widths = [0] * len(events.events) - self.perc_widths = [0] * len(events.events) + self.count_widths = [0] * events.num_events + self.perc_widths = [0] * events.num_events for i, event in enumerate(events.events): # Get count and perc widths of the min and max CCs. (min_count, min_perc) = self.count_and_perc(min_cc, i) @@ -757,16 +720,16 @@ def print_annotated_src_file( else: break - # If there was info on lines past the end of the file, warn. - if line_nums: - for line_num in line_nums: - printer.print_cc(dict_line_cc[line_num], f"<bogus line {line_num}>") - annotated_ccs.line_nums_known_cc += dict_line_cc[line_num] + # If there was info on lines past the end of the file, warn. + if line_nums: + for line_num in line_nums: + printer.print_cc(dict_line_cc[line_num], f"<bogus line {line_num}>") + annotated_ccs.line_nums_known_cc += dict_line_cc[line_num] - print() - warn_bogus_lines(src_file.name) + print() + warn_bogus_lines(src_file.name) - print() + print() # This (partially) consumes `dict_fl_dict_line_cc`. diff --git a/cachegrind/cg_diff.in b/cachegrind/cg_diff.in index 462308b49e..bae0c7abe4 100755 --- a/cachegrind/cg_diff.in +++ b/cachegrind/cg_diff.in @@ -1,13 +1,14 @@ -#! @PERL@ +#! /usr/bin/env python3 +# pyright: strict -##--------------------------------------------------------------------## -##--- Cachegrind's differencer. cg_diff.in ---## -##--------------------------------------------------------------------## +# -------------------------------------------------------------------- +# --- Cachegrind's differencer. cg_diff.in --- +# -------------------------------------------------------------------- # This file is part of Cachegrind, a Valgrind tool for cache # profiling programs. # -# Copyright (C) 2002-2017 Nicholas Nethercote +# Copyright (C) 2002-2023 Nicholas Nethercote # nj...@va... # # This program is free software; you can redistribute it and/or @@ -25,312 +26,340 @@ # # The GNU General Public License is contained in the file COPYING. -#---------------------------------------------------------------------------- -# This is a very cut-down and modified version of cg_annotate. -#---------------------------------------------------------------------------- - -use warnings; -use strict; - -#---------------------------------------------------------------------------- -# Global variables -#---------------------------------------------------------------------------- - -# Version number -my $version = "@VERSION@"; - -# Usage message. -my $usage = <<END -usage: cg_diff [options] <cachegrind-out-file1> <cachegrind-out-file2> - - options for the user, with defaults in [ ], are: - -h --help show this message - -v --version show version - --mod-filename=<expr> a Perl search-and-replace expression that is applied - to filenames, eg. --mod-filename='s/prog[0-9]/projN/' - --mod-funcname=<expr> like --mod-filename, but applied to function names - - cg_diff is Copyright (C) 2002-2017 Nicholas Nethercote. - and licensed under the GNU General Public License, version 2. - Bug reports, feedback, admiration, abuse, etc, to: njn\@valgrind.org. - -END -; - -# --mod-filename expression -my $mod_filename = undef; - -# --mod-funcname expression -my $mod_funcname = undef; - -#----------------------------------------------------------------------------- -# Argument and option handling -#----------------------------------------------------------------------------- -sub process_cmd_line() -{ - my ($file1, $file2) = (undef, undef); - - for my $arg (@ARGV) { - - if ($arg =~ /^-/) { - # --version - if ($arg =~ /^-v$|^--version$/) { - die("cg_diff-$version\n"); - - } elsif ($arg =~ /^--mod-filename=(.*)/) { - $mod_filename = $1; - - } elsif ($arg =~ /^--mod-funcname=(.*)/) { - $mod_funcname = $1; - - } else { # -h and --help fall under this case - die($usage); - } - - } elsif (not defined($file1)) { - $file1 = $arg; - - } elsif (not defined($file2)) { - $file2 = $arg; - - } else { - die($usage); - } - } - - # Must have specified two input files. - if (not defined $file1 or not defined $file2) { - die($usage); - } - - return ($file1, $file2); -} - -#----------------------------------------------------------------------------- -# Reading of input file -#----------------------------------------------------------------------------- -sub max ($$) -{ - my ($x, $y) = @_; - return ($x > $y ? $x : $y); -} - -# Add the two arrays; any '.' entries are ignored. Two tricky things: -# 1. If $a2->[$i] is undefined, it defaults to 0 which is what we want; we turn -# off warnings to allow this. This makes things about 10% faster than -# checking for definedness ourselves. -# 2. We don't add an undefined count or a ".", even though it's value is 0, -# because we don't want to make an $a2->[$i] that is undef become 0 -# unnecessarily. -sub add_array_a_to_b ($$) -{ - my ($a, $b) = @_; - - my $n = max(scalar @$a, scalar @$b); - $^W = 0; - foreach my $i (0 .. $n-1) { - $b->[$i] += $a->[$i] if (defined $a->[$i] && "." ne $a->[$i]); - } - $^W = 1; -} - -sub sub_array_b_from_a ($$) -{ - my ($a, $b) = @_; - - my $n = max(scalar @$a, scalar @$b); - $^W = 0; - foreach my $i (0 .. $n-1) { - $a->[$i] -= $b->[$i]; # XXX: doesn't handle '.' entries - } - $^W = 1; -} - -# Add each event count to the CC array. '.' counts become undef, as do -# missing entries (implicitly). -sub line_to_CC ($$) -{ - my ($line, $numEvents) = @_; - - my @CC = (split /\s+/, $line); - (@CC <= $numEvents) or die("Line $.: too many event counts\n"); - return \@CC; -} - -sub read_input_file($) -{ - my ($input_file) = @_; - - open(INPUTFILE, "< $input_file") - || die "Cannot open $input_file for reading\n"; - - # Read "desc:" lines. - my $desc; - my $line; - while ($line = <INPUTFILE>) { - if ($line =~ s/desc:\s+//) { - $desc .= $line; - } else { - last; - } - } - - # Read "cmd:" line (Nb: will already be in $line from "desc:" loop above). - ($line =~ s/^cmd:\s+//) or die("Line $.: missing command line\n"); - my $cmd = $line; - chomp($cmd); # Remove newline - - # Read "events:" line. We make a temporary hash in which the Nth event's - # value is N, which is useful for handling --show/--sort options below. - $line = <INPUTFILE>; - (defined $line && $line =~ s/^events:\s+//) - or die("Line $.: missing events line\n"); - my @events = split(/\s+/, $line); - my $numEvents = scalar @events; - - my $currFileName; - my $currFileFuncName; - - my %CCs; # hash("$filename###$funcname" => CC array) - my $currCC = undef; # CC array - - my $summaryCC; - - # Read body of input file. - while (<INPUTFILE>) { - # Skip comments and empty lines. - next if /^\s*$/ || /^\#/; - - if (s/^(-?\d+)\s+//) { - my $CC = line_to_CC($_, $numEvents); - defined($currCC) || die; - add_array_a_to_b($CC, $currCC); - - } elsif (s/^fn=(.*)$//) { - defined($currFileName) || die; - my $tmpFuncName = $1; - if (defined $mod_funcname) { - eval "\$tmpFuncName =~ $mod_funcname"; - } - $currFileFuncName = "$currFileName###$tmpFuncName"; - $currCC = $CCs{$currFileFuncName}; - if (not defined $currCC) { - $currCC = []; - $CCs{$currFileFuncName} = $currCC; - } - - } elsif (s/^fl=(.*)$//) { - $currFileName = $1; - if (defined $mod_filename) { - eval "\$currFileName =~ $mod_filename"; - } - # Assume that a "fn=" line is followed by a "fl=" line. - $currFileFuncName = undef; - - } elsif (s/^summary:\s+//) { - $summaryCC = line_to_CC($_, $numEvents); - (scalar(@$summaryCC) == @events) - or die("Line $.: summary event and total event mismatch\n"); - - } else { - warn("WARNING: line $. malformed, ignoring\n"); - } - } - - # Check if summary line was present - if (not defined $summaryCC) { - die("missing final summary line, aborting\n"); - } - - close(INPUTFILE); - - return ($cmd, \@events, \%CCs, $summaryCC); -} - -#---------------------------------------------------------------------------- -# "main()" -#---------------------------------------------------------------------------- -# Commands seen in the files. Need not match. -my $cmd1; -my $cmd2; - -# Events seen in the files. They must match. -my $events1; -my $events2; - -# Individual CCs, organised by filename/funcname/line_num. -# hashref("$filename###$funcname", CC array) -my $CCs1; -my $CCs2; - -# Total counts for summary (an arrayref). -my $summaryCC1; -my $summaryCC2; - -#---------------------------------------------------------------------------- -# Read the input files -#---------------------------------------------------------------------------- -my ($file1, $file2) = process_cmd_line(); -($cmd1, $events1, $CCs1, $summaryCC1) = read_input_file($file1); -($cmd2, $events2, $CCs2, $summaryCC2) = read_input_file($file2); - -#---------------------------------------------------------------------------- -# Check the events match -#---------------------------------------------------------------------------- -my $n = max(scalar @$events1, scalar @$events2); -$^W = 0; # turn off warnings, because we might hit undefs -foreach my $i (0 .. $n-1) { - ($events1->[$i] eq $events2->[$i]) || die "events don't match, aborting\n"; -} -$^W = 1; - -#---------------------------------------------------------------------------- -# Do the subtraction: CCs2 -= CCs1 -#---------------------------------------------------------------------------- -while (my ($filefuncname, $CC1) = each(%$CCs1)) { - my $CC2 = $CCs2->{$filefuncname}; - if (not defined $CC2) { - $CC2 = []; - sub_array_b_from_a($CC2, $CC1); # CC2 -= CC1 - $CCs2->{$filefuncname} = $CC2; - } else { - sub_array_b_from_a($CC2, $CC1); # CC2 -= CC1 - } -} -sub_array_b_from_a($summaryCC2, $summaryCC1); - -#---------------------------------------------------------------------------- -# Print the result, in CCs2 -#---------------------------------------------------------------------------- -print("desc: Files compared: $file1; $file2\n"); -print("cmd: $cmd1; $cmd2\n"); -print("events: "); -for my $e (@$events1) { - print(" $e"); -} -print("\n"); - -while (my ($filefuncname, $CC) = each(%$CCs2)) { - - my @x = split(/###/, $filefuncname); - (scalar @x == 2) || die; - - print("fl=$x[0]\n"); - print("fn=$x[1]\n"); - - print("0"); - foreach my $n (@$CC) { - print(" $n"); - } - print("\n"); -} - -print("summary:"); -foreach my $n (@$summaryCC2) { - print(" $n"); -} -print("\n"); - -##--------------------------------------------------------------------## -##--- end ---## -##--------------------------------------------------------------------## +""" +This script diffs Cachegrind output files. +""" + +# Use `make pydiff` to "build" this script every time it is changed. This runs +# the formatters, type-checkers, and linters on `cg_diff.in` and then generates +# `cg_diff`. +# +# This is a cut-down version of `cg_annotate.in`. + +from __future__ import annotations + +import re +import sys +from argparse import ArgumentParser, Namespace +from collections import defaultdict +from typing import Callable, DefaultDict, NewType, NoReturn + +SearchAndReplace = Callable[[str], str] + + +class Args(Namespace): + """ + A typed wrapper for parsed args. + + None of these fields are modified after arg parsing finishes. + """ + + mod_filename: SearchAndReplace + mod_funcname: SearchAndReplace + cgout_filename1: str + cgout_filename2: str + + @staticmethod + def parse() -> Args: + # We support Perl-style `s/old/new/flags` search-and-replace + # expressions, because that's how this option was implemented in the + # old Perl version of `cg_diff`. This requires conversion from + # `s/old/new/` style to `re.sub`. The conversion isn't a perfect + # emulation of Perl regexps (e.g. Python uses `\1` rather than `$1` for + # using captures in the `new` part), but it should be close enough. The + # only supported flags are `g` (global) and `i` (ignore case). + def search_and_replace(regex: str | None) -> SearchAndReplace: + if regex is None: + return lambda s: s + + # Extract the parts of a `s/old/new/tail` regex. `(?<!\\)/` is an + # example of negative lookbehind. It means "match a forward slash + # unless preceded by a backslash". + m = re.match(r"s/(.*)(?<!\\)/(.*)(?<!\\)/(g|i|gi|ig|)$", regex) + if m is None: + raise ValueError + + # Forward slashes must be escaped in an `s/old/new/` expression, + # but we then must unescape them before using them with `re.sub` + pat = m.group(1).replace(r"\/", r"/") + repl = m.group(2).replace(r"\/", r"/") + tail = m.group(3) + + if "g" in tail: + count = 0 # unlimited + else: + count = 1 + + if "i" in tail: + flags = re.IGNORECASE + else: + flags = re.RegexFlag(0) + + return lambda s: re.sub(re.compile(pat, flags=flags), repl, s, count=count) + + p = ArgumentParser(description="Diff two Cachegrind output files.") + + p.add_argument("--version", action="version", version="%(prog)s-@VERSION@") + + p.add_argument( + "--mod-filename", + type=search_and_replace, + metavar="REGEX", + default=search_and_replace(None), + help="a search-and-replace regex applied to filenames, e.g. " + "`s/prog[0-9]/progN/`", + ) + p.add_argument( + "--mod-funcname", + type=search_and_replace, + metavar="REGEX", + default=search_and_replace(None), + help="like --mod-filename, but for function names", + ) + + p.add_argument( + "cgout_filename1", + nargs=1, + metavar="cachegrind-out-file1", + help="file produced by Cachegrind", + ) + p.add_argument( + "cgout_filename2", + nargs=1, + metavar="cachegrind-out-file2", + help="file produced by Cachegrind", + ) + + return p.parse_args(namespace=Args()) + + +# Args are stored in a global for easy access. +args = Args.parse() + +# A single instance of this class is constructed, from `args` and the `events:` +# line in the cgout file. +class Events: + # The event names. + events: list[str] + + def __init__(self, text: str) -> None: + self.events = text.split() + self.num_events = len(self.events) + + def mk_cc(self, text: str) -> Cc: + """Raises a `ValueError` exception on syntax error.""" + # This is slightly faster than a list comprehension. + counts = list(map(int, text.split())) + + if len(counts) == self.num_events: + pass + elif len(counts) < self.num_events: + # Add zeroes at the end for any missing numbers. + counts.extend([0] * (self.num_events - len(counts))) + else: + raise ValueError + + return Cc(counts) + + def mk_empty_cc(self) -> Cc: + # This is much faster than a list comprehension. + return Cc([0] * self.num_events) + + +class Cc: + """ + This is a dumb container for counts. + + It doesn't know anything about events, i.e. what each count means. It can + do basic operations like `__iadd__` and `__eq__`, and anything more must be + done elsewhere. `Events.mk_cc` and `Events.mk_empty_cc` are used for + construction. + """ + + # Always the same length as `Events.events`. + counts: list[int] + + def __init__(self, counts: list[int]) -> None: + self.counts = counts + + def __repr__(self) -> str: + return str(self.counts) + + def __eq__(self, other: object) -> bool: + if not isinstance(other, Cc): + return NotImplemented + return self.counts == other.counts + + def __iadd__(self, other: Cc) -> Cc: + for i, other_count in enumerate(other.counts): + self.counts[i] += other_count + return self + + def __isub__(self, other: Cc) -> Cc: + for i, other_count in enumerate(other.counts): + self.counts[i] -= other_count + return self + + +# A paired filename and function name. +Flfn = NewType("Flfn", tuple[str, str]) + +# Per-function CCs. +DictFlfnCc = DefaultDict[Flfn, Cc] + + +def die(msg: str) -> NoReturn: + print("cg_diff: error:", msg, file=sys.stderr) + sys.exit(1) + + +def read_cgout_file(cgout_filename: str) -> tuple[str, Events, DictFlfnCc, Cc]: + # The file format is described in Cachegrind's manual. + try: + cgout_file = open(cgout_filename, "r", encoding="utf-8") + except OSError as err: + die(f"{err}") + + with cgout_file: + cgout_line_num = 0 + + def parse_die(msg: str) -> NoReturn: + die(f"{cgout_file.name}:{cgout_line_num}: {msg}") + + def readline() -> str: + nonlocal cgout_line_num + cgout_line_num += 1 + return cgout_file.readline() + + # Read "desc:" lines. + while line := readline(): + if m := re.match(r"desc:\s+(.*)", line): + # The "desc:" lines are unused. + pass + else: + break + + # Read "cmd:" line. (`line` is already set from the "desc:" loop.) + if m := re.match(r"cmd:\s+(.*)", line): + cmd = m.group(1) + else: + parse_die("missing a `command:` line") + + # Read "events:" line. + line = readline() + if m := re.match(r"events:\s+(.*)", line): + events = Events(m.group(1)) + else: + parse_die("missing an `events:` line") + + curr_fl = "" + curr_flfn = Flfn(("", "")) + + # Different places where we accumulate CC data. + dict_flfn_cc: DictFlfnCc = defaultdict(events.mk_empty_cc) + summary_cc = None + + # Compile the one hot regex. + count_pat = re.compile(r"(\d+)\s+(.*)") + + # Line matching is done in order of pattern frequency, for speed. + while True: + line = readline() + + if m := count_pat.match(line): + # The line_num isn't used. + try: + cc = events.mk_cc(m.group(2)) + except ValueError: + parse_die("malformed or too many event counts") + + # Record this CC at the function level. + flfn_cc = dict_flfn_cc[curr_flfn] + flfn_cc += cc + + elif line.startswith("fn="): + curr_flfn = Flfn((curr_fl, args.mod_funcname(line[3:-1]))) + + elif line.startswith("fl="): + # A longstanding bug: the use of `--mod-filename` makes it + # likely that some files won't be found when annotating. This + # doesn't matter much, because we use line number 0 for all + # diffs anyway. It just means we get "This file was unreadable" + # for modified filenames rather than a single "<unknown (line + # 0)>" CC. + curr_fl = args.mod_filename(line[3:-1]) + # A `fn=` line should follow, overwriting the "???". + curr_flfn = Flfn((curr_fl, "???")) + + elif m := re.match(r"summary:\s+(.*)", line): + try: + summary_cc = events.mk_cc(m.group(1)) + except ValueError: + parse_die("too many event counts") + + elif line == "": + break # EOF + + elif line == "\n" or line.startswith("#"): + # Skip empty lines and comment lines. + pass + + else: + parse_die(f"malformed line: {line[:-1]}") + + # Check if summary line was present. + if not summary_cc: + parse_die("missing `summary:` line, aborting") + + # Check summary is correct. + total_cc = events.mk_empty_cc() + for flfn_cc in dict_flfn_cc.values(): + total_cc += flfn_cc + if summary_cc != total_cc: + msg = ( + "`summary:` line doesn't match computed total\n" + f"- summary: {summary_cc}\n" + f"- total: {total_cc}" + ) + parse_die(msg) + + return (cmd, events, dict_flfn_cc, summary_cc) + + +def main() -> None: + filename1 = args.cgout_filename1[0] + filename2 = args.cgout_filename2[0] + + (cmd1, events1, dict_flfn_cc1, summary_cc1) = read_cgout_file(filename1) + (cmd2, events2, dict_flfn_cc2, summary_cc2) = read_cgout_file(filename2) + + if events1.num_events != events2.num_events: + die("events don't match") + + # Subtract file 1's CCs from file 2's CCs, at the Flfn level. + for flfn, flfn_cc1 in dict_flfn_cc1.items(): + flfn_cc2 = dict_flfn_cc2[flfn] + flfn_cc2 -= flfn_cc1 + summary_cc2 -= summary_cc1 + + print(f"desc: Files compared: {filename1}; {filename2}") + print(f"cmd: {cmd1}; {cmd2}") + print("events:", *events1.events, sep=" ") + + # Sort so the output is deterministic. + def key(flfn_and_cc: tuple[Flfn, Cc]) -> Flfn: + return flfn_and_cc[0] + + for flfn, flfn_cc2 in sorted(dict_flfn_cc2.items(), key=key): + # Use `0` for the line number because we don't try to give line-level + # CCs, due to the possibility of code changes causing line numbers to + # move around. + print(f"fl={flfn[0]}") + print(f"fn={flfn[1]}") + print("0", *flfn_cc2.counts, sep=" ") + + print("summary:", *summary_cc2.counts, sep=" ") + + +if __name__ == "__main__": + main() diff --git a/cachegrind/tests/Makefile.am b/cachegrind/tests/Makefile.am index 33baeeea69..16ac524b35 100644 --- a/cachegrind/tests/Makefile.am +++ b/cachegrind/tests/Makefile.am @@ -14,6 +14,7 @@ dist_noinst_SCRIPTS = filter_stderr filter_cachesim_discards # They just serve as input for cg_annotate in `ann1a` and `ann1b`. EXTRA_DIST = \ ann-diff1.post.exp ann-diff1.stderr.exp ann-diff1.vgtest \ + ann-diff2a.cgout ann-diff2b.cgout \ ann1a.post.exp ann1a.stderr.exp ann1a.vgtest ann1.cgout \ ann1b.post.exp ann1b.stderr.exp ann1b.vgtest ann1b.cgout \ ann2.post.exp ann2.stderr.exp ann2.vgtest ann2.cgout \ diff --git a/cachegrind/tests/ann-diff1.post.exp b/cachegrind/tests/ann-diff1.post.exp index 2d7d61ac70..f8d901b0a8 100644 --- a/cachegrind/tests/ann-diff1.post.exp +++ b/cachegrind/tests/ann-diff1.post.exp @@ -24,7 +24,7 @@ Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw -------------------------------------------------------------------------------- Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw file:function -5,000,000 (100.0%) 0 0 -2,000,000 (100.0%) 0 0 0 0 0 a.c:main +5,000,000 (100.0%) 0 0 -2,000,000 (100.0%) 0 0 0 0 0 a.c:MAIN -------------------------------------------------------------------------------- -- Auto-annotated source file: a.c @@ -33,7 +33,6 @@ Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw 5,000,000 (100.0%) 0 0 -2,000,000 (100.0%) 0 0 0 0 0 <unknown (line 0)> - -------------------------------------------------------------------------------- -- Annotation summary -------------------------------------------------------------------------------- diff --git a/cachegrind/tests/ann-diff1.vgtest b/cachegrind/tests/ann-diff1.vgtest index ce3e216c66..b737b713e3 100644 --- a/cachegrind/tests/ann-diff1.vgtest +++ b/cachegrind/tests/ann-diff1.vgtest @@ -2,5 +2,5 @@ # the post-processing of the `ann{1,1b}.cgout` test files. prog: ../../tests/true vgopts: --cachegrind-out-file=cachegrind.out -post: perl ../../cachegrind/cg_diff ann1.cgout ann1b.cgout > ann-diff1.cgout && perl ../../cachegrind/cg_annotate ann-diff1.cgout +post: python ../../cachegrind/cg_diff --mod-funcname="s/main/MAIN/" ann1.cgout ann1b.cgout > ann-diff1.cgout && python ../../cachegrind/cg_annotate ann-diff1.cgout cleanup: rm ann-diff1.cgout diff --git a/cachegrind/tests/ann-diff2-aux/ann-diff2-basic.rs b/cachegrind/tests/ann-diff2-aux/ann-diff2-basic.rs new file mode 100644 index 0000000000..c9e9e05f44 --- /dev/null +++ b/cachegrind/tests/ann-diff2-aux/ann-diff2-basic.rs @@ -0,0 +1,10 @@ +one +two +three +four +five +six +seven +eight +nine +ten diff --git a/cachegrind/tests/ann-diff2.post.exp b/cachegrind/tests/ann-diff2.post.exp new file mode 100644 index 0000000000..742ff3841c --- /dev/null +++ b/cachegrind/tests/ann-diff2.post.exp @@ -0,0 +1,46 @@ +-------------------------------------------------------------------------------- +-- Cachegrind profile +-------------------------------------------------------------------------------- +Files compared: ann-diff2a.cgout; ann-diff2b.cgout +Command: cmd1; cmd2 +Data file: ann-diff2c.cgout +Events recorded: One Two +Events shown: One Two +Event sort order: One Two +Threshold: 0.1 +Include dirs: +User annotated: +Auto-annotation: on + +-------------------------------------------------------------------------------- +-- Summary +-------------------------------------------------------------------------------- +One Two + +2,100 (100.0%) 1,900 (100.0%) PROGRAM TOTALS + +-------------------------------------------------------------------------------- +-- Function summary +-------------------------------------------------------------------------------- +One Two file:function + +1,000 (47.6%) 1,000 (52.6%) aux/ann-diff2-basic.rs:groffN +1,000 (47.6%) 1,000 (52.6%) aux/ann-diff2-basic.rs:fN_ffN_fooN_F4_g5 + 100 (4.8%) -100 (-5.3%) aux/ann-diff2-basic.rs:basic1 + +-------------------------------------------------------------------------------- +-- Auto-annotated source file: aux/ann-diff2-basic.rs +-------------------------------------------------------------------------------- +This file was unreadable + +-------------------------------------------------------------------------------- +-- Annotation summary +-------------------------------------------------------------------------------- +One Two + + 0 0 annotated: files known & above threshold & readable, line numbers known + 0 0 annotated: files known & above threshold & readable, line numbers unknown +2,100 (100.0%) 1,900 (100.0%) unannotated: files known & above threshold & unreadable + 0 0 unannotated: files known & below threshold + 0 0 unannotated: files unknown + diff --git a/cachegrind/tests/ann-diff2.stderr.exp b/cachegrind/tests/ann-diff2.stderr.exp new file mode 100644 index 0000000000..e8084c12c3 --- /dev/null +++ b/cachegrind/tests/ann-diff2.stderr.exp @@ -0,0 +1,17 @@ + + +I refs: +I1 misses: +LLi misses: +I1 miss rate: +LLi miss rate: + +D refs: +D1 misses: +LLd misses: +D1 miss rate: +LLd miss rate: + +LL refs: +LL misses: +LL miss rate: diff --git a/cachegrind/tests/ann-diff2.vgtest b/cachegrind/tests/ann-diff2.vgtest new file mode 100644 index 0000000000..101cac07d1 --- /dev/null +++ b/cachegrind/tests/ann-diff2.vgtest @@ -0,0 +1,6 @@ +# The `prog` doesn't matter because we don't use its output. Instead we test +# the post-processing of the `ann-diff2{a,b}.cgout` test files. +prog: ../../tests/true +vgopts: --cachegrind-out-file=cachegrind.out +post: python ../../cachegrind/cg_diff --mod-filename="s/.*aux\//aux\//i" --mod-funcname="s/(f[a-z]*)[0-9]/\1N/g" ann-diff2a.cgout ann-diff2b.cgout > ann-diff2c.cgout && python ../../cachegrind/cg_annotate ann-diff2c.cgout +cleanup: rm ann-diff2c.cgout diff --git a/cachegrind/tests/ann-diff2a.cgout b/cachegrind/tests/ann-diff2a.cgout new file mode 100644 index 0000000000..bb82b75f6c --- /dev/null +++ b/cachegrind/tests/ann-diff2a.cgout @@ -0,0 +1,9 @@ +desc: Description for ann-diff2a.cgout +cmd: cmd1 +events: One Two + +fl=ann2-diff-AUX/ann-diff2-basic.rs +fn=basic1 +1 1000 1000 + +summary: 1000 1000 diff --git a/cachegrind/tests/ann-diff2b.cgout b/cachegrind/tests/ann-diff2b.cgout new file mode 100644 index 0000000000..9fb733e708 --- /dev/null +++ b/cachegrind/tests/ann-diff2b.cgout @@ -0,0 +1,15 @@ +desc: Description for ann-diff2a.cgout +cmd: cmd2 +events: One Two + +fl=ann2-diff-Aux/ann-diff2-basic.rs +fn=basic1 +1 1100 900 + +fn=f1_ff2_foo3_F4_g5 +3 1000 1000 + +fn=groff5 +5 1000 1000 + +summary: 3100 2900 |
|
From: Nicholas N. <nj...@so...> - 2023-03-28 04:22:13
|
https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=3f1bbe12fec49c6cc85bfb4a5ba1e021fde0b144 commit 3f1bbe12fec49c6cc85bfb4a5ba1e021fde0b144 Author: Nicholas Nethercote <n.n...@gm...> Date: Thu Mar 23 19:36:47 2023 +1100 Make section formatting more consistent. - Every section now has a heading with the long `----` lines above and below. - Event names are always shown below that heading, rather than within it. - Each Unreadable file now gets its own section, much like files that lack any data. Diff: --- cachegrind/cg_annotate.in | 116 +++++++++++++++++-------------------- cachegrind/tests/ann-diff.post.exp | 16 +++-- cachegrind/tests/ann1a.post.exp | 60 ++++++++++++++----- cachegrind/tests/ann1b.post.exp | 16 +++-- cachegrind/tests/ann2.post.exp | 44 +++++++++----- 5 files changed, 152 insertions(+), 100 deletions(-) diff --git a/cachegrind/cg_annotate.in b/cachegrind/cg_annotate.in index bf09c36513..43dce8f10e 100755 --- a/cachegrind/cg_annotate.in +++ b/cachegrind/cg_annotate.in @@ -264,6 +264,7 @@ class Events: self.sort_indices = [event_indices[event] for event in self.sort_events] def mk_cc(self, text: str) -> Cc: + """Raises a `ValueError` exception on syntax error.""" # This is slightly faster than a list comprehension. counts = list(map(int, text.split())) @@ -539,11 +540,15 @@ class CcPrinter: # Used in various places in the output. -FANCY: str = "-" * 80 +def print_fancy(text: str) -> None: + fancy = "-" * 80 + print(fancy) + print("--", text) + print(fancy) -def print_header(desc: str, cmd: str, events: Events) -> None: - print(FANCY) +def print_cachegrind_profile(desc: str, cmd: str, events: Events) -> None: + print_fancy("Cachegrind profile") print(desc, end="") print("Command: ", cmd) print("Data file: ", args.cgout_filename[0]) @@ -570,17 +575,16 @@ def print_header(desc: str, cmd: str, events: Events) -> None: print() -def print_summary_cc(events: Events, summary_cc: Cc) -> None: +def print_summary(events: Events, summary_cc: Cc) -> None: printer = CcPrinter(events, [summary_cc], summary_cc) - - print(FANCY) + print_fancy("Summary") printer.print_events("") - print(FANCY) + print() printer.print_cc(summary_cc, "PROGRAM TOTALS") print() -def print_flfn_ccs( +def print_function_summary( events: Events, dict_flfn_cc: DictFlfnCc, summary_cc: Cc ) -> set[str]: # Only the first threshold percentage is actually used. @@ -607,10 +611,9 @@ def print_flfn_ccs( sorted_ccs = list(map(lambda flfn_and_cc: flfn_and_cc[1], sorted_flfns_and_ccs)) printer = CcPrinter(events, sorted_ccs, summary_cc) - - print(FANCY) + print_fancy("Function summary") printer.print_events(" file:function") - print(FANCY) + print() # Print per-function counts. for flfn, flfn_cc in sorted_flfns_and_ccs: @@ -666,7 +669,7 @@ def mk_warning(msg: str) -> str: def warn_src_file_is_newer(src_filename: str, cgout_filename: str) -> None: msg = f"""\ -@ Source file '{src_filename}' is more recent than input file '{cgout_filename}'. +@ Source file '{src_filename}' is newer than data file '{cgout_filename}'. @ Annotations may not be correct. """ print(mk_warning(msg)) @@ -681,28 +684,17 @@ def warn_bogus_lines(src_filename: str) -> None: def print_annotated_src_file( events: Events, - dict_line_cc: DictLineCc | None, - ann_type: str, + dict_line_cc: DictLineCc, src_file: TextIO, annotated_ccs: AnnotatedCcs, summary_cc: Cc, ) -> None: - print(FANCY) - print("-- ", ann_type, "-annotated source: ", src_file.name, sep="") - print(FANCY) - - # Get file's CCs. - if not dict_line_cc: - print(f" No information has been collected for {src_file.name}") - print() - return - # If the source file is more recent than the cgout file, issue warning. if os.stat(src_file.name).st_mtime_ns > os.stat(args.cgout_filename[0]).st_mtime_ns: warn_src_file_is_newer(src_file.name, args.cgout_filename[0]) printer = CcPrinter(events, list(dict_line_cc.values()), summary_cc) - + # The starting fancy has already been printed by the caller. printer.print_events("") print() @@ -783,8 +775,7 @@ def print_annotated_src_files( threshold_src_filenames: set[str], dict_fl_dict_line_cc: DictFlDictLineCc, summary_cc: Cc, -) -> tuple[list[str], AnnotatedCcs]: - unreadable_auto_filenames: list[str] = [] +) -> AnnotatedCcs: annotated_ccs = AnnotatedCcs(events) def pair_with(label: str) -> Callable[[str], tuple[str, str]]: @@ -810,8 +801,11 @@ def print_annotated_src_files( include_dirnames = args.include.copy() include_dirnames.insert(0, "") + def print_ann_fancy(ann_type: str, src_filename: str) -> None: + print_fancy(f"{ann_type}-annotated source file: {src_filename}") + for src_filename, ann_type in sorted(all_src_filenames): - annotated = False + readable = False for include_dirname in include_dirnames: if include_dirname == "": full_src_filename = src_filename @@ -820,45 +814,45 @@ def print_annotated_src_files( try: with open(full_src_filename, "r", encoding="utf-8") as src_file: - # The pop will fail if it's a user-specified filename that - # isn't mentioned in the cgout file. - print_annotated_src_file( - events, - dict_fl_dict_line_cc.pop(src_filename, None), - ann_type, - src_file, - annotated_ccs, - summary_cc, - ) - annotated = True + dict_line_cc = dict_fl_dict_line_cc.pop(src_filename, None) + if dict_line_cc is not None: + print_ann_fancy(ann_type, src_file.name) # includes full path + print_annotated_src_file( + events, + dict_line_cc, + src_file, + annotated_ccs, + summary_cc, + ) + else: + # This only happens for user-specified files that are + # readable but not mentioned in the cgout file. + print_ann_fancy(ann_type, src_filename) + print("This file was not mentioned by the data file") + print() + + readable = True break except OSError: pass - if not annotated: - unreadable_auto_filenames.append(src_filename) + if not readable: dict_line_cc = dict_fl_dict_line_cc.pop(src_filename, None) add_dict_line_cc_to_cc(dict_line_cc, annotated_ccs.unreadable_cc) + print_ann_fancy(ann_type, src_filename) + print("This file was unreadable") + print() + # Sum the CCs remaining in `dict_fl_dict_line_cc`, which are all in files # below the threshold. for dict_line_cc in dict_fl_dict_line_cc.values(): add_dict_line_cc_to_cc(dict_line_cc, annotated_ccs.below_threshold_cc) - return (unreadable_auto_filenames, annotated_ccs) + return annotated_ccs -def print_unreadable_auto_filenames(unreadable_auto_filenames: list[str]) -> None: - if unreadable_auto_filenames: - print(FANCY) - print("The following files chosen for auto-annotation could not be read:") - print(FANCY) - for filename in sorted(unreadable_auto_filenames): - print(" ", filename) - print() - - -def print_annotated_ccs( +def print_annotation_summary( events: Events, annotated_ccs: AnnotatedCcs, summary_cc: Cc, @@ -867,9 +861,9 @@ def print_annotated_ccs( # lines above. if args.auto or args.src_filenames: printer = CcPrinter(events, annotated_ccs.ccs(), summary_cc) - print(FANCY) + print_fancy("Annotation summary") printer.print_events("") - print(FANCY) + print() total_cc = events.mk_empty_cc() for (cc, label) in zip(annotated_ccs.ccs(), AnnotatedCcs.labels): @@ -900,19 +894,17 @@ def main() -> None: # Each of the following calls prints a section of the output. - print_header(desc, cmd, events) + print_cachegrind_profile(desc, cmd, events) - print_summary_cc(events, summary_cc) + print_summary(events, summary_cc) - threshold_src_filenames = print_flfn_ccs(events, dict_flfn_cc, summary_cc) + threshold_src_filenames = print_function_summary(events, dict_flfn_cc, summary_cc) - (unreadable_auto_filenames, annotated_ccs) = print_annotated_src_files( + annotated_ccs = print_annotated_src_files( events, threshold_src_filenames, dict_fl_dict_line_cc, summary_cc ) - print_unreadable_auto_filenames(unreadable_auto_filenames) - - print_annotated_ccs(events, annotated_ccs, summary_cc) + print_annotation_summary(events, annotated_ccs, summary_cc) if __name__ == "__main__": diff --git a/cachegrind/tests/ann-diff.post.exp b/cachegrind/tests/ann-diff.post.exp index 986edf35ef..9f8f276ff1 100644 --- a/cachegrind/tests/ann-diff.post.exp +++ b/cachegrind/tests/ann-diff.post.exp @@ -1,4 +1,6 @@ -------------------------------------------------------------------------------- +-- Cachegrind profile +-------------------------------------------------------------------------------- Files compared: ann1.cgout; ann1b.cgout Command: ./a.out; ./a.out Data file: ann-diff.cgout @@ -11,17 +13,21 @@ User annotated: Auto-annotation: on -------------------------------------------------------------------------------- -Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw +-- Summary -------------------------------------------------------------------------------- +Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw + 5,000,000 (100.0%) 0 0 -2,000,000 (100.0%) 0 0 0 0 0 PROGRAM TOTALS -------------------------------------------------------------------------------- -Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw file:function +-- Function summary -------------------------------------------------------------------------------- +Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw file:function + 5,000,000 (100.0%) 0 0 -2,000,000 (100.0%) 0 0 0 0 0 a.c:main -------------------------------------------------------------------------------- --- Auto-annotated source: a.c +-- Auto-annotated source file: a.c -------------------------------------------------------------------------------- Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw @@ -29,8 +35,10 @@ Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw -------------------------------------------------------------------------------- -Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw +-- Annotation summary -------------------------------------------------------------------------------- +Ir I1mr ILmr Dr D1mr DLmr Dw D1mw DLmw + 0 0 0 0 0 0 0 0 0 annotated: files known & above threshold & readable, line numbers known 5,000,000 (100.0%) 0 0 -2,000,000 (100.0%) 0 0 0 0 0 annotated: files known & above threshold & readable, line numbers unknown 0 0 0 0 0 0 0 0 0 unannotated: files known & above threshold & unreadable diff --git a/cachegrind/tests/ann1a.post.exp b/cachegrind/tests/ann1a.post.exp index 05e9c5afa6..b8a5f53b28 100644 --- a/cachegrind/tests/ann1a.post.exp +++ b/cachegrind/tests/ann1a.post.exp @@ -1,4 +1,6 @@ -------------------------------------------------------------------------------- +-- Cachegrind profile +-------------------------------------------------------------------------------- I1 cache: 32768 B, 64 B, 8-way associative D1 cache: 32768 B, 64 B, 8-way associative LL cache: 19922944 B, 64 B, 19-way associative @@ -13,13 +15,17 @@ User annotated: Auto-annotation: on -------------------------------------------------------------------------------- -Ir I1mr ILmr +-- Summary -------------------------------------------------------------------------------- +Ir I1mr ILmr + 5,229,753 952 931 PROGRAM TOTALS -------------------------------------------------------------------------------- -Ir I1mr ILmr file:function +-- Function summary -------------------------------------------------------------------------------- +Ir I1mr ILmr file:function + 5,000,015 1 1 a.c:main 47,993 19 19 /build/glibc-OTsEL5/glibc-2.27/elf/dl-lookup.c:do_lookup_x 28,534 11 11 /build/glibc-OTsEL5/glibc-2.27/elf/dl-lookup.c:_dl_lookup_symbol_x @@ -31,7 +37,42 @@ Ir I1mr ILmr file:function 6,898 2 2 /build/glibc-OTsEL5/glibc-2.27/elf/dl-misc.c:_dl_name_match_p -------------------------------------------------------------------------------- --- Auto-annotated source: a.c +-- Auto-annotated source file: /build/glibc-OTsEL5/glibc-2.27/elf/../sysdeps/x86_64/dl-machine.h +-------------------------------------------------------------------------------- +This file was unreadable + +-------------------------------------------------------------------------------- +-- Auto-annotated source file: /build/glibc-OTsEL5/glibc-2.27/elf/dl-lookup.c +-------------------------------------------------------------------------------- +This file was unreadable + +-------------------------------------------------------------------------------- +-- Auto-annotated source file: /build/glibc-OTsEL5/glibc-2.27/elf/dl-misc.c +-------------------------------------------------------------------------------- +This file was unreadable + +-------------------------------------------------------------------------------- +-- Auto-annotated source file: /build/glibc-OTsEL5/glibc-2.27/elf/dl-tunables.c +-------------------------------------------------------------------------------- +This file was unreadable + +-------------------------------------------------------------------------------- +-- Auto-annotated source file: /build/glibc-OTsEL5/glibc-2.27/elf/dl-tunables.h +-------------------------------------------------------------------------------- +This file was unreadable + +-------------------------------------------------------------------------------- +-- Auto-annotated source file: /build/glibc-OTsEL5/glibc-2.27/elf/do-rel.h +-------------------------------------------------------------------------------- +This file was unreadable + +-------------------------------------------------------------------------------- +-- Auto-annotated source file: /build/glibc-OTsEL5/glibc-2.27/string/../sysdeps/x86_64/strcmp.S +-------------------------------------------------------------------------------- +This file was unreadable + +-------------------------------------------------------------------------------- +-- Auto-annotated source file: a.c -------------------------------------------------------------------------------- Ir I1mr ILmr @@ -44,19 +85,10 @@ Ir I1mr ILmr 2 0 0 } -------------------------------------------------------------------------------- -The following files chosen for auto-annotation could not be read: --------------------------------------------------------------------------------- - /build/glibc-OTsEL5/glibc-2.27/elf/../sysdeps/x86_64/dl-machine.h - /build/glibc-OTsEL5/glibc-2.27/elf/dl-lookup.c - /build/glibc-OTsEL5/glibc-2.27/elf/dl-misc.c - /build/glibc-OTsEL5/glibc-2.27/elf/dl-tunables.c - /build/glibc-OTsEL5/glibc-2.27/elf/dl-tunables.h - /build/glibc-OTsEL5/glibc-2.27/elf/do-rel.h - /build/glibc-OTsEL5/glibc-2.27/string/../sysdeps/x86_64/strcmp.S - +-- Annotation summary -------------------------------------------------------------------------------- Ir I1mr ILmr --------------------------------------------------------------------------------- + 5,000,015 1 1 annotated: files known & above threshold & readable, line numbers known 0 0 0 annotated: files known & above threshold & readable, line numbers unknown 179,512 136 134 unannotated: files known & above threshold & unreadable diff --git a/cachegrind/tests/ann1b.post.exp b/cachegrind/tests/ann1b.post.exp index 302416a44d..4ad3a2eddf 100644 --- a/cachegrind/tests/ann1b.post.exp +++ b/cachegrind/tests/ann1b.post.exp @@ -1,4 +1,6 @@ -------------------------------------------------------------------------------- +-- Cachegrind profile +-------------------------------------------------------------------------------- I1 cache: 32768 B, 64 B, 8-way associative D1 cache: 32768 B, 64 B, 8-way associative LL cache: 19922944 B, 64 B, 19-way associative @@ -13,13 +15,17 @@ User annotated: a.c Auto-annotation: off -------------------------------------------------------------------------------- -Dw Dr Ir +-- Summary -------------------------------------------------------------------------------- +Dw Dr Ir + 18,005 (100.0%) 4,057,955 (100.0%) 5,229,753 (100.0%) PROGRAM TOTALS -------------------------------------------------------------------------------- -Dw Dr Ir file:function +-- Function summary -------------------------------------------------------------------------------- +Dw Dr Ir file:function + 3 (0.0%) 4,000,004 (98.6%) 5,000,015 (95.6%) a.c:main 4,543 (25.2%) 17,566 (0.4%) 47,993 (0.9%) /build/glibc-OTsEL5/glibc-2.27/elf/dl-lookup.c:do_lookup_x 3,083 (17.1%) 5,750 (0.1%) 28,534 (0.5%) /build/glibc-OTsEL5/glibc-2.27/elf/dl-lookup.c:_dl_lookup_symbol_x @@ -28,7 +34,7 @@ Dw Dr Ir file:function 0 5,158 (0.1%) 25,408 (0.5%) /build/glibc-OTsEL5/glibc-2.27/string/../sysdeps/x86_64/strcmp.S:strcmp -------------------------------------------------------------------------------- --- User-annotated source: a.c +-- User-annotated source file: a.c -------------------------------------------------------------------------------- Dw Dr Ir @@ -41,8 +47,10 @@ Dw Dr Ir 0 2 (0.0%) 2 (0.0%) } -------------------------------------------------------------------------------- -Dw Dr Ir +-- Annotation summary -------------------------------------------------------------------------------- +Dw Dr Ir + 3 (0.0%) 4,000,004 (98.6%) 5,000,015 (95.6%) annotated: files known & above threshold & readable, line numbers known 0 0 0 annotated: files known & above threshold & readable, line numbers unknown 0 0 0 unannotated: files known & above threshold & unreadable diff --git a/cachegrind/tests/ann2.post.exp b/cachegrind/tests/ann2.post.exp index 204a472d3e..315e13474d 100644 --- a/cachegrind/tests/ann2.post.exp +++ b/cachegrind/tests/ann2.post.exp @@ -1,4 +1,6 @@ -------------------------------------------------------------------------------- +-- Cachegrind profile +-------------------------------------------------------------------------------- Command: ann2 Data file: ann2.cgout Events recorded: A SomeCount VeryLongEventName @@ -13,13 +15,17 @@ User annotated: ann2-unmentioned.rs Auto-annotation: on -------------------------------------------------------------------------------- -A SomeCount VeryLongEventName +-- Summary -------------------------------------------------------------------------------- +A SomeCount VeryLongEventName + 100,000 (100.0%) 100,000 (100.0%) 0 PROGRAM TOTALS -------------------------------------------------------------------------------- -A SomeCount VeryLongEventName file:function +-- Function summary -------------------------------------------------------------------------------- +A SomeCount VeryLongEventName file:function + 70,091 (70.1%) 90,291 (90.3%) 0 ann2-basic.rs:f0 15,000 (15.0%) 600 (0.6%) 0 ann2-basic.rs:f1 9,000 (9.0%) 6,000 (6.0%) 0 ann2-could-not-be-found.rs:f1 @@ -34,7 +40,7 @@ A SomeCount VeryLongEventName file:function 500 (0.5%) 0 0 ann2-basic.rs:f4 -------------------------------------------------------------------------------- --- Auto-annotated source: ann2-basic.rs +-- Auto-annotated source file: ann2-basic.rs -------------------------------------------------------------------------------- A SomeCount VeryLongEventName @@ -62,12 +68,17 @@ A SomeCount VeryLongEventName 300 (0.3%) 0 0 twenty -------------------------------------------------------------------------------- --- Auto-annotated source: ann2-more-recent-than-cgout.rs +-- Auto-annotated source file: ann2-could-not-be-found.rs +-------------------------------------------------------------------------------- +This file was unreadable + +-------------------------------------------------------------------------------- +-- Auto-annotated source file: ann2-more-recent-than-cgout.rs -------------------------------------------------------------------------------- @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@ WARNING @@ WARNING @@ WARNING @@ WARNING @@ WARNING @@ WARNING @@ WARNING @@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ -@ Source file 'ann2-more-recent-than-cgout.rs' is more recent than input file 'ann2.cgout'. +@ Source file 'ann2-more-recent-than-cgout.rs' is newer than data file 'ann2.cgout'. @ Annotations may not be correct. @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@ -80,7 +91,7 @@ A SomeCount VeryLongEventName -- line 4 ---------------------------------------- -------------------------------------------------------------------------------- --- Auto-annotated source: ann2-negatives.rs +-- Auto-annotated source file: ann2-negatives.rs -------------------------------------------------------------------------------- A SomeCount VeryLongEventName @@ -102,7 +113,12 @@ A SomeCount VeryLongEventName -- line 13 ---------------------------------------- -------------------------------------------------------------------------------- --- Auto-annotated source: ann2-past-the-end.rs +-- User-annotated source file: ann2-no-such-file.rs +-------------------------------------------------------------------------------- +This file was unreadable + +-------------------------------------------------------------------------------- +-- Auto-annotated source file: ann2-past-the-end.rs -------------------------------------------------------------------------------- A SomeCount VeryLongEventName @@ -122,26 +138,22 @@ A SomeCount VeryLongEventName @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ -------------------------------------------------------------------------------- --- User-annotated source: ann2-unmentioned.rs +-- User-annotated source file: ann2-unmentioned.rs -------------------------------------------------------------------------------- - No information has been collected for ann2-unmentioned.rs +This file was not mentioned by the data file -------------------------------------------------------------------------------- --- Auto-annotated source: ann2-aux/ann2-via-I.rs +-- Auto-annotated source file: ann2-aux/ann2-via-I.rs -------------------------------------------------------------------------------- A SomeCount VeryLongEventName 1,000 (1.0%) 500 (0.5%) 0 one -------------------------------------------------------------------------------- -The following files chosen for auto-annotation could not be read: --------------------------------------------------------------------------------- - ann2-could-not-be-found.rs - ann2-no-such-file.rs - +-- Annotation summary -------------------------------------------------------------------------------- A SomeCount VeryLongEventName --------------------------------------------------------------------------------- + 84,500 (84.5%) 94,700 (94.7%) 990 (n/a) annotated: files known & above threshold & readable, line numbers known 5,100 (5.1%) -900 (-0.9%) -990 (n/a) annotated: files known & above threshold & readable, line numbers unknown 9,000 (9.0%) 6,000 (6.0%) 0 unannotated: files known & above threshold & unreadable |