Testing PRs on Debian

[Daniel Marjamäki]

I am experimenting with a webserver that will execute the test-my-pr script on PRs. I want more or less that it will provide daca@home feedback directly when somebody works on a PR.

http://ec2-16-170-140-253.eu-north-1.compute.amazonaws.com/

It is only executed on open non-draft PRs that have a successful CI.

It will execute on 1000 packages if the PR modifies tokenizer or valueflow. Otherwise it will execute on 100 packages. When I started it , it would count packages even if they did not have C/C++ code that is fixed now..

AI-generated verdicts will be provided for a random subset of the warnings (up to 20 or so). Those should be taken with a grain of salt for now.

Please provide feedback. Do you think I should tweak the reports or output somehow?

[CHR]

Please provide feedback. Do you think I should tweak the reports or output somehow?

Having a link to the actual PR would be nice (instead of/additional to linking to the AWS files).
How is reanalysis supposed to work? E.g. for #8918, the latest commit shown is from Oct 5, while there is a newer commit from Oct 7 already (https://github.com/cppcheck-opensource/cppcheck/pull/8918/changes/97161058bc8dbc7b18acf69ae1cf324a21507f65).

[CHR]

Good idea 👍
Having the user name in a column would be nice.
Also, some PRs seem to be missing. Are they still being processed?

[Daniel Marjamäki]

It's still being processed. If we will have trouble to keep up with open PRs I think we can launch more runner VMs.
It might be low with 100/1000 .. I will see if it makes sense to increase these values later. At least if a runner is idle it could fill up more packages..

[CHR]

It looks like merged PRs are not flushed out, e.g. https://github.com/cppcheck-opensource/cppcheck/pull/8934

[john borland]

Ok I just noticed some of the dates in those PR files I got the packages
from.

I will remove all the PR files from before Oct 6. Given that is about half
of them I will probably add a few more and rerun my code. Hopefully I will
see better coverage numbers on the low end.

What ever happened to http://cppcheck1.osuosl.org:8000/?

When I try to go to the link it says the website can't be reached. I was
thinking of pulling the package list from there but the site seems down. I
was hoping to also get the list of packages that crash or hang up cop
check. That way I could have my script avoid them.

On Thu, Oct 22, 2026, 4:21 AM CHR chrchr@users.sourceforge.net wrote:

It looks like merged PRs are not flushed out, e.g.
https://github.com/cppcheck-opensource/cppcheck/pull/8934

---

Testing PRs on Debian
https://sourceforge.net/p/cppcheck/discussion/development/thread/1cf7f0973e/?limit=25#9c1d/af4c

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/cppcheck/discussion/development/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

[john borland]

Hopefully the table below looks ok. I will attach 2 CSV files. The
first one is called StatFile.csv and that is this table below. Which shows
some stats about code coverage of a few of the CppCheck source code files
and all the numbers are %. I checked 4015 Debian packages. I can say the
Overall Coverage numbers are under estimates. When I looked at the gcov
data I found that it was saying there were 220048 source lines, but going
through the output it was counting files outside of cppcheck
like /usr/include/c++/14/bits/chrono.h, /usr/include/c++/14/bits/stl_tree.h
and other system files. So I will write a parser for a later date to fix
that. The 2nd file is Main.coverage.csv it shows a few more files and all
the packages that were checked. One of the reasons I started looking into
this is gcc has made some changes to code coverage tooling(gcov) in version
14 and 15. I often prototype a lot of these types of things in bash just
to quickly find the interesting parts. Long term I will probably rewrite it
all in python. The statical mode tends to be very interesting for code
coverage. In the past I have used that for estimating the number of paths
in a code base. One of the changes to gcc 15 is to give you that
information. So I was using this as a way to test how good my old
estimates were. I just thought this information might be useful for the PR
script to maybe pick 10 packages off the list with high code coverage in
conjunction with a large amount of random packages. I am just hoping that
the packages with larger coverage would help find issues at the PR level
before code is let in and then gets caught in the cppcheck1.osuosl.org
http://cppcheck1.osuosl.org:8000/ (Which still shows that it can't be
reached by at least me).

Once I fix the overall coverage issue I might make another csv file showing
the frequency of coverage %. If there is any other information you would
like to know please feel free to let me know.

FileName Mean StdDev Median Mode ModeFreq
Range Max Min
Overallcoverage 22.25 5.78 22.8 4.34 0.473 32.35 35.99 3.64
checkothercpp 9.66 2.52 9.87 0 4.334 14.84 14.84 0
tokenlistcpp 51.18 13.52 50.62 1.23 1.519 84.41 84.41 0
astutilscpp 55.77 17.96 58.39 0 2.565 88.27 88.27 0
symboldatabasecpp 40.69 14.62 38.91 0 3.587 78.51 78.51 0
tokencpp 57.21 20.65 55.78 0 3.138 94.94 94.94 0
forwardanalyzercpp 72.06 23.47 80.46 0 6.476 96.04 96.04 0
valueflowcpp 48.18 17.17 49.42 0 4.334 88.06 88.06 0
checkclasscpp 4.36 4.19 1.03 1.03 53.25 13.94 14.37 0.43
checkconditioncpp 3.54 0.75 3.7 3.7 95.666 3.7 3.7 0
programmemorycpp 52.39 16.96 56.96 0 6.351 89.75 89.75 0
vfvaluecpp 34.17 13.71 40.15 40.91 8.045 58.33 58.33 0
checkiocpp 16.98 8.69 17.33 7.04 6.027 45.29 45.29 0
templatesimplifiercpp 10.83 18.21 1.67 0.29 18.555 79.66 79.66 0
tokenizecpp 33.2 11.73 31.1 0.01 1.619 76.56 76.57 0.01
checkstlcpp 9.82 12.51 0.12 0.12 53.25 47.02 47.02 0

On Sun, Oct 25, 2026 at 3:33 PM john borland amishscientist@users.sourceforge.net wrote:

Ok I just noticed some of the dates in those PR files I got the packages
from.

I will remove all the PR files from before Oct 6. Given that is about half
of them I will probably add a few more and rerun my code. Hopefully I will
see better coverage numbers on the low end.

What ever happened to http://cppcheck1.osuosl.org:8000/?

When I try to go to the link it says the website can't be reached. I was
thinking of pulling the package list from there but the site seems down. I
was hoping to also get the list of packages that crash or hang up cop
check. That way I could have my script avoid them.

On Thu, Oct 22, 2026, 4:21 AM CHR chrchr@users.sourceforge.net wrote:

It looks like merged PRs are not flushed out, e.g.
https://github.com/cppcheck-opensource/cppcheck/pull/8934

---

Testing PRs on Debian

https://sourceforge.net/p/cppcheck/discussion/development/thread/1cf7f0973e/?limit=25#9c1d/af4c

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/cppcheck/discussion/development/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

---

Testing PRs on Debian
https://sourceforge.net/p/cppcheck/discussion/development/thread/1cf7f0973e/?limit=25#9c1d/af4c/680e

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/cppcheck/discussion/development/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

[john borland]

One thing I recomend when it comes to AI is to keep internal metrics so that you can better help you evluete AI future solutions. That includes keeping track of the work done by AI which you can put a tag in an issue or a tag in the git commit message. You will want to be able to match that tag to a prompt, token usage and model version number. That way if you want to make changes later to your AI process you can test it aginst past solutions to see if the change adds value.

On a side note one thing I never got around to was trying to come up with better usage metrics for valueflow. While I was mainly looking for perfomance use case issues. I think there might be some value in value flow code coverage vs the debian packages. I know GCC 14 added a new feature called MCDC coverage -fcondition-coverage -fprofile-arcs -ftest-coverage It might make since to run this vs the debian packages maybe once a release because we might find a better core set of packages to run So instead of 100 and 1000 you could 70 and 850. You might find some long run time pages are just using the same paths in the value flow vs a lot of diffrent paths. I think as an excersize I can run cppcheck with the GCC MCDC vs serval debian packages and post my finding and output in a new thread if it provides anything useful.

[Daniel Marjamäki]

One thing I recomend when it comes to AI is to keep internal metrics so that you can better help you evluete AI future solutions

how do you suggest that I do that?

[john borland]

I think one thing that matters a lot is how easy is it for you to integrate into your workflow. For me given that I learned to code inside a unix terminal and never evolved to using an IDE to code with AI isnt to difficult to record my interactions with. The script command https://www.redhat.com/en/blog/linux-script-command does alot of the heavy lifting. I also write a bit of data like starting and stopping git hash as well as the ai model version and token usage to a file. That way I can restore the state and test with a different model. In my mind I think this is useful to see which kind of AI workloads that can be shifted to slower local AI models and save tokens for fast interactive sessions when I am working directly with AI. For the most part I am only tracking my use of AI. I have given some thought to how I would do it from a software project level by using a git submodule. Lets take cppcheck as an example. I would have my AI meta data repo with cppcheck as a submodule to that repo. I even think some pre commit hooks could be used to help automate some data collection in the AI meta data repo. Also this helps with comparisons because cppcheck lives outside the repo. You need to do a 3 way diff to evaluate different AI models. To me I like all the AI meta data out side of the main project repo that way it doesn't pollute the main project repo. I could probably throw something together if you are intereste, but ever since vscode has come out as a popular IDE I see fewer people interested in terminal based solutions.

[john borland]

Depending on how many tokens you want to use here are other things you can ask AI about a PR.

• Will this PR require the cppcheck manual to change?
  • if yes which sections.
• What unit test cover these changes?
  • if no unit test is found can please write one.
• What do you think the intent of this PR is?

[john borland]

I wrote a couple of bash scripts to run over the weekend to do code coverage of the released 2.22.0 version of cppcheck with debian packages (It's still running). I wasn't sure how you knows a package had C/C++ code in them, so I just pulled from the following 9 PR's below and hopped for the best.

1. 2026-02-18_12-13-45-d234d745fefe-results.txt
2. 2026-10-05_07-31-15-7f232989d1f9-results.txt
3. 2026-10-09_20-01-23-da724f2e5700-results.txt
4. 2026-09-08_13-51-12-2a0c507aec8b-results.txt
5. 2026-10-05_14-59-08-c5bef2ae24cd-results.txt
6. 2026-10-10_20-28-58-72fa07506cc7-results.txt
7. 2026-10-02_13-25-25-9781751b1038-results.txt
8. 2026-10-05_22-41-06-51b980ae69f3-results.txt
9. 2026-10-12_10-41-10-ee06c4593b71-results.txt

That ended up being 4880 uniq packages. My scripts has currently run cppcheck with the folling arguments check=all -D__GNUC__ on 4260 the debian packages.

Here is the Top 75 packages for coverage.

 1  35.74% ./dart-sdk_3.13.4+dfsg1.orig.gcov.output.out.out
 2  34.70% ./wpewebkit_2.54.1.orig.gcov.output.out.out
 3  34.64% ./mariadb-10.5_10.5.23.orig.gcov.output.out.out
 4  34.55% ./kylin-ai-vector-engine_1.2.0.1.orig.gcov.output.out.out
 5  34.47% ./mozjs128_128.14.0.orig.gcov.output.out.out
 6  34.39% ./mozjs78_78.15.0.orig.gcov.output.out.out
 7  34.36% ./mesa_26.2.4.orig.gcov.output.out.out
 8  34.24% ./gromacs_2026.4.orig.gcov.output.out.out
 9  34.21% ./mariadb_11.8.9+ds.orig.gcov.output.out.out
10  34.15% ./tensorflow_2.14.1+dfsg.orig.gcov.output.out.out
11  34.07% ./ardour_9.8.0+ds.orig.gcov.output.out.out
12  34.04% ./audacity_3.7.8+dfsg.orig.gcov.output.out.out
13  34.02% ./cloudcompare_2.13.2+git20240821+ds.orig.gcov.output.out.out
14  33.94% ./nodejs_24.21.0+dfsg+~cs24.13.4.orig.gcov.output.out.out
15  33.93% ./scummvm_2026.1.0+dfsg.orig.gcov.output.out.out
16  33.90% ./trafficserver_9.2.5+ds.orig.gcov.output.out.out
17  33.89% ./lnav_0.13.2.orig.gcov.output.out.out
18  33.77% ./openjfx_21.0.12+3.orig.gcov.output.out.out
19  33.69% ./android-platform-frameworks-native_10.0.0+r36.orig.gcov.output.out.out
20  33.48% ./pytorch_2.12.1+dfsg.orig.gcov.output.out.out
21  33.48% ./msc-generator_8.6.4+dfsg.orig.gcov.output.out.out
22  33.46% ./qt6-3d_6.11.2+dfsg.orig.gcov.output.out.out
23  33.42% ./rspamd_4.1.5.orig.gcov.output.out.out
24  33.41% ./cppcheck_2.22.0.orig.gcov.output.out.out
25  33.39% ./openmsx_21.0+dfsg.orig.gcov.output.out.out
26  33.27% ./onednn_3.12.1+ds.orig.gcov.output.out.out
27  33.24% ./gargoyle-free_2026.1.1+dfsg.orig.gcov.output.out.out
28  33.24% ./0ad_0.28.0.orig.gcov.output.out.out
29  33.22% ./asymptote_3.15+ds.orig.gcov.output.out.out
30  33.21% ./ocp_3.5.0+ds.orig.gcov.output.out.out
31  33.18% ./tulip_6.0.1+dfsg.orig.gcov.output.out.out
32  33.17% ./coot_1.1.18+dfsg.orig.gcov.output.out.out
33  33.13% ./supercollider_3.14.1+repack2.orig.gcov.output.out.out
34  33.12% ./mrpt_2.15.14+ds.orig.gcov.output.out.out
35  33.10% ./indi_2.2.5+dfsg.orig.gcov.output.out.out
36  33.09% ./openmw_0.51.0.orig.gcov.output.out.out
37  33.08% ./horizon-eda_2.6.0.orig.gcov.output.out.out
38  33.08% ./crawl_0.34.1.orig.gcov.output.out.out
39  33.06% ./monero_0.18.5.3+~0+20260428.orig.gcov.output.out.out
40  33.03% ./openfoam_1912.200626.orig.gcov.output.out.out
41  32.96% ./android-cuttlefish_1.55.4+dfsg.1.orig.gcov.output.out.out
42  32.94% ./scilab_2024.1.0+dfsg1.orig.gcov.output.out.out
43  32.90% ./ltt-control_2.16.0.orig.gcov.output.out.out
44  32.85% ./gnudatalanguage_1.1.3.orig.gcov.output.out.out
45  32.83% ./pymol_3.1.0+dfsg.orig.gcov.output.out.out
46  32.83% ./netradiant_1.5.0+git20260429+dfsg.orig.gcov.output.out.out
47  32.81% ./digikam_9.1.0.orig.gcov.output.out.out
48  32.80% ./ucx_1.22.0.orig.gcov.output.out.out
49  32.80% ./dawn_0.0~git20260909.1a75cb7f.orig.gcov.output.out.out
50  32.75% ./grpc_1.83.0.orig.gcov.output.out.out
51  32.74% ./mmseqs2_18-8cc5c+ds.orig.gcov.output.out.out
52  32.66% ./ares_148+dfsg.orig.gcov.output.out.out
53  32.62% ./bitcoin-knots_29.4.knots20260508.2.orig.gcov.output.out.out
54  32.61% ./stella_7.0+dfsg.orig.gcov.output.out.out
55  32.60% ./ddnet_20.0+ds.orig.gcov.output.out.out
56  32.58% ./ugene_52.1+dfsg.orig.gcov.output.out.out
57  32.58% ./protobuf_36.2.orig.gcov.output.out.out
58  32.54% ./librepcb_1.2.0+dfsg.orig.gcov.output.out.out
59  32.53% ./krita_6.0.4.1+dfsg.orig.gcov.output.out.out
60  32.51% ./autofdo_0.19.orig.gcov.output.out.out
61  32.50% ./telegram-desktop_5.7.2+ds.orig.gcov.output.out.out
62  32.48% ./scribus_1.6.6+dfsg.orig.gcov.output.out.out
63  32.48% ./darkradiant_3.9.0.orig.gcov.output.out.out
64  32.46% ./poppler_26.07.0.orig.gcov.output.out.out
65  32.46% ./courier_2.0.6.orig.gcov.output.out.out
66  32.46% ./bpfcc_0.37.0+ds.orig.gcov.output.out.out
67  32.45% ./mkvtoolnix_102.0.orig.gcov.output.out.out
68  32.41% ./thorvg_1.0.6+dfsg.orig.gcov.output.out.out
69  32.41% ./aqsis_1.8.2.orig.gcov.output.out.out
70  32.39% ./ucc_1.9.0.orig.gcov.output.out.out
71  32.38% ./siril_1.4.2.orig.gcov.output.out.out
72  32.36% ./simutrans_124.5.orig.gcov.output.out.out
73  32.33% ./thrift_0.24.0.orig.gcov.output.out.out
74  32.33% ./casadi_3.8.1+ds2.orig.gcov.output.out.out
75  32.32% ./vulkan-validationlayers_1.4.363.0.orig.gcov.output.out.out

Here is the bottom 25 packages for coverage. Given how some of these things are named like Golang for example it might not have a lot of C/C++ code in them.

 1  3.64% ./golang-github-petermattis-goid_0.0~git20260226.17d1149.orig.gcov.output.out.out
 2  4.10% ./biococoa_2.2.2.orig.gcov.output.out.out
 3  4.10% ./crazy-complete_0.3.9.orig.gcov.output.out.out
 4  4.10% ./luacheck_1.2.0.orig.gcov.output.out.out
 5  4.12% ./bettercap-caplets_0+git20240106.orig.gcov.output.out.out
 6  4.17% ./rust-cc_1.2.60.orig.gcov.output.out.out
 7  4.19% ./golang-github-hillu-go-yara_4.3.4.orig.gcov.output.out.out
 8  4.20% ./libwx-glcanvas-perl_0.09.orig.gcov.output.out.out
 9  4.20% ./php-pspell_1.0.1.orig.gcov.output.out.out
10  4.21% ./php-uuid_1.3.0.orig.gcov.output.out.out
11  4.30% ./cstream_4.0.0.orig.gcov.output.out.out
12  4.31% ./cccolutils_1.5.orig.gcov.output.out.out
13  4.31% ./cd-discid_1.4.orig.gcov.output.out.out
14  4.31% ./credcheck_5.0.orig.gcov.output.out.out
15  4.31% ./gkrellshoot_0.4.4.orig.gcov.output.out.out
16  4.31% ./haskell-openglraw_3.3.4.1.orig.gcov.output.out.out
17  4.31% ./libapache2-mod-bw_0.92.orig.gcov.output.out.out
18  4.31% ./libfuture-xs-perl_0.15.orig.gcov.output.out.out
19  4.31% ./purple-rocketchat_0.1~hg20200403.800ef89.orig.gcov.output.out.out
20  4.31% ./sagenb-export_3.4.orig.gcov.output.out.out
21  4.31% ./uim-chewing_0.1.0.orig.gcov.output.out.out
22  4.32% ./autossh_1.4g.orig.gcov.output.out.out
23  4.32% ./gperiodic_3.0.3.orig.gcov.output.out.out
24  4.32% ./indi-webcam_1.1+20260316195637.orig.gcov.output.out.out
25  4.32% ./libapache2-mod-authnz-external_3.3.3.orig.gcov.output.out.out

I know in the comments above you talked about doing more packages based on changes to tokenizer or valueflow.

Here are the top 50 for valueflow.cpp

 1  88.37% ./dart-sdk_3.13.4+dfsg1.orig.gcov.output.out.out
 2  86.52% ./kylin-ai-vector-engine_1.2.0.1.orig.gcov.output.out.out
 3  85.82% ./audacity_3.7.8+dfsg.orig.gcov.output.out.out
 4  85.71% ./nodejs_24.21.0+dfsg+~cs24.13.4.orig.gcov.output.out.out
 5  85.68% ./wpewebkit_2.54.1.orig.gcov.output.out.out
 6  85.30% ./tensorflow_2.14.1+dfsg.orig.gcov.output.out.out
 7  85.10% ./ardour_9.8.0+ds.orig.gcov.output.out.out
 8  84.94% ./lnav_0.13.2.orig.gcov.output.out.out
 9  84.90% ./mesa_26.2.4.orig.gcov.output.out.out
10  84.85% ./mariadb-10.5_10.5.23.orig.gcov.output.out.out
11  84.84% ./telegram-desktop_5.7.2+ds.orig.gcov.output.out.out
12  84.72% ./gromacs_2026.4.orig.gcov.output.out.out
13  84.30% ./qt6-3d_6.11.2+dfsg.orig.gcov.output.out.out
14  84.21% ./android-platform-frameworks-native_10.0.0+r36.orig.gcov.output.out.out
15  84.05% ./mozjs128_128.14.0.orig.gcov.output.out.out
16  84.05% ./mariadb_11.8.9+ds.orig.gcov.output.out.out
17  83.75% ./mozjs78_78.15.0.orig.gcov.output.out.out
18  83.70% ./msc-generator_8.6.4+dfsg.orig.gcov.output.out.out
19  83.70% ./horizon-eda_2.6.0.orig.gcov.output.out.out
20  83.69% ./pytorch_2.12.1+dfsg.orig.gcov.output.out.out
21  83.56% ./cloudcompare_2.13.2+git20240821+ds.orig.gcov.output.out.out
22  83.33% ./rspamd_4.1.5.orig.gcov.output.out.out
23  83.29% ./coot_1.1.18+dfsg.orig.gcov.output.out.out
24  83.28% ./onednn_3.12.1+ds.orig.gcov.output.out.out
25  83.28% ./monero_0.18.5.3+~0+20260428.orig.gcov.output.out.out
26  83.26% ./ltt-control_2.16.0.orig.gcov.output.out.out
27  83.26% ./cppcheck_2.22.0.orig.gcov.output.out.out
28  83.23% ./crawl_0.34.1.orig.gcov.output.out.out
29  83.21% ./openmsx_21.0+dfsg.orig.gcov.output.out.out
30  83.19% ./0ad_0.28.0.orig.gcov.output.out.out
31  83.09% ./perfetto_56.0.orig.gcov.output.out.out
32  83.09% ./dawn_0.0~git20260909.1a75cb7f.orig.gcov.output.out.out
33  82.98% ./asymptote_3.15+ds.orig.gcov.output.out.out
34  82.77% ./android-cuttlefish_1.55.4+dfsg.1.orig.gcov.output.out.out
35  82.68% ./cherrytree_1.2.0+dfsg.orig.gcov.output.out.out
36  82.63% ./bitcoin-knots_29.4.knots20260508.2.orig.gcov.output.out.out
37  82.55% ./tulip_6.0.1+dfsg.orig.gcov.output.out.out
38  82.49% ./trafficserver_9.2.5+ds.orig.gcov.output.out.out
39  82.45% ./openmw_0.51.0.orig.gcov.output.out.out
40  82.45% ./grpc_1.83.0.orig.gcov.output.out.out
41  82.39% ./poppler_26.07.0.orig.gcov.output.out.out
42  82.39% ./highs_1.14.0+ds.orig.gcov.output.out.out
43  82.32% ./indi_2.2.5+dfsg.orig.gcov.output.out.out
44  82.29% ./ddnet_20.0+ds.orig.gcov.output.out.out
45  82.26% ./pymol_3.1.0+dfsg.orig.gcov.output.out.out
46  82.19% ./gargoyle-free_2026.1.1+dfsg.orig.gcov.output.out.out
47  82.17% ./scummvm_2026.1.0+dfsg.orig.gcov.output.out.out
48  82.11% ./ocp_3.5.0+ds.orig.gcov.output.out.out
49  81.94% ./openjfx_21.0.12+3.orig.gcov.output.out.out
50  81.91% ./autofdo_0.19.orig.gcov.output.out.out

Here are the top 50 for tokenize.cpp

 1  74.30% ./dart-sdk_3.13.4+dfsg1.orig.gcov.output.out.out
 2  71.69% ./wpewebkit_2.54.1.orig.gcov.output.out.out
 3  69.96% ./mozjs128_128.14.0.orig.gcov.output.out.out
 4  69.63% ./mozjs78_78.15.0.orig.gcov.output.out.out
 5  69.39% ./mesa_26.2.4.orig.gcov.output.out.out
 6  68.64% ./mariadb-10.5_10.5.23.orig.gcov.output.out.out
 7  67.79% ./kylin-ai-vector-engine_1.2.0.1.orig.gcov.output.out.out
 8  67.75% ./openjfx_21.0.12+3.orig.gcov.output.out.out
 9  67.72% ./gromacs_2026.4.orig.gcov.output.out.out
10  67.56% ./mariadb_11.8.9+ds.orig.gcov.output.out.out
11  67.01% ./trafficserver_9.2.5+ds.orig.gcov.output.out.out
12  66.98% ./cloudcompare_2.13.2+git20240821+ds.orig.gcov.output.out.out
13  66.59% ./ardour_9.8.0+ds.orig.gcov.output.out.out
14  66.46% ./audacity_3.7.8+dfsg.orig.gcov.output.out.out
15  66.19% ./supercollider_3.14.1+repack2.orig.gcov.output.out.out
16  66.15% ./android-platform-frameworks-native_10.0.0+r36.orig.gcov.output.out.out
17  65.99% ./tensorflow_2.14.1+dfsg.orig.gcov.output.out.out
18  65.84% ./nodejs_24.21.0+dfsg+~cs24.13.4.orig.gcov.output.out.out
19  65.71% ./rspamd_4.1.5.orig.gcov.output.out.out
20  65.31% ./lnav_0.13.2.orig.gcov.output.out.out
21  65.13% ./scummvm_2026.1.0+dfsg.orig.gcov.output.out.out
22  64.01% ./msc-generator_8.6.4+dfsg.orig.gcov.output.out.out
23  63.79% ./pytorch_2.12.1+dfsg.orig.gcov.output.out.out
24  63.79% ./asymptote_3.15+ds.orig.gcov.output.out.out
25  63.67% ./qt6-3d_6.11.2+dfsg.orig.gcov.output.out.out
26  63.67% ./monero_0.18.5.3+~0+20260428.orig.gcov.output.out.out
27  63.56% ./protobuf_36.2.orig.gcov.output.out.out
28  63.55% ./ucx_1.22.0.orig.gcov.output.out.out
29  63.49% ./mrpt_2.15.14+ds.orig.gcov.output.out.out
30  63.35% ./ocp_3.5.0+ds.orig.gcov.output.out.out
31  63.18% ./krita_6.0.4.1+dfsg.orig.gcov.output.out.out
32  63.08% ./gargoyle-free_2026.1.1+dfsg.orig.gcov.output.out.out
33  63.01% ./onednn_3.12.1+ds.orig.gcov.output.out.out
34  62.98% ./autofdo_0.19.orig.gcov.output.out.out
35  62.83% ./ugene_52.1+dfsg.orig.gcov.output.out.out
36  62.43% ./dawn_0.0~git20260909.1a75cb7f.orig.gcov.output.out.out
37  62.40% ./indi_2.2.5+dfsg.orig.gcov.output.out.out
38  62.36% ./darkradiant_3.9.0.orig.gcov.output.out.out
39  62.13% ./ltt-control_2.16.0.orig.gcov.output.out.out
40  62.11% ./openmw_0.51.0.orig.gcov.output.out.out
41  62.11% ./cccl_2.5.0.orig.gcov.output.out.out
42  61.99% ./crawl_0.34.1.orig.gcov.output.out.out
43  61.89% ./universal-ctags_6.2.1.orig.gcov.output.out.out
44  61.81% ./grpc_1.83.0.orig.gcov.output.out.out
45  61.77% ./librepcb_1.2.0+dfsg.orig.gcov.output.out.out
46  61.60% ./coot_1.1.18+dfsg.orig.gcov.output.out.out
47  61.58% ./mkvtoolnix_102.0.orig.gcov.output.out.out
48  61.48% ./android-cuttlefish_1.55.4+dfsg.1.orig.gcov.output.out.out
49  61.46% ./qtads_3.4.0+dfsg.orig.gcov.output.out.out
50  61.39% ./numpy_2.4.6+ds.orig.gcov.output.out.out

I still have all the gcov data at the file level if you are intrested.

My idea is pretty simple whenever Cppcheck makes a release (Normally cppcheck does 3 or 4 a year) I could run the coverage data. I have a feeling that structurally these numbers probably don't change too much. I see this having two advantages. One you could add the top 10 to your random list of packages just to make sure you have some with good code coverage. Two you can give developers a script to run against that static 10 locally before submitting their PR. That might reduce the load on your PR system and reduce churn on the PR's themselves.

Let me know if this is of any interest to you.

Thanks