Cppcheck-2.6

[Daniel Marjamäki]

I believe it would be good to more or less have feature freeze from now on. So we can release cppcheck-2.6 later.

If you see "low hanging fruit" tasks; tasks that are small, simple, riskfree and high ROI, feel free to work on that.

[Paul Fultz]

I think it would be good to merge PR 3410, as this will fix some FPs.

[CHR]

Last time. there was a release candidate for 2.5. Unfortunately, a number of regressions slipped in after the RC was announced. Maybe this can be avoided this time around.

[Daniel Marjamäki]

yes I'd hope we can create a RC in ~ 2-4 weeks when most of the crashes have been fixed. And when we have that, maybe it's possible to be more strict about merging PRs.. I will try.

[Richard Smith]

HEAD has the following exception on our codebase. Note I have to edit the path to prevent spambot from rejecting my post.

~~~ "Cppcheck addons misra.py", line 2077, in misra_10_1
e1_et = getEssentialType(token.astOperand1).split(' ')[-1]
AttributeError: 'NoneType' object has no attribute 'split'

~~~

Unhandled NoneTypes seem to be a very common exception. It has occurred on every one of my upgrade attempts. Seems to me that handling this case needs improvement. At minimum it should produce more information on what the tokens are so I can track down where this happening.

[Daniel Marjamäki]

Thanks! This bug should be fixed.

[Daniel Marjamäki]

To clarify; I fixed it with https://github.com/danmar/cppcheck/commit/43fa7d2ebe05595cfc045eaaea5e15bdc9347678 please continue your testing.

[Richard Smith]

I can confirm it is fixed. Thank you.

With all the new rules come a lot of new messages that I need to look into. I'll follow up with a dedicated thread if I find anything else.

[Absol]

Static analyzer by Microsoft wrote false positive about simple.cpp or cppcheck can't detect problems ?

target : cppcheck-main\externals\simplecpp\simplecpp.cpp
cppcheck output:

Checking simplecpp.cpp ...
simplecpp.h:164:9: error: syntax error [syntaxError]
        enum Type {
        ^
Checking simplecpp.cpp: SIMPLECPP_EXPORT;_WIN32...
Checking simplecpp.cpp: SIMPLECPP_IMPORT;_WIN32...
Checking simplecpp.cpp: _WIN32...
Checking simplecpp.cpp: _WIN32;__CYGWIN__;__MINGW32__...
Checking simplecpp.cpp: __CYGWIN__...

microsoft check output:

simplecpp.h (163): warning C26812: The enumeration type "simplecpp :: Output :: Type" is out of scope. Try to use "enum class" instead of "enum" (Enum.3).
simplecpp.cpp (224): warning C26439: This function cannot "throw". Declare it as "noexcept" (f.6).
simplecpp.cpp (247): warning C26439: This function cannot "throw". Declare it as "noexcept" (f.6).
simplecpp.cpp (1416): warning C28182: NULL pointer dereference. "argtok" contains the same NULL value found in "argtok-> next".
simplecpp.cpp (1444): warning C6011: Dereference of null "endToken".
simplecpp.cpp (1758): warning C6011: Dereferencing the null pointer "tok3".
simplecpp.cpp (1761): warning C6011: Dereferencing the null pointer "tok2".
simplecpp.cpp (1778): warning C28182: NULL pointer dereference. "lastToken" contains the same NULL value as in "defToken-> next". For the earliest location where this could have happened, see line 1758
simplecpp.cpp (1795): warning C26451: Arithmetic overflow: using the "+" operator on byte value 4 and casting the result to byte value 8. Cast the value to a wider type before calling the "+" operator to avoid overflow (io.2 ).
simplecpp.cpp (1798): warning C26451: Arithmetic overflow: using the "+" operator on byte value 4 and casting the result to byte value 8. Cast the value to a wider type before calling the "+" operator to avoid overflow (io.2 ).
simplecpp.cpp (1810): warning C26451: Arithmetic overflow: using the "+" operator on byte value 4 and casting the result to byte value 8. Cast the value to a wider type before calling the "+" operator to avoid overflow (io.2 ).
simplecpp.cpp (1812): warning C26451: Arithmetic overflow: using the "+" operator on byte value 4 and casting the result to byte value 8. Cast the value to a wider type before calling the "+" operator to avoid overflow (io.2 ).
simplecpp.cpp (2498): warning C26451: Arithmetic overflow: using the "-" operator on byte value 4 and casting the result to byte value 8. Cast the value to a wider type before calling the "-" operator to avoid overflow (io.2 ).
simplecpp.cpp (2498): warning C26451: Arithmetic overflow: using the "<<" operator on byte value 4 and casting the result to byte value 8. Cast the value to a wider type before calling the "<<" operator to avoid overflow (io .2).
simplecpp.cpp (2748): warning C6011: Dereferencing a rawtok null pointer.
simplecpp.cpp (3153): warning C26812: The enumeration type "simplecpp :: preprocess :: __ l17 :: IfState" is out of scope. Try to use "enum class" instead of "enum" (Enum.3).

[Absol]

Also I found infinite loop with cppcheck:

c:\Users\Admin\Downloads\cppcheck-main (1)\cppcheck-main\bin\debug>cppcheck.exe "C:\Users\Admin\Downloads\cppcheck-main (1)\cppcheck-main\lib\astutils.cpp"
Checking C:\Users\Admin\Downloads\cppcheck-main (1)\cppcheck-main\lib\astutils.cpp ...

I wait more than 5 minutes for file analyzing....And NOTHING!

[Daniel Marjamäki]

I think that cppcheck is getting slower and I also feel currently that it is getting too slow.

However a debug build that does not use the match compiler will be many times slower than a real release build with match compiler.

We have a selfcheck in our CI. We run cppcheck on the cppcheck source code. That step takes 7m45s currently. I would assume it runs a release build with match compiler.

[Absol]

but this is one file, not all cppcheck project.

[Daniel Marjamäki]

yes.

with a proper release build:

$ time ./cppcheck -D__CPPCHECK__ lib/astutils.cpp 
Checking lib/astutils.cpp ...
Checking lib/astutils.cpp: __CPPCHECK__=1...

real    0m5,436s
user    0m5,420s
sys     0m0,016s

with a normal debug build:

$ time ./cppcheck -D__CPPCHECK__ lib/astutils.cpp 
Checking lib/astutils.cpp ...
Checking lib/astutils.cpp: __CPPCHECK__=1...

real    1m11,583s
user    1m11,565s
sys     0m0,016s

And I run in Linux. I believe Windows is slower.

[Absol]

I test with DEBUGx64 build, and this woks bigger than 5 minutes (I close after 5 min) at PC with ryzen7 win10

[Daniel Marjamäki]

I think it's not very interesting to discuss performance of a debug build without matchcompiler. that is definitely slow.

Do you volounteer to test if the release binary is that slow also?

you can download a nightly build from here:
https://github.com/danmar/cppcheck/actions/workflows/release-windows.yml

The "deploy" artifact contains the files. I believe you can just extract that somewhere on your computer and run.

The "installer" artifact contains an msi installer.

ryzen7

my computer is no real high end computer it's ~3 years old by now.

[Daniel Marjamäki]

Can you test this in a visual studio command prompt..

cd cppcheck
mkdir build
cd build
cmake MATCHCOMPILER=YES -DCMAKE_BUILD_TYPE=Release ..
nmake
bin\cppcheck.exe -D__CPPCHECK__ ..\lib\astutils.cpp

[Daniel Marjamäki]

on my old windows laptop (~8 years old). when I build cppcheck with the above cmake+nmake commands the analysis finish in ~55 seconds. switching to linux then it takes ~6 seconds (make CXXFLAGS=-O2 MATCHCOMPILER=yes). Same computer. Not sure if Windows is really THAT much slower.

[Absol]

I tried run it from artifact and it works without problem.

[Absol]

Strange but now builded DEBUG x64 also works without problem.

[Paul Fultz]

I dont think the matchcompiler is the culprit here. The Match function only takes about 5% of the time. Enabling the match compiler on a debug builds only show a slight improvement:

Match off:
real    1m2.216s
user    1m2.175s
sys     0m0.016s
Match on:
real    0m59.209s
user    0m59.173s
sys     0m0.012s

Running the perf report with the Match compiler show these are the top functions:

   2.42%  cppcheck  cppcheck             [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_data
   2.01%  cppcheck  cppcheck             [.] getTokenArgumentFunction
   1.77%  cppcheck  cppcheck             [.] std::char_traits<char>::length
   1.68%  cppcheck  cppcheck             [.] Token::str[abi:cxx11]
   1.58%  cppcheck  cppcheck             [.] Token::exprId
   1.58%  cppcheck  libstdc++.so.6.0.25  [.] __gnu_debug::_Safe_iterator_base::_M_singular
   1.39%  cppcheck  libstdc++.so.6.0.25  [.] 0x00000000000bce86
   1.36%  cppcheck  cppcheck             [.] MatchCompiler::equalN<2u>
   1.32%  cppcheck  libstdc++.so.6.0.25  [.] __gnu_debug::_Safe_iterator_base::_M_detach
   1.31%  cppcheck  cppcheck             [.] MatchCompiler::makeConstString<2u>
   1.26%  cppcheck  cppcheck             [.] isVariableChangedByFunctionCall
   1.12%  cppcheck  cppcheck             [.] MatchCompiler::operator==<2u>
   1.08%  cppcheck  cppcheck             [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string
   1.05%  cppcheck  cppcheck             [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::size
   0.99%  cppcheck  cppcheck             [.] Token::next
   0.92%  cppcheck  cppcheck             [.] Token::astParent
   0.88%  cppcheck  cppcheck             [.] isExpressionChangedAt<isExpressionChangedAt(Token const*, Token const*, int, bool, Settings const*, bool, int)::{lambda()#1}, void>
   0.78%  cppcheck  cppcheck             [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string
   0.76%  cppcheck  cppcheck             [.] MatchCompiler::ConstString<2u>::ConstString
   0.76%  cppcheck  cppcheck             [.] __gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<Token const**, std::__cxx1998::vector<Token const*, std::allocator<Token cons
   0.68%  cppcheck  cppcheck             [.] __gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<Token const**, std::__cxx1998::vector<Token const*, std::allocator<Token cons
   0.67%  cppcheck  cppcheck             [.] Token::isUnaryOp

Running with -O2 shows these functions:

   6.70%  cppcheck  libstdc++.so.6.0.25  [.] 0x00000000000bce86
   6.61%  cppcheck  libstdc++.so.6.0.25  [.] __gnu_debug::_Safe_iterator_base::_M_singular
   6.16%  cppcheck  libstdc++.so.6.0.25  [.] __gnu_debug::_Safe_iterator_base::_M_detach
   4.67%  cppcheck  cppcheck             [.] getTokenArgumentFunction
   3.30%  cppcheck  cppcheck             [.] findExpression
   2.90%  cppcheck  libstdc++.so.6.0.25  [.] 0x00000000000bce69
   2.60%  cppcheck  cppcheck             [.] isVariableChangedByFunctionCall
   2.42%  cppcheck  libc-2.27.so         [.] cfree@GLIBC_2.2.5
   2.15%  cppcheck  cppcheck             [.] isExpressionChangedAt<isExpressionChangedAt(Token const*, Token const*, int, bool, Settings const*, bool, int)::{lambda()#1}, void>
   2.06%  cppcheck  libc-2.27.so         [.] malloc
   2.01%  cppcheck  libc-2.27.so         [.] __strchr_sse2
   1.88%  cppcheck  libstdc++.so.6.0.25  [.] __gnu_debug::_Safe_iterator_base::_M_attach
   1.88%  cppcheck  cppcheck             [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>
   1.67%  cppcheck  cppcheck             [.] __gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<Token const**, std::__cxx1998::vector<Token const*, std::allocator<Token cons
   1.66%  cppcheck  cppcheck             [.] std::_Function_handler<bool (Token const*), isExpressionChanged(Token const*, Token const*, Token const*, Settings const*, bool, int):
   1.53%  cppcheck  libstdc++.so.6.0.25  [.] 0x00000000000bce60
   1.45%  cppcheck  libstdc++.so.6.0.25  [.] __gnu_debug::_Safe_iterator_base::_M_can_compare
   1.28%  cppcheck  libstdc++.so.6.0.25  [.] 0x00000000000ba201
   1.27%  cppcheck  cppcheck             [.] std::__find_if<__gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<Token const**, std::__cxx1998::vector<Token const*, std::alloc
   1.26%  cppcheck  libstdc++.so.6.0.25  [.] 0x00000000000ba266
   1.25%  cppcheck  cppcheck             [.] Library::getFunctionName[abi:cxx11]
   1.17%  cppcheck  cppcheck             [.] std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<
   1.05%  cppcheck  cppcheck             [.] astFlattenRecursive

We should try to look at improving the performance of isVariableChangedByFunctionCall or getTokenArgumentFunction functions.

Also, the -O2 shows findExpression, which this could be reduced by caching the expression token in removeModifiedVars.

[Paul Fultz]

One thing we could do is check each token if its being changed, by function call, or inconclusive, and then store it in the token class. We can store two bits, one for indirect=0 and another indirect=1. And then pack each bit pairs into a single integer.

This way we can see if its modified faster during analysis. It also would enable the addons to easily check if a variable or expression is modified.

[Daniel Marjamäki]

That sounds good to me.

[john borland]

Has anyone tried making a flame graph https://www.brendangregg.com/flamegraphs.html
In some of my performance critical code at work I put them in our CI system. They are pretty easy to look at. Also there is a diff script for them so I have it diff the flamegraph with the last tagged release.

[Daniel Marjamäki]

sounds interesting. I had not even heard about flame graphs before.

[john borland]

This is a flame graph of cppcheck head builded unoptimized with match compiler on. I ran it on cppcheck/lib/ with check=all