cppcheck with defines not working as expected

[Krishna]

Hi @danielmarjamaki,

We are trying to hide preprocessor code from being checked by cppcheck(As seeing more execution time for cppcheck to expand & validate the preprocessor code). But that is not working as expected. So created a sample code and attached to this thread to demonstrate.

I'm running the below command

cppcheck.exe --enable=warning --enable=style --quiet --language=c --enable=performance --xml --inline-suppr --suppress=preprocessorErrorDirective -iC:\Users\Desktop\Cppcheck\private -DCPPCHECK_TEST .\Cppcheck\

In the attachment there is a main.h which includes the following piece of code

#ifdef CPPCHECK_TEST
#define OVERFLOW    1<<123
#else
#define OVERFLOW    1<<1
#endif

As i'm using the -DCPPCHECK_TEST with cppcheck here i'm expecting it should report OVERFLOW error. But it's not happening.

Same checked with 2.4 & 2.6 versions.

Thanks

[Daniel Marjamäki]

If you don't use the OVERFLOW macro then no warning is written. Cppcheck preprocesses the code.

[Krishna]

Hi @danielmarjamaki,

Thanks for the reply.
But that was using in main.c

int main(void)
{
    int temp = OVERFLOW;
    int status = foo(MACRO_123);
    printf("%d %d", status,temp);
}

[Daniel Marjamäki]

This command is strange:

-IC:\Users\Desktop\Cppcheck\Include\main.h

hmm.. Cppcheck expects that -I points out a include path. I assume Cppcheck would find the bug if the parameter is: -IC:\Users\Desktop\Cppcheck\Include

Maybe we need to adapt Cppcheck to handle this strange iccarm command line parameter better..

[Krishna]

Hi @danielmarjamaki,

I have fixed include path and tried, still not reporting overflow error.

Please find the attachment.

command that I have used -

cppcheck.exe --enable=warning --enable=style --quiet --language=c --enable=performance --xml --inline-suppr --suppress=preprocessorErrorDirective -iD:\Cppcheck\private -DCPPCHECK_TEST D:\Cppcheck\

Thanks

[Daniel Marjamäki]

ok I was confused I thought the json file was imported..

Can you please add a -I so the main.h file is found.

[Krishna]

Hi @danielmarjamaki,

In the json file main.h path was added already.

[
{
  "directory": "D:/Cppcheck",
  "command": "D:\\Gen3\\GEN3_ENV_ROOT\\IAR\\Ver9_10_1\\arm\\bin\\iccarm.exe  --silent D:\Cppcheck\src\main.c -ID:\Cppcheck\Include -ID:\Cppcheck\private",
  "file": "D:\Cppcheck\src\main.c"
}
]

[Daniel Marjamäki]

If you intend to use json file.. remove the D:\Cppcheck from the command and add --project=config.json.. something like:

cppcheck.exe --enable=warning --enable=style --quiet --language=c --enable=performance --xml --inline-suppr --suppress=preprocessorErrorDirective -iD:\Cppcheck\private -DCPPCHECK_TEST --project=config.json

[Krishna]

Hi @danielmarjamaki,

With the --project option it is reporting an error. Actually, we are trying to hide a group of macros that are defined in some header files and these macros included in all source files. These macros get assigned with a unique constant value at the preprocessing stage. We observed these header files causing more time execution for cppcheck. But the above hiding a piece of code is not working for us(You can see our conversation here - https://sourceforge.net/p/cppcheck/discussion/general/thread/83bb029837/?limit=25&page=1#174f).

Please clarify below points:
1. Is it because of preprocessing stage doing by cppcheck?
2. Is there a way to know preprocessing execution time for every file?

Thanks.

[Daniel Marjamäki]

I feel that it's unlikely that it's the preprocessor itself that takes much time.

If you run like this:

./cppcheck --showtime=summary .. file

then cppcheck will print some timing statistics. I believe the preprocessing is covered by item "Tokenizer::createTokens". If you look at the bottom is says "Overall time:".

[Krishna]

Hi @danielmarjamaki,

Here is the overall summary of our code with cppcheck

Tokenizer::simplifyTokens1: 796.793s (avg. 3.3062s - 241 result(s))
Tokenizer::simplifyTokens1::ValueFlow: 264.826s (avg. 1.09886s - 241 result(s))
Tokenizer::createTokens: 183.26s (avg. 0.704846s - 260 result(s))
Tokenizer::tokenize::simplifyTypedef: 104.468s (avg. 0.433477s - 241 result(s))
Tokenizer::tokenize::setVarId: 99.194s (avg. 0.411593s - 241 result(s))
Tokenizer::simplifyTokens1::createSymbolDatabase: 55.909s (avg. 0.231988s - 241 result(s))
Tokenizer::simplifyTokens1::createAst: 41.21s (avg. 0.170996s - 241 result(s))
Tokenizer::simplifyTokens1::setValueType: 26.162s (avg. 0.108556s - 241 result(s))
Tokenizer::tokenize::findGarbageCode: 19.049s (avg. 0.0790415s - 241 result(s))
Other::runChecks: 16.637s (avg. 0.0690332s - 241 result(s))
Type::runChecks: 11.019s (avg. 0.045722s - 241 result(s))
Sizeof::runChecks: 8.127s (avg. 0.033722s - 241 result(s))
Condition::runChecks: 8.112s (avg. 0.0336598s - 241 result(s))
Null pointer::runChecks: 4.762s (avg. 0.0197593s - 241 result(s))
Bounds checking::runChecks: 3.987s (avg. 0.0165436s - 241 result(s))
String::runChecks: 2.441s (avg. 0.0101286s - 241 result(s))
Uninitialized variables::runChecks: 2.092s (avg. 0.0086805s - 241 result(s))
UnusedVar::runChecks: 1.971s (avg. 0.00817842s - 241 result(s))
Assert::runChecks: 1.812s (avg. 0.00751867s - 241 result(s))
IO using format string::runChecks: 0.268s (avg. 0.00111203s - 241 result(s))
Leaks (auto variables)::runChecks: 0.202s (avg. 0.000838174s - 241 result(s))
Check function usage::runChecks: 0.19s (avg. 0.000788382s - 241 result(s))
Boolean::runChecks: 0.157s (avg. 0.000651452s - 241 result(s))
Auto Variables::runChecks: 0.136s (avg. 0.000564315s - 241 result(s))
Memory leaks (address not taken)::runChecks: 0.134s (avg. 0.000556017s - 241 result(s))
Vaarg::runChecks: 0.086s (avg. 0.000356846s - 241 result(s))
Memory leaks (struct members)::runChecks: 0.053s (avg. 0.000219917s - 241 result(s))
Memory leaks (function variables)::runChecks: 0.035s (avg. 0.000145228s - 241 result(s))
64-bit portability::runChecks: 0.026s (avg. 0.000107884s - 241 result(s))
Exception Safety::runChecks: 0.001s (avg. 4.14938e-06s - 241 result(s))
Class::runChecks: 0s (avg. 0s - 241 result(s))
Boost usage::runChecks: 0s (avg. 0s - 241 result(s))
Memory leaks (class variables)::runChecks: 0s (avg. 0s - 241 result(s))
Unused functions::runChecks: 0s (avg. 0s - 241 result(s))
Using postfix operators::runChecks: 0s (avg. 0s - 241 result(s))
STL usage::runChecks: 0s (avg. 0s - 241 result(s))
Overall time: 1653.12s

It is evident that Tokenizer::simplifyTokens1 taking 13 mins approximately.

However if I remove the 2 included paths from database file path(-I based) which have defined many macros, following is the overall cppcheck summary

Tokenizer::simplifyTokens1: 234.397s (avg. 0.972602s - 241 result(s))
Tokenizer::createTokens: 107.04s (avg. 0.411692s - 260 result(s))
Tokenizer::tokenize::simplifyTypedef: 94.575s (avg. 0.392427s - 241 result(s))
Tokenizer::simplifyTokens1::createAst: 15.956s (avg. 0.0662075s - 241 result(s))
Tokenizer::tokenize::setVarId: 9.962s (avg. 0.0413361s - 241 result(s))
Tokenizer::tokenize::findGarbageCode: 9.553s (avg. 0.039639s - 241 result(s))
Tokenizer::simplifyTokens1::createSymbolDatabase: 0.162s (avg. 0.000672199s - 241 result(s))
Tokenizer::simplifyTokens1::ValueFlow: 0.123s (avg. 0.041s - 3 result(s))
Uninitialized variables::runChecks: 0.012s (avg. 0.004s - 3 result(s))
Other::runChecks: 0.003s (avg. 0.001s - 3 result(s))
Tokenizer::simplifyTokens1::setValueType: 0.003s (avg. 0.001s - 3 result(s))
UnusedVar::runChecks: 0.002s (avg. 0.000666667s - 3 result(s))
IO using format string::runChecks: 0.001s (avg. 0.000333333s - 3 result(s))
Auto Variables::runChecks: 0.001s (avg. 0.000333333s - 3 result(s))
Memory leaks (address not taken)::runChecks: 0.001s (avg. 0.000333333s - 3 result(s))
Memory leaks (function variables)::runChecks: 0s (avg. 0s - 3 result(s))
Memory leaks (struct members)::runChecks: 0s (avg. 0s - 3 result(s))
Null pointer::runChecks: 0s (avg. 0s - 3 result(s))
STL usage::runChecks: 0s (avg. 0s - 3 result(s))
Sizeof::runChecks: 0s (avg. 0s - 3 result(s))
String::runChecks: 0s (avg. 0s - 3 result(s))
64-bit portability::runChecks: 0s (avg. 0s - 3 result(s))
Assert::runChecks: 0s (avg. 0s - 3 result(s))
Boolean::runChecks: 0s (avg. 0s - 3 result(s))
Boost usage::runChecks: 0s (avg. 0s - 3 result(s))
Bounds checking::runChecks: 0s (avg. 0s - 3 result(s))
Check function usage::runChecks: 0s (avg. 0s - 3 result(s))
Class::runChecks: 0s (avg. 0s - 3 result(s))
Condition::runChecks: 0s (avg. 0s - 3 result(s))
Exception Safety::runChecks: 0s (avg. 0s - 3 result(s))
Type::runChecks: 0s (avg. 0s - 3 result(s))
Leaks (auto variables)::runChecks: 0s (avg. 0s - 3 result(s))
Unused functions::runChecks: 0s (avg. 0s - 3 result(s))
Memory leaks (class variables)::runChecks: 0s (avg. 0s - 3 result(s))
Using postfix operators::runChecks: 0s (avg. 0s - 3 result(s))
Vaarg::runChecks: 0s (avg. 0s - 3 result(s))
Overall time: 471.791s

From above data, It is evident that removing those included paths from database file is saving so much time.

Another surprise is if I run with --file-filter for a single source file the overall time it is showing negligible and for the same file it is taking 9 sec if I run with complete database file.

Removing a included paths from database file is not reporting some of the real errors and hiding those macros using -D not saving time.

Any suggestion is helpful.

Thanks

[Krishna]

Hi @danielmarjamaki,

PFA for overall summary when run cppcheck passing database file as it is and when removing 2 included paths(-I) which havig the bunch of macros defined header files.

Observed Tokenizer::simplifyTokens1 is taking 13 mins in first case. What stage it is? Can be able to optimize this?

-D method is not saving the time and removing the included paths from database file is not reporting the some actual cppcheck errors

Looking forward for optimized solution by analysing the attached timing summary.

Thanks.

[Daniel Marjamäki]

Observed Tokenizer::simplifyTokens1 is taking 13 mins in first case. What stage it is? Can be able to optimize this?

Feel free to profile.

The Tokenizer::simplifyTokens1 is one of the most heavy functions in Cppcheck. It is known that most of the time is spent in that.

its the whole conversion from "raw tokens from preprocessor" to "full cppcheck data" that happens in that stage.

During Tokenizer::simplifyTokens1 it runs valueflow and simplifytypedef and setvarid and creates symboldatabase and ast.. and more.. so more than half of that 796s is accounted for in the other items.. but there are ~200s if I count correctly that are not accounted for and I am not sure what it might do during those 200s. There are various other transformations.