3 minute run in 2.4 EL8 still running 20 hours later with 2.16.0

[Andrew Brennan]

As explained in the subject, I built 2.16.0 from source yesterday to check and see if the new version would work better for our RHEL 8 build node. The codebase I'm running on is fairly large (~ 1 million LOC), but with 2.4 it took minutes to run. When I ran it against 2.16.0, with -j32, it's still running 20+ hours later.

I did a similar update on RHEL 7, though not as big of an update jump, I went from 2.7.1 to 2.7.4 and my 1/2 run took 5-1/2 hours.

I've also considered, but have yet to explore, if the configurations in /usr/share/cppcheck/cfg are the driving force more than the new version itself.

UPDATE
The issue was a 100-line if(stringa="val"|| stringb="valb") that commenting out reduced the run time for the bottleneck file from 54 hours to 5 minutes.

[CHR]

Performance regressions do happen: https://trac.cppcheck.net/query?status=!closed&component=Performance
It would help if you could identify the file(s) that cause the hang and maybe also post a code example.

[Daniel Marjamäki]

ouch.

I assume you built with MATCHCOMPILER and some optimisation flags.

The option --showtime=file-total might be interesting that shows the time it takes to process each file.

[Daniel Marjamäki]

some possible suggestions :

• if you build latest git head then you can also try --check-level=reduced and see if that has a positive effect.

• if you run with --report-progress then cppcheck is supposed to show every ~ 10 seconds what it is working on. I guess this works best with -j1. You can start Cppcheck and 1-2 hours later look at the status from cppcheck where it is .. it's possible it is working hard on something then.

[Andrew Brennan]

I'm going to run with showtime=file-total and some other debug flags on and get back with some better debugging info in a few days.

[Andrew Brennan]

valueFlowCondition(SimpleConditionHandler{}, tokenlist, symboldatabase, errorLogger, settings, skipped functions): 65.0797 seconds

I see that line in few places now that it's slowing down. I can't yet tell where in the code it is but the debug file I'm writing the output to is currently at 17GB.

[Andrew Brennan]

Ok, it looks like it's down to the last file taking over a day since in less than 24 hours my output says 792/793 files checked. What I don't know yet is what file is taking the longest, but I have a suspicion that it's our main GUI source. Easy fix to that would be to skip it.

[Andrew Brennan]

Tokenizer::simplifyTokens1::ValueFlow: 66526.7s (That's 18-1/2 hours)

[Oliver Stöneberg]

FYI there is the task of introducing internal timeout handling (we currently do it externally and non-granular in an internal test setup) which would help with diagnosing such things. Unfortunately that still has to wait until I am done with the executor rework.

[Andrew Brennan]

It's one file in my project that's the driver - its ~2200 lines of code and it's taking cppcheck over 54 hours to complete.

I'm going to go through the file and see what about it makes it take so much longer, but my early guess is that it has a lot of string lookups in an if statement that checks a map iterator like so:

if( iterator->first == "String1" || iterator->first == "String2" || ... || iterator->first == "String n" |against about 100 different values, including enums to static maps in a header that has 4 different #ifdef configs (_WIN32, PROJECT_PREPROC_A, etc.).

The class in question is very similar to another class in the same folder that does most of the same things, except for that if statement. I'll see when I get back as I'm test this theory now by rerunning cppcheck with that block of code commented out.

[Andrew Brennan]

My comment above is the bottleneck - commenting out that block reduces the time for that file from 54 hours to just a hair over 5 minutes.

[Oliver Stöneberg]

Yes. The performance is horrible on big if-chains. We also experience it ourselves and leads to annoying long runtime in some of our CI workflows. See https://trac.cppcheck.net/ticket/11262.

[Andrew Brennan]

I don't really like the way our code is in that giant if statement so I'm going to discuss paths forward with my team before I put 2.16.0 into the production testing scheme for RHEL8.

There is still definitely other places the newer version is slower, but just finding and changing this should yield a 4-fold decrease and should reduce down to an hour once I find the other big if statement in a file that takes 13 hours. But that will have to wait a few weeks.

In the meantime, I'll close this appropriately in a moment.

Or I would have if this was a ticket.....