Running python addons - fix

[Daniel Marjamäki]

For information, I can't reproduce your problems with cppcheck-2.1 or cppcheck HEAD :-(

But I got a report from 1 more user who had this issue also. I have installed cppcheck-2.1 in C:\Program Files\Cppcheck also. Tested command that works fine for me:

"C:\Program Files\Cppcheck\cppcheck.exe" --addon=misra.py somefile.c

[Daniel Marjamäki]

if(QString::fromStdString(arg).contains("Program Files"))
           args2 << "C:\\Program Files\\Cppcheck\\addons\\misra.py";
else
        args2 << QString::fromStdString(arg);

If arg is quoted maybe you can just remove the quotes. If arg is not quoted then I don't understand what difference this would make.

[Daniel Marjamäki]

For information, I rewrote the split anyway, so it will now unquote the strings.

[eranzim]

I have a similar problem. On Win10, Cppcheck 2.1, python 3.8.5, got this on many files:

Bailing out from checking C:\Code\A\e.c since there was an internal error: Failed to execute addon (command: 'python.exe "C:\Program Files\Cppcheck\addons\threadsafety.py" --cli C:\Code\A\e.c.analyzerinfo.dump')

I tried looking at it with procmon, and found that python.exe tries to open this path: C:\Code\A\"C:\Program Files\Cppcheck\addons\threadsafety.py", which is of course invalid.

[Julien]

Thank you very much ! Your commits seems to solve the path issues but at the moment I can only test on another computer.
On this one I can now see the MISRA reports output on s1 but nothing is displayed on the GUI.
I will try on the second computer from which I was working this week-end tonight and let you know if this one shows misra results.

[Julien]

Using your latest nightly build this is fixed ! Thanks
I tried using python 2.7 and 3.8.5 on Win10.

The last thing is calling from cmd line :
cppcheck --addon=misra.json --project=test.cppcheck
returns errors without rules text :
...style: misra violation (use --rule-texts=<file> to get proper output) [misra-c2012-21.12]
using following misra.json :

{
   "script":"misra.py",
   "args":[
      "--rule-texts=C:/MISRA/misra_rules.txt"
   ]
}

If I try without a json using :
cppcheck --addon="misra.py --rule-texts=C:/MISRA/misra_rules.txt" --project=test.cppcheck
I get the same output as the previous command without rules text.

[Julien]

Note : there should not be any spaces on misra_rules.txt path to get things working on the GUI.

I am not sure this is the good place to speak about this but I would have a feature request :
That would be nice to keep misra rules code on the warning message displayed on the gui as this could be usefull to suppress rules or find corresponding rule with example on misra pdf.
I can create a new discussion thread if you prefer for this request.

[Daniel Marjamäki]

If I try without a json using : cppcheck --addon="misra.py --rule-texts=C:/MISRA/misra_rules.txt" --project=test.cppcheck

very interesting approach I have not considered that at all before but I think we can try to support that :-)

[Julien]

You are speaking about the quotes around misra.py ? Without quotes cppcheck does not recognize --rule-texts command.

[Daniel Marjamäki]

The last thing is calling from cmd line : cppcheck --addon=misra.json --project=test.cppcheck

yes I have seen this also. I am not sure how to solve it properly. If you enable misra in the test.cppcheck then the --addon should ideally somehow not be needed.

[Julien]

Yes removing --addon does the same thing as my previous command. When you say you have seen this also you are speaking about the rules not recognized ?

[Julien]

Daniel, I still have this problem mentionned in my previous post :

Calling from the cmd line :
cppcheck --project=test.cppcheck
returns errors without rules text :
...style: misra violation (use --rule-texts=<file> to get proper output) [misra-c2012-21.12]
Running the same project from Cppcheck GUI works fine, it gives me reports with correct rules.
I haven't found out where the problem could be and what differs from cmdline and GUI.

I am using the GUI while coding as this is easier to read but my pipeline use the cmdline and I would like to keep the project configuration.

I will continue searching but any suggestion would be apreciated.

[Daniel Marjamäki]

As far as I remember, if you run with a cppcheck build folder. then some additional details will be saved in that. For instance there should be some "*-cmd" files that contain the commmands used.

[Daniel Marjamäki]

if that does not work.. I would like that you try to figure out what the --rule-text flag is somehow. maybe you can add some hack in the misra.py so it writes this info. I suggest in def main() in misra.py..

[Julien]

Great idea, using a temporary file to write misra script arguments was really helpfull !

• From GUI misra is called with following arguments :

--cli
--rule-texts=C:/MISRA/misra_rules.txt
C:\Project\Cppcheck\FW-cppcheck-build-dir\main.a1.dump

• From cmdline cppcheck -v --project=Cppcheck/FW.cppcheck:

--cli
Cppcheck\FW-cppcheck-build-dir\main.a1.dump

The rule argument --rule-texts is missing.

[Julien]

Cppcheck GUI knows where the rules are because it can be specified in Preferences > Addons.
Project.cppcheck doesn't save the rules path and it's fine because I don't want to have this path in the project file (it depends on the machine) but is there a way to call cppcheck from cmdline with a project configuration and specify the rule path ?

Using cppcheck --project=Cppcheck/FW.cppcheck --addon=Cppcheck\misra.json with following json :

{
   "script":"misra.py",
   "args":[
      "--rule-texts=C:/MISRA/misra_rules.txt"
   ]
}

First return an error :

Bailing out from checking src\file.c since there was an internal error: Failed to execute 'python.exe "C:\Program Files\Cppcheck\addons\misra.py" --cli Cppcheck\FW-cppcheck-build-dir\file.a1.dump'. ['--cli', 'Cppcheck\FW-cppcheck-build-dir\file.a1.dump']

And immediately return the correct misra.py output for the first project file parsed and then returns only Bailing out... for every next files on the project.

Running without project argument : cppcheck --addon=Cppcheck/misra.json src/file.c gives also Bailing out... followed by correct output :

{"file": "src/file.c", "linenr": 69, "column": 13, "severity": "style", "message": "Required Operands ...", "addon": "misra", "errorId": "c2012-10.1", "extra": "Undefined"}

I don't understand why it first fail calling misra.py and immediately run the script. Is it possible that file.c.dump is not yet accessible while the script is called for the first time ?

[Julien]

Bailing out error was caused by my modification to misra.py.

Running cppcheck --project=Cppcheck/FW.cppcheck --addon=Cppcheck\misra.json is working well when misra addon is removed from FW.cppcheck.

That would be nice if we could have an option to specify rules path without removing addon from .cppcheckand using a json.

[eranzim]

I also still have the same problem I reported in my previous post, on CppCheck 2.3:
[C:/Code/A/e.c:0] (information) Bailing out from checking C:\Code\A\e.c since there was an internal error: Failed to execute addon (command: 'python.exe "C:\Program Files\Cppcheck\addons\threadsafety.py" --cli C:\Code\A\A-cppcheck-build-dir\e.c.analyzerinfo.dump') [internalError]

[Julien]

Do you have python in your system path ?

[Julien]

You can also run the command in 2 time to get more details about the python error :

cppcheck --dump --quiet C:/Code/A/e.c
python threadsafety.py C:/Code/A/e.c.dump

[eranzim]

I do have python 3.8.5 in my path. Running the above lines, the first has no output and the second just writes:

Checking c:\Code\A\e.c.dump...
Checking c:\Code\A\e.c.dump, config ...

I tried running the command that the gui reports fails:

python.exe "c:\Program Files\Cppcheck\addons\threadsafety.py" --cli c:\Code\A\A-cppcheck-build-dir\e.c.analyzerinfo.dump

And got:

Checking c:\Code\A\A-cppcheck-build-dir\e.c.analyzerinfo.dump...
Traceback (most recent call last):
  File "c:\Program Files\Cppcheck\addons\threadsafety.py", line 33, in <module>
    data = cppcheckdata.CppcheckData(arg)
  File "c:\Program Files\Cppcheck\addons\cppcheckdata.py", line 872, in __init__
    for event, node in ElementTree.iterparse(self.filename, events=('start', 'end')):
  File "C:\Python38\lib\xml\etree\ElementTree.py", line 1227, in iterator
    yield from pullparser.read_events()
  File "C:\Python38\lib\xml\etree\ElementTree.py", line 1302, in read_events
    raise event
  File "C:\Python38\lib\xml\etree\ElementTree.py", line 1274, in feed
    self._parser.feed(data)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 467, column 1

Opening c:\Code\A\A-cppcheck-build-dir\e.c.analyzerinfo.dump in line 467, I indeed found some bad xml:

 ...
    <tok fileIndex="0" linenr="96" column="1" str="}"/>
  </rawtokens>
  <suppressions>
  </suppressions>
<<dump cfg="_WIN32=1;WIN32=1;NDEBUG=1;_LIB=1;_MSC_VER=1900">
  <standards>
    <c version="c11"/>
    <cpp version="c++20"/>
  </standards>
  <directivelist>
...

(line 467 is the one that starts with "<<dump", which is indeed some bad xml...)

[eranzim]

Just making sure my reply wasn't missed... For some reason the .analyzerinfo.dump file has a double '<'.

[Julien]

I cannot help you much I am not using this plugin but did you try to remove extra '<' and run it again ? Does the dump file is wrong before you run the plugin ?

[eranzim]

The dump file is wrong before I run the plugin. The plugin simply tries to parse the xml and fails, raising an exception. I'm also not running that plugin in particular, just running a full scan with the GUI and getting these error messages on practically all the scanned files in the project. I just checked another file, and I see its xml is corrupted in a different way (not <<), see attached screenshot. It looks to me like a multithreading issue, where two threads write to the same file simultaneously (is that possible?). I tried setting the thread count to 1; there are still many failures of this plugin, but I tried running a few of them manually and this time they raise a different exception (see attached).

[Julien]

Can you share a failing file with steps to reproduce ? I could run it on my side.