Cppcheck 2.0 Windows, please hide cmd.exe when Clang is used as the parser

[Zufu Liu]

Here is the project file https://github.com/zufuliu/notepad2/blob/master/Notepad2.cppcheck
after <parser>clang</parser> is added into it (via File -> Edit Project File -> Analysis tab),
every time Cppcheck check a file, cmd.exe is popup.

[Daniel Marjamäki]

ouch! thanks for reporting.

[Daniel Marjamäki]

I created this trac ticket: https://trac.cppcheck.net/ticket/9725

[Daniel Marjamäki]

It's strange.. I have not seen the cmd popup with cppcheck 2.0 .. but it seems to me it should have been shown. Now that should work better; I rewrote the code so it should work better. But well I am not sure if it's fixed or not.
Do you have Qt on your machine? It would be great if you could try out latest git head.

[Zufu Liu]

Sorry, I don't have Qt (It's not free, need a Qt account).
Can you bundle the Qt runtime, z3 DLL and all configuration files in the AppVeyor build like
https://ci.appveyor.com/project/danmar/cppcheck/builds/33007162/job/0n8u1kcbig1oi3sg/artifacts

i.e make these build a portable (compressed with 7z) Cppcheck.

[Zufu Liu]

Just build bbe6157e1638630f3360714fee3790121199d8c4 follow .github\workflows\CI-windows.yml, the cmd.exe is gone.

Some discoveries:
1. #include <stdexcept> is missing from gui\cppchecklibrarydata.cpp, it cause build error for std::runtime_error with VS 2019 16.6, Qt 5.14.2.
2. build.bat is lagged.
3. the .clang-tidy file can be make more readable by putting each check on separator line, like run: | in CI-windows.yml
4. clang parser not work on Windows (at least for my VS project, tested Clang 10 ad Clang 11): Failed to execute 'clang.exe -cc1 -ast-dump -x c++ -internal-isystem ... (clang: error: unknown argument: '-internal-isystem', but clang has is a -isystem option), cppcheck-gui.exe finally exits without any message. however following command works:

based on http://clang.llvm.org/docs/IntroductionToTheClangAST.html and https://stackoverflow.com/questions/32447542/how-do-i-get-clang-to-dump-the-ast-without-color
run on https://github.com/zufuliu/notepad2/tree/master/src folder for Helpers.c (include paths for Windows SDK and Visual C++ headers ignored):

clang-cl -Xclang -ast-dump -fsyntax-only -D_WIN64 -DNDEBUG -DUNICODE -D_UNICODE -D_WIN32_WINNT=0x0502 -DWINVER=0x0502 -D_CRT_SECURE_NO_WARNINGS -D_SCL_SECURE_NO_WARNINGS Helpers.c 1>1.log 2>2.log

clang -Xclang -ast-dump -fsyntax-only -D_WIN64 -DNDEBUG -DUNICODE -D_UNICODE -D_WIN32_WINNT=0x0502 -DWINVER=0x0502 -D_CRT_SECURE_NO_WARNINGS -D_SCL_SECURE_NO_WARNINGS Helpers.c 1>1.log 2>2.log

Attachment is patch to show console messages when running cppcheck-gui.exe on command line.

[Daniel Marjamäki]

Some discoveries:

sounds like good suggestions. please feel free to create github pull requests.

[Daniel Marjamäki]

about build.bat I guess that can be removed. Not sure why it was added, I have the feeling it was used in the releases before somehow.

[Daniel Marjamäki]

It is interesting that you want to use the clang parser when scanning notepad2. I have the feeling that the import is not really rock solid yet.. We should try to make the clang import work well for you.

[Daniel Marjamäki]

For information the clang command was updated with: https://github.com/danmar/cppcheck/commit/31324573fa2766ad9ee3d65802ba8cd1c11b3a58

Thanks for those suggestions. The code is much cleaner now.

[Zufu Liu]

when running cppcheck.exe --project=D:\notepad2\np2\Notepad2.cppcheck, cppcheck prints following:
text Checking D:/notepad2/np2/scintilla/lexers/LexAsm.cxx... Checking D:\notepad2\np2\scintilla\lexers\LexAsm.cxx Release|Win32... Checking D:\notepad2\np2\scintilla\lexers\LexAsm.cxx: _WIN32=1;NDEBUG=1;WIN32=1;_WINDOWS=1;SCI_LEXER=1;NO_CXX11_REGEX=1;UNICODE=1;_UNICODE=1;_CRT_SECURE_NO_WARNINGS=1;_SCL_SECURE_NO_WARNINGS=1;_WIN32_WINNT=0x0501;WINVER=0x0501;_MSC_VER=1900...
where is _MSC_VER=1900 is come from, I never defined it, it cause clang warning for macro redefinition.

I meet is a index error (for scintilla\lexers\LexCPP.cxx) in clangimport.cpp line 1304:

        const int level = (pos1 - 1) / 2;
        if (level == 0 || tree.empty())
            continue;

        AstNodePtr newNode = std::make_shared<AstNode>(nodeType, ext, &data);
        tree[level - 1]->children.push_back(newNode);

where pos1 is zero and level is -1.

and a null pointer error (for scintilla\src\CellBuffer.cxx) in symboldatabase.cpp line 5667.
cpp const Scope *scope = type->scope(); valuetype->typeScope = scope->check->findScope(typeTokens.front(), scope);

[Zufu Liu]

Sorry the index error is for LexCSS.cpp.
I now use following changes, note -std=c++17 is not parsed from Visual Studio project (<LanguageStandard>stdcpp17</LanguageStandard>).

diff --git a/lib/clangimport.cpp b/lib/clangimport.cpp
index 82ccbf671..f9e721055 100644
--- a/lib/clangimport.cpp
+++ b/lib/clangimport.cpp
@@ -1297,7 +1297,7 @@ void clangimport::parseClangAstDump(Tokenizer *tokenizer, std::istream &f)
         }

         const int level = (pos1 - 1) / 2;
-        if (level == 0 || tree.empty())
+        if (level <= 0 || tree.empty())
             continue;

         AstNodePtr newNode = std::make_shared<AstNode>(nodeType, ext, &data);
diff --git a/lib/cppcheck.cpp b/lib/cppcheck.cpp
index b20d97228..0185999e6 100644
--- a/lib/cppcheck.cpp
+++ b/lib/cppcheck.cpp
@@ -237,7 +237,10 @@ static std::string getDefinesFlags(const std::string &semicolonSeparatedString)
 {
     std::string flags;
     for (const std::string &d: split(semicolonSeparatedString, ";"))
+    {
+        if (d != "_MSC_VER")
         flags += "-D" + d + " ";
+    }
     return flags;
 }

@@ -323,7 +326,7 @@ unsigned int CppCheck::check(const std::string &path)
         if (!mSettings.quiet)
             mErrorLogger.reportOut(std::string("Checking ") + path + "...");

-        const std::string lang = Path::isCPP(path) ? "-x c++" : "-x c";
+        const std::string lang = Path::isCPP(path) ? "-x c++ -std=c++17" : "-x c";
         const std::string analyzerInfo = mSettings.buildDir.empty() ? std::string() : AnalyzerInformation::getAnalyzerInfoFile(mSettings.buildDir, path, "");
         const std::string clangcmd = analyzerInfo + ".clang-cmd";
         const std::string clangStderr = analyzerInfo + ".clang-stderr";

[Daniel Marjamäki]

hmm.. I can't reproduce the pos1=0 issue. Your fix looks safe, but I'm a bit worried there is something I am missing.

If you run this command:

mkdir 111
cppcheck --clang --project=build\VS2017\Notepad2.sln --file-filter=scintilla/lexers/LexCPP.cxx --std=c++17 --cppcheck-build-dir=111

Then in the folder "111" there should be files created. The "clang-cmd" tells you what command Cppcheck executes..

Can you generate the AST and see which line starts with a "-" (no spaces before it)?

[Zufu Liu]

It's wild, I can't reproduce with LexCPP.cxx or lexCSS.cxx.
the bug for CellBuffer.cxx still exists.

content of CellBuffer.cxx.analyzerinfo.clang-cmd:

clang.exe -fsyntax-only -Xclang -ast-dump -fno-color-diagnostics -x c++ -std=c++17 -ID:/notepad2/np2/scintilla/include/ -ID:/notepad2/np2/scintilla/lexlib/ -ID:/notepad2/np2/scintilla/src/ -ID:/notepad2/np2/src/ -D_WIN32=1 -D_WIN64=1 -D_DEBUG=1 -D_WIN64=1 -D_WINDOWS=1 -DSCI_LEXER=1 -DNO_CXX11_REGEX=1 -DUNICODE=1 -D_UNICODE=1 -D_CRT_SECURE_NO_WARNINGS=1 -D_SCL_SECURE_NO_WARNINGS=1 -D_WIN32_WINNT=0x0601 -DWINVER=0x0601 -D_MSC_VER=1900 D:/notepad2/np2/scintilla/src/CellBuffer.cxx 2> 111/CellBuffer.cxx.analyzerinfo.clang-stderr

run this command produce 18.7MB ast on stdout but CellBuffer.a1 is empty.

[Daniel Marjamäki]

For information.. I get a different AST import issue for CellBuffer.cxx I am looking at that.

[Daniel Marjamäki]

hmm.. for me the CellBuffer.cxx works fine. the CellBuffer.a1 is not empty. I run from the command line but that shouldn't really change anything. I do use the notepad2.sln.
I made some tweaks.. can you try latest cppcheck?

[Zufu Liu]

OK, I will test it later.

[Zufu Liu]

The running is still on going, but I find another bug for cppcheck command line.

cppcheck.exe --clang test.c
Checking test.c...
Failed to execute 'clang.exe -fsyntax-only -Xclang -ast-dump -fno-color-diagnostics -x c -std=c++20 test.c 2>&1'

clang.exe -fsyntax-only -Xclang -ast-dump -fno-color-diagnostics -x c -std=c++20 test.c
error: invalid argument '-std=c++20' not allowed with 'C'

[Zufu Liu]

There still have some bugs somewhere. the same file is been checked with different c++ standards, sometimes c++20, other times c++14. even worse, cppcheck-gui.exe crashed after run with -std=c++14, while cppeheck.exe continue checking other files.

std::cout << exe << " " << args2 << std::endl;
std::string output2;
if (!mExecuteCommand(exe,split(args2),redirect2,&output2) || output2.find("TranslationUnitDecl") == std::string::npos) {
    std::cerr << "Failed to execute '" << exe << " " << args2 << " " << redirect2 << "'" << std::endl;
    return 0;
}

then run cppcheck.exe --clang --project=D:\notepad2\np2\build\VS2017\Notepad2.sln --file-filter=*LexAsm.cxx

or cppcheck-gui.exe -p D:\notepad2\np2\Notepad2.cppcheck (with <parser>clang</parser> in Notepad2.cppcheck)

both projects only contains <LanguageStandard>stdcpplatest</LanguageStandard> and <LanguageStandard>stdcpp17</LanguageStandard>, stdcpp17 will be removed after VS2019 16.6 is deployed https://github.com/actions/virtual-environments/issues/932

also, it's strange that --file-filter=LexAsm.cxx not work: cppcheck: error: could not find any files matching the filter.

[Daniel Marjamäki]

also, it's strange that --file-filter=LexAsm.cxx not work: cppcheck: error: could not find any files matching the filter.

Use full path. That should work.

[Daniel Marjamäki]

sometimes c++20, other times c++14.

c++14 was wrongly chosen instead of c++17.. fixed.

then run cppcheck.exe --clang --project=D:\notepad2\np2\build\VS2017\Notepad2.sln --file-filter=*LexAsm.cxx

I suggest that you put --file-filter before --project.

Try:

cd d:\notepad2\np2
cppcheck.exe --clang --file-filter=scintilla\lexers\LexAsm.cxx --project=build\VS2017\Notepad2.sln