Menu
▾
▴
#232 cctest: add automatically test for doxygen document reading
As suggestion by OBF in Code::Blocks / Tickets / #224 Duplication in documentation tooltip, we may need a cctest case for the doxygen document.
As suggestion by OBF in Code::Blocks / Tickets / #224 Duplication in documentation tooltip, we may need a cctest case for the doxygen document.
I've wrote a basic implementation, but it only does substring tests, so it's not suited for a full match or duplicates. I guess it'll require regex support to handle those.
Notes:
1) I've also fixed some trimming while I was at it. (wxString::Trim(bool trim_right) defaults to
trueand cctest did two Trim().. one with defaulttrueand one with implicittrue)2) the code inside the constructor isn't required since it's usually the default anyway... but maybe do it implicit nontheless
3) the patch to
src/plugins/codecompletion/parser/parser_base.cppshould be properly redone, or at least every use of it verified since it'll probably always default tostoreDocumentation = trueeven in situations where it "shouldn't"4) the last "test" should fail... there are 2 special test values for doxygen comments, "*" as a "non-empty" wildcard, and "-" to check for empty docs
But it'll only work with this patch also applied:
And a test case (which currently fails all #define comments in my build...)
Thanks, save the test file as ccc_parse_doxygen_document.cpp, and run the tests, here is the result.
So, a "-" rule means the token don't have doxygen documents?
EDIT: Oh, you have already stated that:
Last edit: ollydbg 2016-09-07
It looks like token->m_Doc is always empty string, not sure why, let's look into the code.
EDIT:
it looks like m_TokenizerOptions.storeDocumentation is false when parsing the file.
Last edit: ollydbg 2016-09-07
OK, found the reason:
and
bool ParserBase::Reparse(const wxString& file, cb_unused bool isLocal) { FileLoader* loader = new FileLoader(file); (*loader)(); ParserThreadOptions opts; opts.useBuffer = false; // default opts.parentIdxOfBuffer = -1; // default opts.initLineOfBuffer = -1; // default opts.bufferSkipBlocks = false; // default opts.bufferSkipOuterBlocks = false; // default opts.isTemp = false; // default opts.followLocalIncludes = true; // default opts.followGlobalIncludes = true; // default opts.wantPreprocessor = true; // default opts.parseComplexMacros = true; // default opts.platformCheck = true; // default opts.handleFunctions = true; // default opts.handleVars = true; // default opts.handleClasses = true; // default opts.handleEnums = true; // default opts.handleTypedefs = true; // default opts.loader = loader; ParserThread* pt = new ParserThread(this, file, true, opts, m_TokenTree); bool success = pt->Parse(); delete pt; return success; }But the ParserThreadOptions has the field storeDocumentation = false
struct ParserThreadOptions { ParserThreadOptions() : useBuffer(false), fileOfBuffer(), parentIdxOfBuffer(-1), initLineOfBuffer(1), bufferSkipBlocks(false), bufferSkipOuterBlocks(false), isTemp(false), followLocalIncludes(true), followGlobalIncludes(true), wantPreprocessor(true), parseComplexMacros(true), platformCheck(true), handleFunctions(true), handleVars(true), handleClasses(true), handleEnums(true), handleTypedefs(true), storeDocumentation(false), loader(nullptr) {}OK, now, I can see some tests are passed, see the result below:
With such changes
From af1b96e8ec639907f54e137d721677289f4542a3 Mon Sep 17 00:00:00 2001 From: asmwarrior <asmwarrior@gmail.com> Date: Wed, 7 Sep 2016 21:59:38 +0800 Subject: patch3: two changes here: 1: enable the store doxygen option 2: first mark the file in the TokenTree, so that the doxygen document is stored in the Token::m_Doc, otherwise, it was wrongly stored in the m_ImpDoc. See Token* ParserThread::DoAddToken() about how the doxygen document are added to the Token. diff --git a/src/plugins/codecompletion/parser/parser_base.cpp b/src/plugins/codecompletion/parser/parser_base.cpp index 82e1a55..4a4641d 100644 --- a/src/plugins/codecompletion/parser/parser_base.cpp +++ b/src/plugins/codecompletion/parser/parser_base.cpp @@ -229,8 +229,15 @@ bool ParserBase::Reparse(const wxString& file, cb_unused bool isLocal) opts.handleEnums = true; // default opts.handleTypedefs = true; // default + opts.storeDocumentation = true; // enable this option to enable cctest for doxygen doc reading + opts.loader = loader; + // the file should first put in the TokenTree, so the index is correct when initlize the + // Tokenizer object inside the ParserThread::ParserThread() + + m_TokenTree->ReserveFileForParsing(file, true); + ParserThread* pt = new ParserThread(this, file, true, opts, m_TokenTree); bool success = pt->Parse(); delete pt;oh right... my patch to
src/plugins/codecompletion/parser/parser_base.cpponly covered the case where manual input in cctest is used...Though, can't we just do it the way Code::Blocks does? (I guess it never uses
ParseBuffer()but it should use files^^)Or what's the difference and why?
The function call "m_TokenTree->ReserveFileForParsing(file, true);" follows the way the native codecompletion's parser does.
Each file in the TokenTree has an index, so we need to call ReserveFileForParsing() first to give a file index for the file. And later, both the Parserthread and the Tokenizer will use this file index.
For parsing the buffer, I'm not quite sure, maybe, an editor buffer does not have a file index, so it use the file index default 0.
OK, I found a bug that the doxygen documents are not added to the Token, because macro definitions are added in the Tokenizer class(not in the parserthread class), so, here is a simple fix:
From 20e4325687084f3747fae52d2a0c1c2eede850ce Mon Sep 17 00:00:00 2001 From: asmwarrior <asmwarrior@gmail.com> Date: Wed, 7 Sep 2016 22:35:02 +0800 Subject: * fix a bug that we have forget adding the doxygen documents for macro definitions. diff --git a/src/plugins/codecompletion/parser/tokenizer.cpp b/src/plugins/codecompletion/parser/tokenizer.cpp index 467135a..16bdc1e 100644 --- a/src/plugins/codecompletion/parser/tokenizer.cpp +++ b/src/plugins/codecompletion/parser/tokenizer.cpp @@ -2029,4 +2029,7 @@ void Tokenizer::AddMacroDefinition(wxString name, int line, wxString para, wxStr // update the definition token->m_Args = para; // macro call's formal args token->m_FullType = substitues; // replace list + + // this will append the doxygen style comments to the Token + SetLastTokenIdx(token->m_Index); }After this fix, I get such results:
This is the expect result. The 1 FAIL is also correct, because DEF1 did have doxygen documents.
Great job, White Tiger!
The modified patch is in trunk now(rev10902), Thanks!