Menu
▾
▴
1 Attachments
#793 Code completion stop working with enum defined inside function body
Some times codecompletion stops working, see the video on the link and the attached project.
Looks like the issue only happens when the struct
struct _READ_DATA {
uint32_t addr;
uint32_t size;
} read_data, *pread_data;
is declared inside the function, if I move the declaration outside the function the issue doesn't happen.
It also seems to only affect autocompletion inside the function there the previous struct was declared,
if I move to other function autocompletion is working.
Reparsing the project makes autocompletion work again but read_data and pread_data are never shown in the list.
https://drive.google.com/open?id=1hNgStc4F1fWPXsv8k0hvWhmanfPaiuyM
I'm running C::B svn build rev 11558, SDK version 1.37.0, scintilla version 3.7.5, wx3.0.4 on Debian testing 64bits.
Thanks for the report.
I will take some time to debug it.
BTW: The screen shot application is quite good. Can you tell me which one do you use, thanks.
I debugged a while, for simplicity, I use such testing code:
struct OUT { uint32_t addr; uint32_t size; }; int main() { struct IN { uint32_t addr; uint32_t size; } read_data, *pread_data; int test_x, test_y; struct OUT read_data2; struct OUT *pread_data2; read_data. return 0; }When I hit the dot key after the " read_data", I see the call stack:
You see, the "struct IN" is simplified to the buffer, thus all the contents of the struct IN is lost.
struct IN {\n\n\n} read_data, *pread_data;I think the function call m_Parser->ParseBuffer() will parse the simplified buffer.
So, the solution here is: we don't need such simplification of the buffer, right?
The simplification is just try to remove the nested { xxx xxx xxx }.
Seems like the issue is somewhere else, I've modified the code to not remove the struct definition
It doesn't remove (or prevent not adding) the definition from
bufferbut it is still not suggestingread_dataorpread_data.OK, I apply your changes, and do some further test. I enabled the "Debug SmartSense" (set the s_DebugSmartSense variable to true), and have the following result in the C::B's Debug log.
It looks like the IN is not parsed correctly.
I think I find the reason. It is located in the in plugins\codecompletion\parser\parserthread.cpp, around the line 1584, in the function body Token* ParserThread::DoAddToken()
newToken = new Token(newname, m_FileIdx, line, ++m_TokenTree->m_TokenTicketCount); TRACE(_T("DoAddToken() : Created token='%s', file_idx=%u, line=%d, ticket=%lu"), newname.wx_str(), m_FileIdx, line, static_cast<unsigned long>(m_TokenTree->m_TokenTicketCount)); Token* finalParent = localParent ? localParent : m_LastParent; if (kind == tkVariable && m_Options.parentIdxOfBuffer != -1) finalParent = m_TokenTree->at(m_Options.parentIdxOfBuffer); newToken->m_ParentIndex = finalParent ? finalParent->m_Index : -1; newToken->m_TokenKind = kind; newToken->m_Scope = m_LastScope; newToken->m_BaseArgs = baseArgs;You look at the line:
if (kind == tkVariable && m_Options.parentIdxOfBuffer != -1) finalParent = m_TokenTree->at(m_Options.parentIdxOfBuffer);When parsing the function body, it looks like "m_Options.parentIdxOfBuffer != -1" is true. So, the function Token is set as the finalParent of the new created Token.
Thus:
struct IN { uint32_t addr; uint32_t size; ...when we add the "addr" as a new Token, the parent should be the "struct IN", not the function "main".
But with the above code, the function "main" is set as the parent, which is not correct.
We can simply remove this condition check, but I'm not sure this cause any other issue.
With the above changes, It still get failed.
It looks like this is wrong?
Surely IN is not a scope containing the current carret (code completion) position.
If the two line
if (kind == tkVariable && m_Options.parentIdxOfBuffer != -1) finalParent = m_TokenTree->at(m_Options.parentIdxOfBuffer);were removed, there are extra errors.
This is cause by the member variable "m_LastParent". When we start parsing the function body of the main(), we should set the m_LastParent as the Token "main". While we enter into the enum IN's body, the m_LastParent should be Token "IN".
Some sample code of handling this is like below:
Token* lastParent = m_LastParent; // save status, will restore after DoParse() TokenScope lastScope = m_LastScope; m_LastParent = newToken; // default scope is: public for namespaces (actually no, but emulate it) m_LastScope = tsPublic; DoParse(); m_LastParent = lastParent; m_LastScope = lastScope;Last edit: ollydbg 2019-02-20
Now, with this patch:
src/plugins/codecompletion/parser/parserthread.cpp | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/plugins/codecompletion/parser/parserthread.cpp b/src/plugins/codecompletion/parser/parserthread.cpp index 57e3cddb..b1135acb 100644 --- a/src/plugins/codecompletion/parser/parserthread.cpp +++ b/src/plugins/codecompletion/parser/parserthread.cpp @@ -517,6 +517,10 @@ bool ParserThread::Parse() bool result = false; m_ParsingTypedef = false; + // if we are parsing the function body, we set the m_LastParent as the function token + if (m_Options.parentIdxOfBuffer != -1) + m_LastParent = m_TokenTree->at(m_Options.parentIdxOfBuffer); + do { if (!m_TokenTree || !m_Tokenizer.IsOK()) @@ -1581,8 +1585,8 @@ Token* ParserThread::DoAddToken(TokenKind kind, m_FileIdx, line, static_cast<unsigned long>(m_TokenTree->m_TokenTicketCount)); Token* finalParent = localParent ? localParent : m_LastParent; - if (kind == tkVariable && m_Options.parentIdxOfBuffer != -1) - finalParent = m_TokenTree->at(m_Options.parentIdxOfBuffer); +// if (kind == tkVariable && m_Options.parentIdxOfBuffer != -1) +// finalParent = m_TokenTree->at(m_Options.parentIdxOfBuffer); newToken->m_ParentIndex = finalParent ? finalParent->m_Index : -1; newToken->m_TokenKind = kind;I have such log:
But I'm not sure why the result is 0.
I think search "read_data" under "386" (The main Token) should return at least one match.
But it looks like "read_data" is not parsed correctly.
With the above changes, this sample code works OK
struct OUT { uint32_t addr; uint32_t size; }; int main() { struct IN { uint32_t addr; uint32_t size; }; IN read_data;//, *pread_data; int test_x, test_y; struct OUT read_data2; struct OUT *pread_data2; read_data.; return 0; }But:
They should have Parent as "main", but now they are empty parent(the global token), which is wrong.
Last edit: ollydbg 2019-02-20
Hi ollydbg, I haven't tried your changes yet but I found this comment on src/plugins/codecompletion/parser/parserthread.cpp:624
And that happens at the end of parsing the IN struct definition but the function doesn't return, is that behavior correct?
As the method DoParse doesn't return upon finding the struct closing '}' it continues to parse the
read_dataand*pread_datadeclarations but it fails at line 1231 because m_Str is emptyI just removed the if condition I mentioned before (I don't know if the condition or the comment is wrong), applied your changes and now it works better, this is the log
But as you can see it doesn't parse the
read_data2declaration.EDIT: I also applied your changes.
Last edit: Fabián Inostroza 2019-02-24
Idealy, the body of the main should parse like below, I have add the comments there
// DoParse Enter(we try to parse the body of main() function) struct IN // { // DoParse Enter(we try to parse the body of IN) uint32_t addr; // uint32_t size; // } // DoParse Exit(exit the body of IN) read_data, *pread_data; // // int test_x, test_y; // // struct OUT read_data2; // struct OUT *pread_data2; // // read_data. // DoParse Exit(we stop at the current caret)From your log message:
Yes, I think the variable
read_data2is lost, so there are some parsing issue.I think this code is wrong here as you already found:
case ParserConsts::clbrace_chr: { m_LastParent = 0L; m_LastScope = tsUndefined; m_Str.Clear(); // the only time we get to find a } is when recursively called by e.g. HandleClass // we have to return now... if (!m_Options.useBuffer || m_Options.bufferSkipBlocks) { m_Tokenizer.SetState(oldState); // This uses the top-level oldState (renamed shadowed versions below) return; } } break;Especially the if condition here:
Even in parsing the function body, we may have recursive call of DoParse, so we should return here.
I think the if condition used here are for simplicity, because a user may write some { } in side the function body, for simplicity, we don't handle those {} in the function body. This will make our parser failed to handle the struct defined inside the function body as your test code.
{ // might be instantiation, see the following // e.g. struct HiddenStruct { int val; }; struct HiddenStruct yy; if (m_ParsingTypedef) { m_Tokenizer.UngetToken(); break; } if (TokenExists(current, m_LastParent, tkClass)) { if (!TokenExists(next, m_LastParent, tkVariable)) { wxString farnext; m_Tokenizer.GetToken(); // go ahead of identifier farnext = m_Tokenizer.PeekToken(); // struct Point p1, p2; // current="Point", next="p1" if (farnext == ParserConsts::semicolon || farnext == ParserConsts::comma) { while (m_Options.handleVars && ( farnext == ParserConsts::semicolon || farnext == ParserConsts::comma )) { if (m_Str.IsEmpty()) m_Str = current; DoAddToken(tkVariable, next, m_Tokenizer.GetLineNumber()); if (farnext == ParserConsts::comma) { m_Tokenizer.GetToken(); // eat the "," next = m_Tokenizer.GetToken(); // next = "p2" farnext = m_Tokenizer.PeekToken(); // farnext = "," or the final ";" continue; } else // we meet a ";", so break the loop { m_Str.Clear(); break; } } m_Tokenizer.GetToken(); // eat semi-colon break; } else m_Tokenizer.UngetToken(); // restore the identifier } } } lastCurrent = current; }Now, for this code, there is a check:
But in your test code, the "OUT" is not defined in the main, but in global scope, so it get failed.
I simple solution is we can remove such check. I don't have much time to debug it.
The interesting is "struct OUT * p;" such patter works.