#33 It took too long for edyuk to build code completion database

Plugins (1)
Trent Zhou

When I added /usr/include to code completion
directory, I found edyuk took too long to start up.

trent@trent-laptop:~$ find /usr/include/ -type 'f' |wc
13942 13942 653225
trent@trent-laptop:~$ ps -e |grep edyuk
8058 pts/1 00:00:00 edyuk
8065 pts/1 01:01:44 edyuk.bin

You can see there are 13942 files in /usr/include, and edyuk has already be running for one hour without showing the IDE screen.

I'm using ubuntu on a Intel(R) Pentium(R) Dual CPU T2390 @ 1.86GHz laptop with 2GB ram.

After diving into the code, I found the problem is in the function QCppParser::update in qcppparser.cpp:

345 void QCppParser::update(QCodeNode *n,
346 QTokenList::const_iterator begin,
347 QTokenList::const_iterator end,
348 bool bNeedCxt,
349 QTokenList *ns)

If I remove the invocation of this function in line 339, edyuk could start up within seconds.

I believe this function could be optimized...


  • fullmetalcoder

    • assigned_to: nobody --> fullmetalcoder
    • status: open --> open-accepted
  • fullmetalcoder

    Adding /usr/include is not very wise because it contains many files and subfolders (Edyuk scan pathes added to completion db recursively). A more sensible approach would be to select the folders you're actually interested in and adding them as separate pathes.

    Improving the current C++ parser would be extremely difficult and would not lead to huge speed improvements.

    However, it is true that while it performs well for relatively small code bodies it is way too slow when generating completion db.

    So I've started working on a new implementation with some extremely promising results already. On my T7700 @ 2.4GHz, the WIP code lex text at about 10Mb/s and parse it at about 25M tokens/s.

    I just need to find some time to implement a proper preprocessor and bind a completion db generator to the new parser and I'll be able to replace the old one.

    Considering the fact that Qt headers weigh less than 2Mb, completion db creation time could be reduced by a factor 10 or more.