Thread: [Doxygen-users] doxygen 1.8.9 (and also 1.8.7) extremely slow for vhdl
Brought to you by:
dimitri
From: Walter F.J. M. <w.f...@re...> - 2014-12-30 16:03:18
|
Hi, I'm using doxygen to generate a 'source code browser pages' for a medium sized VHDL project. Some is publicly visible under http://www.retro11.de/doxy/w11/vhd/html/hierarchy.html Total volume is ~ 450 files ~ 85000 lines With doxygen 1.8.7 it took about 32 sec to generate the files ! With doxygen 1.8.9 this slowed down to a snake-speed crawl. Even after more than 18 CPU hours (!!) the files aren't done. The job isn't hung, it's very slowly progressing, one sees a new 'Generating docs' line very few minutes. Resource consumption seems reasonable top gives PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 30 10 125660 73620 4688 R 99.8 1.8 1100:32 doxygen I also tried doxygen 1.8.8, same observation, it proceeds, but at a dreadfully slow pace. In 1.8.8 the VHDL parser was changed, in 1.8.9 many VHDL fixes were done. So my question: Does anybody else see such dramatic performance issues with doxygen 1.8.8/1.8.9 and VHDL ? With best regards, Walter |
From: Dimitri v. H. <do...@gm...> - 2014-12-30 18:10:59
|
Hi Walter, > On 30 Dec 2014, at 16:24 , Walter F.J. Mueller <w.f...@re...> wrote: > > Hi, > > I'm using doxygen to generate a 'source code browser pages' for a > medium sized VHDL project. Some is publicly visible under > http://www.retro11.de/doxy/w11/vhd/html/hierarchy.html > Total volume is > ~ 450 files > ~ 85000 lines > > With doxygen 1.8.7 it took about 32 sec to generate the files ! > > With doxygen 1.8.9 this slowed down to a snake-speed crawl. Even after > more than 18 CPU hours (!!) the files aren't done. The job isn't hung, > it's very slowly progressing, one sees a new 'Generating docs' line > very few minutes. Resource consumption seems reasonable top gives > > PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 30 10 125660 73620 4688 R 99.8 1.8 1100:32 doxygen > > I also tried doxygen 1.8.8, same observation, it proceeds, but at > a dreadfully slow pace. > > In 1.8.8 the VHDL parser was changed, in 1.8.9 many VHDL fixes > were done. So my question: > > Does anybody else see such dramatic performance issues with > doxygen 1.8.8/1.8.9 and VHDL ? That doesn't sound right. Would be nice to understand what doxygen is doing but I haven't been able to reproduce this. What options are enabled? Can you do some profiling? Can you run with the '-d time' option for instance? Does this also happen if you use a smaller subset of the code? Regards, Dimitri |
From: Walter F.J. M. <w.f...@re...> - 2014-12-31 15:25:11
|
On 12/30/2014 07:10 PM, Dimitri van Heesch wrote: > That doesn't sound right. > Would be nice to understand what doxygen is doing but I haven't been able to reproduce this. > > What options are enabled? > Can you do some profiling? Can you run with the '-d time' option for instance? > Does this also happen if you use a smaller subset of the code? Hi Dimitri, the CPU time seems to grow dramatically with increasing project size. The case I described has all designs I have included. To trace the issue I've setup a test environment with the sources of a single design, now 117 files and 28428 lines of code. Now run times for "doxygen -d time" are doxygen-1.8.7 real 0m07.963s user 0m08.510s sys 0m02.143s Spent 0.849 seconds in Generating file sources... Spent 1.106 seconds in Generating class documentation... Spent 3.919 seconds in Running dot... doxygen-1.8.9 real 8m32.386s user 8m32.438s sys 0m02.231s Spent 329.133 seconds in Generating file sources... Spent 177.439 seconds in Generating class documentation... Spent 4.096 seconds in Running dot... I've also done some profiling with "valgrind --tool=callgrind". A quick looks seems to indicate that a lot of time is burned in QGList::contains(), suggesting that a list is searched. The pertinent part of the callgrind_annotate output for 1.8.9 is 150,241,637,969 < VhdlDocGen::findMember(...) 150,241,637,969 * VhdlDocGen::findMemberDef(...) 794,272 > QMapPrivate<QCString, MemberDef*>::insert(...) 21,869 > QGList::append(...) 1,541,745,005 > QCString::StringRep::StringRep(...) 1,824,725,951 > Definition::symbolName(...) 51,057 > ClassDef::getMemberList(...) 19,746 > QGListIterator::operator++(...) 22,775 > MemberListIterator::MemberListIterator(...) 102,891,546,408 > QGList::contains(...) 14,700,790,148 > ...:__strcmp_sse2_unaligned (...) 4,437,837,537 > QCString::operator+=(...) 812 > QCString::StringRep::data(...) 21,024 > QGListIterator::~QGListIterator(...) 1,686,722,110 > QCString::StringRep::~StringRep(...) 500,621 > QCString::StringRep::StringRep(...) 2,160 > QMap<QCString, MemberDef*>::detach(...) 8,384,846,588 > operator+(...) 1,215 > QGListIterator::toFirst(...) Note that I aborted valgrind after about an hour, running doxygen under valgrind is a factor 20 slower that running it directly. The same section of a callgrind_annotate output for 1.8.7, done after a full document creation run, looks like 14,153,463 < VhdlDocGen::writeVHDLTypeDocumentation(...) 1,722,043,706 < generateMemLink(...) 654,130,388 < VhdlDocGen::writeStringLink(...) 2,390,327,557 * VhdlDocGen::findMember(...) 510,960 > QGDictIterator::toFirst(...) 8,278,421 > QCString::QCString(...) 133,736,676 > VhdlDocGen::getClass(...) 1,955,829,369 > VhdlDocGen::findAllPackages(...) 2,402,164 > QCString::upper(...) 3,670,712 > ClassDef::className(...) 298,851 > ClassDef::protection(...) 15,384,427 > QGDict::QGDict(...) 11,590,565 > QGDictIterator::QGDictIterator(...) 13,977,784 > QGDict::~QGDict(...) 736,485 > QGDict::clear(...) 3,625,505 > QCString::assign(...) 154,335 > Definition::getOuterScope(...) 218,592,559 > VhdlDocGen::findMemberDef(...) 5,995,264 > QGDictIterator::~QGDictIterator(...) 8,458,217 > QCString::~QCString(...) 2,396,524 > QCString::lower(...) I hope that helps a bit. With best regards, Walter |