Re: [Doxygen-develop] SQLite store for entries generated by Doxygen

[Dimitri V. H. <do...@gm...>]

Hi Tuxdna,

Very interesting experiment you did!

Since I'm also very interested in the performance, I just tried your build on my MacBook, 
running doxygen on its own sources (I first removed the printf's to avoid printing overhead). 

Taking the best time out of three subsequent runs (using 'time doxygen') 
gave 280 seconds "real" time for the patched version (release build), 
against only 27 seconds for the official 1.7.4 release! 

So it seems that due to disk caching and OS file caching the performance of seeking is 
not that bad after all. Or alternatively, the overhead of SQLite queries is rather significant.

Can you reproduce my results? Do you think there is room for improvement?

Regards,
  Dimitri

P.S. I noticed some more warnings while running the patched version on doxygen sources,
so maybe there is still some information not stored or read back correctly.

On Aug 10, 2011, at 10:37 , tuxdna wrote:

> Hello!
> 
> Please ignore my previous email titled "dox...@li...".
> I sent it by mistake.
> 
> Currently Doxygen stores all the entries that it generates onto a file
> ( via src/marshall.cpp ), where it sequentially writes everything
> onto the disk. This is very fast while writing. However it is going to
> be slow while reading back random entries ( disk spinning too much ).
> Please correct if I am wrong here. I haven't explored all the options
> if already provided by Doxygen to make it fast.
> 
> I have been working on an alternative store which uses SQLite3 [1].
> I have a working version of the code hosted on github ( branch sqlitedb [2] ).
> Please note this is my first attempt with modifications to Doxygen.
> 
> I generated documentation using sqlitedb implementation ( of qtools/ folder ),
> and also generated documentation with Doxygen 1.7.4 present on the system.
> 
> This is what I found - the documentation was identical in both the cases
> with two exceptions:
> * Doxygen version string was different in both the outputs ( which is obvious )
> * "struct" is termed as "class" ( I think it is due to some recent
> changes in Doxygen itself )
> 
> SQLite3 database that is generated is entry_store_s3.db
> 
> Please do share your thoughts on this implementation.
> 
> Thanks and regards,
> /tuxdna
> 
> [1] http://sqlite.org/index.html
> [2] https://github.com/tuxdna/Doxygen/tree/sqlitedb
> <Doxyfile.custom>------------------------------------------------------------------------------
> uberSVN's rich system and user administration capabilities and model 
> configuration take the hassle out of deploying and managing Subversion and 
> the tools developers use with it. Learn more about uberSVN and get a free 
> download at:  http://p.sf.net/sfu/wandisco-dev2dev
> _______________________________________________
> Doxygen-develop mailing list
> Dox...@li...
> https://lists.sourceforge.net/lists/listinfo/doxygen-develop