Re: [Doxygen-develop] SQLite store for entries generated by Doxygen

[Sam P. <the...@gm...>]

I got a script that uses mozillas dxr to create a sqlite database of
my code structures, and then another script to pull in the comments
relating to those code structures.
I would be very interested in using the same database scheme as you,
to make my tool more portable.



On Wed, Aug 10, 2011 at 7:45 AM, Dimitri Van Heesch <do...@gm...> wrote:
> Hi Tuxdna,
>
> Very interesting experiment you did!rur
>
> Since I'm also very interested in the performance, I just tried your build on my MacBook,
> running doxygen on its own sources (I first removed the printf's to avoid printing overhead).
>
> Taking the best time out of three subsequent runs (using 'time doxygen')
> gave 280 seconds "real" time for the patched version (release build),
> against only 27 seconds for the official 1.7.4 release!
>
> So it seems that due to disk caching and OS file caching the performance of seeking is
> not that bad after all. Or alternatively, the overhead of SQLite queries is rather significant.
>
> Can you reproduce my results? Do you think there is room for improvement?
>
> Regards,
>  Dimitri
>
> P.S. I noticed some more warnings while running the patched version on doxygen sources,
> so maybe there is still some information not stored or read back correctly.
>
> On Aug 10, 2011, at 10:37 , tuxdna wrote:
>
>> Hello!
>>
>> Please ignore my previous email titled "dox...@li...".
>> I sent it by mistake.
>>
>> Currently Doxygen stores all the entries that it generates onto a file
>> ( via src/marshall.cpp ), where it sequentially writes everything
>> onto the disk. This is very fast while writing. However it is going to
>> be slow while reading back random entries ( disk spinning too much ).
>> Please correct if I am wrong here. I haven't explored all the options
>> if already provided by Doxygen to make it fast.
>>
>> I have been working on an alternative store which uses SQLite3 [1].
>> I have a working version of the code hosted on github ( branch sqlitedb [2] ).
>> Please note this is my first attempt with modifications to Doxygen.
>>
>> I generated documentation using sqlitedb implementation ( of qtools/ folder ),
>> and also generated documentation with Doxygen 1.7.4 present on the system.
>>
>> This is what I found - the documentation was identical in both the cases
>> with two exceptions:
>> * Doxygen version string was different in both the outputs ( which is obvious )
>> * "struct" is termed as "class" ( I think it is due to some recent
>> changes in Doxygen itself )
>>
>> SQLite3 database that is generated is entry_store_s3.db
>>
>> Please do share your thoughts on this implementation.
>>
>> Thanks and regards,
>> /tuxdna
>>
>> [1] http://sqlite.org/index.html
>> [2] https://github.com/tuxdna/Doxygen/tree/sqlitedb
>> <Doxyfile.custom>------------------------------------------------------------------------------
>> uberSVN's rich system and user administration capabilities and model
>> configuration take the hassle out of deploying and managing Subversion and
>> the tools developers use with it. Learn more about uberSVN and get a free
>> download at:  http://p.sf.net/sfu/wandisco-dev2dev
>> _______________________________________________
>> Doxygen-develop mailing list
>> Dox...@li...
>> https://lists.sourceforge.net/lists/listinfo/doxygen-develop
>
>
> ------------------------------------------------------------------------------
> uberSVN's rich system and user administration capabilities and model
> configuration take the hassle out of deploying and managing Subversion and
> the tools developers use with it. Learn more about uberSVN and get a free
> download at:  http://p.sf.net/sfu/wandisco-dev2dev
> _______________________________________________
> Doxygen-develop mailing list
> Dox...@li...
> https://lists.sourceforge.net/lists/listinfo/doxygen-develop
>



-- 
Thank you,

Sam Price
(707) 742-3726