doxygen-develop

[Doxygen-develop] Running out of memory

[Jacob F. <jf...@se...>]

Hello,
I recently posted to the user list that we are running doxygen on a =
large project and it is running out of memory.  I might be able to =
contribute some work to get past this problem if it represents a bug, or =
some room for improvement.  So, I joined this list.
=20
Does Doxygen normally use so much memory relative to the code it is =
parsing?  (3.5 MB of code -> 2+ GB of pagefile usage)
Or is this an unusual case?
=20
If that is 'par for the course', then what options are there for =
preventing Doxygen from running out of memory?
=20
Along this line, I started to think that perhaps Doxygen could start to =
use a robust database to manage the large amount of information it has =
to hold onto (eg MySQL). I know this is a non-trivial step, so I'm not =
talking about it casually.
=20
This could also be a step towards wish-list item #31 (Only regenerate =
those files that have changed.)
=20
Alternatively, Doxygen could query asset managers for source code =
changes directly. (eg CVS, VSS, etc.) I assume that most people with =
large projects use some kind of source control. Does anyone have =
experience in this area?
=20
And I'm sure you've considered a 3rd alternative where Doxygen could =
build doc-object files like a regular compiler.  (I guess these would be =
like the tag files?)
=20
Anyway, don't flame me for my pipedream-type post. ;-)  I'm just =
interested in where things are going, and where I might be able to help.
=20
Cheers and thanks for all your work,
Jacob Foshee
=20

Re: [Doxygen-develop] Running out of memory

[Dimitri v. H. <do...@gm...>]

On 7/7/05, Jacob Foshee <jf...@se...> wrote:
> =20
> Hello,=20
> I recently posted to the user list that we are running doxygen on a large
> project and it is running out of memory.  I might be able to contribute s=
ome
> work to get past this problem if it represents a bug, or some room for
> improvement.  So, I joined this list.=20
>  =20
> Does Doxygen normally use so much memory relative to the code it is parsi=
ng?
>  (3.5 MB of code -> 2+ GB of pagefile usage)=20
> Or is this an unusual case?=20

This doesn't seem like a normal amount. I would expect memory usage up to
about 100-300 Mb or so, but certainly not more. Is this during parsing alre=
ady?=20
and with which version of doxygen? Can you provide a stack trace at the=20
moment is it using a lot of memory (say >1Gb)?

>  =20
> If that is 'par for the course', then what options are there for preventi=
ng
> Doxygen from running out of memory?=20

Disabling features such as the source browser and search engine helps, but
in your case the problem is something else I think.

>  =20
> Along this line, I started to think that perhaps Doxygen could start to u=
se
> a robust database to manage the large amount of information it has to hol=
d
> onto (eg MySQL). I know this is a non-trivial step, so I'm not talking ab=
out
> it casually.=20
>  =20
> This could also be a step towards wish-list item #31 (Only regenerate tho=
se
> files that have changed.)=20
>  =20
> Alternatively, Doxygen could query asset managers for source code changes
> directly. (eg CVS, VSS, etc.) I assume that most people with large projec=
ts
> use some kind of source control. Does anyone have experience in this area=
?=20
>  =20
> And I'm sure you've considered a 3rd alternative where Doxygen could buil=
d
> doc-object files like a regular compiler.  (I guess these would be like t=
he
> tag files?)=20

I have been thinking about such options as well, but one of the main
problems are
the relations that doxygen makes between source files (think of the
cross references
in the source browser or the collaboration diagrams).=20

These make it very hard to determine which files (i.e. doc-objects) need to=
 be=20
updated beforehand (i.e. without first reading them all in memory, and
recomputing therelations). Without this there would only be a
performance decrease.

Regards,
  Dimitri