Re: Running out of memory

On 7/7/05, Jacob Foshee <[email protected]> wrote:
>  
> Hello, 
> I recently posted to the user list that we are running doxygen on a large
> project and it is running out of memory.  I might be able to contribute some
> work to get past this problem if it represents a bug, or some room for
> improvement.  So, I joined this list. 
>   
> Does Doxygen normally use so much memory relative to the code it is parsing?
>  (3.5 MB of code -> 2+ GB of pagefile usage) 
> Or is this an unusual case? 

This doesn't seem like a normal amount. I would expect memory usage up to
about 100-300 Mb or so, but certainly not more. Is this during parsing already? 
and with which version of doxygen? Can you provide a stack trace at the 
moment is it using a lot of memory (say >1Gb)?

>   
> If that is 'par for the course', then what options are there for preventing
> Doxygen from running out of memory? 

Disabling features such as the source browser and search engine helps, but
in your case the problem is something else I think.

>   
> Along this line, I started to think that perhaps Doxygen could start to use
> a robust database to manage the large amount of information it has to hold
> onto (eg MySQL). I know this is a non-trivial step, so I'm not talking about
> it casually. 
>   
> This could also be a step towards wish-list item #31 (Only regenerate those
> files that have changed.) 
>   
> Alternatively, Doxygen could query asset managers for source code changes
> directly. (eg CVS, VSS, etc.) I assume that most people with large projects
> use some kind of source control. Does anyone have experience in this area? 
>   
> And I'm sure you've considered a 3rd alternative where Doxygen could build
> doc-object files like a regular compiler.  (I guess these would be like the
> tag files?) 

I have been thinking about such options as well, but one of the main
problems are
the relations that doxygen makes between source files (think of the
cross references
in the source browser or the collaboration diagrams). 

These make it very hard to determine which files (i.e. doc-objects) need to be 
updated beforehand (i.e. without first reading them all in memory, and
recomputing therelations). Without this there would only be a
performance decrease.

Regards,
  Dimitri


-------------------------------------------------------
This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening
July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual
core and dual graphics technology at this free one hour event hosted by HP,
AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar