Re: Excess memory usage growth for a process

http://blogs.msdn.com/alejacma/archive/2009/06/30/sos-cheat-sheet-net-2-0-3-0-3-5.aspx

!DumpHeap -min 85000
!EEHeap -GC; !DumpHeap <LOH begin> <LOH allocated>
    -> Shows all objects in Large Object Heap (LOH/Gen 3).




On Wed, Jul 29, 2009 at 1:06 PM, Greg Young <[email protected]> wrote:

> Yes with ants you have to wait for the problem to happen.
>
> if you google SOS you can find some tutorials on it (it is a separate
> download from windbg ... it can also be used in visual studio!). John
> Robbins also has a great book covering the subject that I highly
> recommend.
>
> Cheers,
>
> greg
>
> On Wed, Jul 29, 2009 at 1:01 PM, Eddie Lascu<[email protected]> wrote:
> > Shoot, I didn't know SOS can do that for you. Can you share more about
> this
> > tool (a link to download it)? Is it part of the WinDbg tool that comes
> from
> > Microsoft? It's true that with ANTS I am at the mercy of the system,
> hoping
> > it will go nuts again and will have the memory size shut up crazy high.
> >
> > Cheers,
> > Eddie
> >
> >
> >
> > -----Original Message-----
> > From: Discussion of advanced .NET topics.
> > [mailto:[email protected]] On Behalf Of Greg Young
> > Sent: Wednesday, July 29, 2009 12:46 PM
> > To: [email protected]
> > Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
> >
> > The wonderful thing about SOS is you dont need to run a profiler ...
> > when it happens take a memory dump, look whats actually happening in
> > the running app, is your LOH fragmented? Otherwise you are just
> > sticking your thumb up your butt and saying it will rain next
> > thursday.
> >
> > On Wed, Jul 29, 2009 at 12:43 PM, Eddie Lascu<[email protected]>
> wrote:
> >> I am using Reg Gate ANTS to profile the app, but it may take too much to
> >> happen again. Before the spike I saw last weekend, the processes ran for
> > 3-4
> >> months with no significant change in the size of memory used.
> >> Will look into Process.WorkingSet.
> >>
> >> Thanks,
> >> Eddie
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: Discussion of advanced .NET topics.
> >> [mailto:[email protected]] On Behalf Of Greg Young
> >> Sent: Wednesday, July 29, 2009 12:38 PM
> >> To: [email protected]
> >> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
> >>
> >> Task manager shows you working set not total memory used but yes the
> >> process class can help you with that (Process.WorkingSet as an
> >> example).
> >>
> >> Have you verified the LOH being the problem by using SOS?
> >>
> >> Cheers,
> >>
> >> Greg
> >>
> >> On Wed, Jul 29, 2009 at 12:33 PM, Eddie Lascu<[email protected]>
> wrote:
> >>> Hi guys,
> >>>
> >>> In light of this article:
> >>>
> >>>
> >>
> >
> http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-ob
> >> ject-heap/
> >>>
> >>> I am no longer sure what I have is a memory leak. I could see how some
> of
> >> the things that happen in my application match the scenarios described
> in
> >> the above article. Hence, I am ready to think about a workaround. My
> >> services are set up to run automatically and restart in case they crash.
> >> While they never crash, I could leverage that feature and use it to fix
> > this
> >> excess memory usage that I experience every once in a blue moon. What I
> >> thought was to monitor the memory used and terminate the application in
> > case
> >> it exceeds a set threshold. The Windows Service Manager will then
> restart
> >> the service and all the memory will be reclaimed by the OS.
> >>>
> >>> My question at this point is whether the .NET Framework has some metric
> >> classes that can help monitor the total amount of memory allocated
> within
> > a
> >> process. Surely System.Diagnostics must have something like that. Is
> > anyone
> >> aware of a way to find out what memory is used by a process, just like
> the
> >> Task Manager does?
> >>>
> >>> Cheers,
> >>> Eddie
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: Discussion of advanced .NET topics.
> >> [mailto:[email protected]] On Behalf Of Shawn
> > Wildermuth
> >>> Sent: Monday, July 27, 2009 12:49 PM
> >>> To: [email protected]
> >>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
> >>>
> >>> Also, your 'using' statement stuff should help, but are you sure that
> all
> >>> your object (XML classes and db connection classes) that support
> >> IDisposable
> >>> are being cleaned up?
> >>>
> >>> Thanks,
> >>>
> >>> Shawn Wildermuth
> >>> http://wildermuth.com
> >>> https://agilitrain.com
> >>> Microsoft MVP (C#), MCSD.NET, Author and Speaker
> >>>
> >>> The Silverlight Tour is coming to a city near you!
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: Discussion of advanced .NET topics.
> >>> [mailto:[email protected]] On Behalf Of John Warner
> >>> Sent: Monday, July 27, 2009 12:28 PM
> >>> To: [email protected]
> >>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
> >>>
> >>> I don't disagree with you just note that before 'some change' working
> OK
> >>> no memory leaks, after change, memory leak. We both know your code did
> > not
> >>> recompile itself to include a memory leak ...
> >>> That reminds has the framework/runtimes been updated/patched?
> >>>
> >>> John Warner
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Discussion of advanced .NET topics. [mailto:ADVANCED-
> >>>> [email protected]] On Behalf Of Eddie Lascu
> >>>> Sent: Monday, July 27, 2009 12:00 PM
> >>>> To: [email protected]
> >>>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
> >>>> process
> >>>>
> >>>> Hi John,
> >>>>
> >>>> I could check that with our system administrators. Do you think that
> can
> >>> be
> >>>> a possible explanation?
> >>>> I just talked with one of the DB Admin and she said she made some
> >>> changes
> >>>> on
> >>>> some Stored Produces, but nothing as dramatic as a patch update. She
> >>> also
> >>>> said that the drivers for the network cards were updated last week,
> but
> >>> I
> >>>> can't see how that can affect the memory used by my processes.
> >>>>
> >>>> Thanks,
> >>>> Eddie
> >>>>
> >>>>
> >>>> -----Original Message-----
> >>>> From: Discussion of advanced .NET topics.
> >>>> [mailto:[email protected]] On Behalf Of
> >>>> John Warner
> >>>> Sent: Monday, July 27, 2009 11:49 AM
> >>>> To: [email protected]
> >>>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
> >>>> process
> >>>>
> >>>> Not what you are seeking, but has anything on the server changed? An
> >>>> Oracle patch or something of that nature?
> >>>>
> >>>> John Warner
> >>>>
> >>>>
> >>>> > -----Original Message-----
> >>>> > From: Discussion of advanced .NET topics. [mailto:ADVANCED-
> >>>> > [email protected]] On Behalf Of Eddie Lascu
> >>>> > Sent: Monday, July 27, 2009 11:33 AM
> >>>> > To: [email protected]
> >>>> > Subject: [ADVANCED-DOTNET] Excess memory usage growth for a
> >>>> process
> >>>> >
> >>>> > Hi guys,
> >>>> >
> >>>> >
> >>>> >
> >>>> > I need your help with an issue I am having right now. I have several
> >>>> windows
> >>>> > services deployed. These services gather some XML files from the
> >>>> internet
> >>>> > and then parse the content and save the information in the database.
> >>> The
> >>>> > database is Oracle 11g, so the access is done through ODP.NET. I
> had
> >>>> some
> >>>> > issues before (and some of you with good memory may recall my
> previous
> >>>> > questions) where I noticed the memory used by these services growing
> >>>> very
> >>>> > high. At the time, I understood that some classes in the ODP.NET
> >>> library
> >>>> use
> >>>> > some JAVA code and they are known for not cleaning properly after
> >>> them.
> >>>> As a
> >>>> > result I rewrote my code in such a way that every dynamic allocation
> >>> of
> >>>> an
> >>>> > object from the ODP.NET library was encompassed in an "using"
> >>> statement,
> >>>> > like this:
> >>>> >
> >>>> >
> >>>> >
> >>>> > using (OracleCommand objDbCommand = new
> >>>> > OracleCommand(strSqlSelectStatement,
> >>>> > objConnection))
> >>>> >
> >>>> >
> >>>> >
> >>>> > All was fine for a very long time. All my services kept a constant
> >>> size
> >>>> when
> >>>> > it came to the total memory used, a size that hovered around 55-60
> MB.
> >>>> This
> >>>> > was the case for months. At the beginning I checked the size almost
> on
> >>> a
> >>>> > daily basis. After a while, my confidence grew and I only checked it
> >>>> once a
> >>>> > week or so. Every time I looked at the size of all my processes (I
> had
> >>> 4
> >>>> > that followed pretty much the same approach on dealing with those
> >>> Oracle
> >>>> > objects) the size was around that 55-60 MB mark.
> >>>> >
> >>>> >
> >>>> >
> >>>> > Now, all of a sudden, this last weekend, the size for some of the
> >>>> services
> >>>> > shut up to about 690 MB, despite the fact the service was running
> with
> >>>> the
> >>>> > exact same code as it did a month ago. This represents a potential
> >>> fatal
> >>>> > problem for the server that can get to a point where it can no
> longer
> >>>> > allocate memory, so I need to find a way to understand what the heck
> >>> is
> >>>> > going on. Unfortunately, I do not have any metrics implemented in my
> >>>> > application to log the total amount of memory used. That would have
> >>> been
> >>>> a
> >>>> > very good indication on when exactly did this surge started to
> happen.
> >>> I
> >>>> > could have checked the Event Viewer to see if the server got hit by
> a
> >>>> > nuclear bomb or something on that precise moment. I really don't get
> >>>> this. I
> >>>> > mean that's why we moved to .NET in the first place, to be rid of
> such
> >>>> > concerns, like making sure all dynamically allocated objects are
> then
> >>>> > destroyed.
> >>>> >
> >>>> >
> >>>> >
> >>>> > Can anyone give me an idea on how to investigate this issue? Are
> there
> >>>> tools
> >>>> > out there that could be used to pick at the memory allocated by a
> >>>> process
> >>>> > and see what are those objects that take up so much  memory? For the
> >>>> time
> >>>> > being, I have the processes running, but two of them are at or close
> >>> to
> >>>> 690
> >>>> > MB and I don't know how long will I be able to keep them running.
> >>>> >
> >>>> >
> >>>> >
> >>>> > If you had an issue like this, what would you try to do?
> >>>> >
> >>>> >
> >>>> >
> >>>> > Any suggestion will be greatly appreciated.
> >>>> >
> >>>> >
> >>>> >
> >>>> > TIA,
> >>>> >
> >>>> > Eddie
> >>>> >
> >>>> >
> >>>> >
> >>>> >
> >>>> > ===================================
> >>>> > View archives and manage your subscription(s) at
> >>>> > http://peach.ease.lsoft.com/archives
> >>>>
> >>>> ===================================
> >>>> View archives and manage your subscription(s) at
> >>>> http://peach.ease.lsoft.com/archives
> >>>>
> >>>> ===================================
> >>>> View archives and manage your subscription(s) at
> >>>> http://peach.ease.lsoft.com/archives
> >>>
> >>> ===================================
> >>> View archives and manage your subscription(s) at
> >>> http://peach.ease.lsoft.com/archives
> >>>
> >>> ===================================
> >>> View archives and manage your subscription(s) at
> >> http://peach.ease.lsoft.com/archives
> >>>
> >>> ===================================
> >>> View archives and manage your subscription(s) at
> >> http://peach.ease.lsoft.com/archives
> >>>
> >>
> >>
> >>
> >> --
> >> Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
> >> de votre attention
> >>
> >> ===================================
> >> View archives and manage your subscription(s) at
> >> http://peach.ease.lsoft.com/archives
> >>
> >> ===================================
> >> View archives and manage your subscription(s) at
> > http://peach.ease.lsoft.com/archives
> >>
> >
> >
> >
> > --
> > Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
> > de votre attention
> >
> > ===================================
> > View archives and manage your subscription(s) at
> > http://peach.ease.lsoft.com/archives
> >
> > ===================================
> > View archives and manage your subscription(s) at
> http://peach.ease.lsoft.com/archives
> >
>
>
>
> --
> Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
> de votre attention
>
> ===================================
> View archives and manage your subscription(s) at
> http://peach.ease.lsoft.com/archives
>

===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives