Re: Excess memory usage growth for a process

Based on my preliminary reading of what this has to offer, it seems you =
can
integrate it with your VS IDE (not an option for me as my processes are
already deployed) or use it via WinDbg, but you need to look at the =
debug
version or the runtime version that has the .pdb file (not an option =
either
as I only have the sole runtime file deployed, but much easily fixable). =
Is
this statement accurate?

Thanks,
Eddie

-----Original Message-----
From: Discussion of advanced .NET topics.
[mailto:[email protected]] On Behalf Of Adam Tuliper
Sent: Wednesday, July 29, 2009 1:24 PM
To: [email protected]
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process

http://blogs.msdn.com/alejacma/archive/2009/06/30/sos-cheat-sheet-net-2-0=
-3-
0-3-5.aspx

!DumpHeap -min 85000
!EEHeap -GC; !DumpHeap <LOH begin> <LOH allocated>
    -> Shows all objects in Large Object Heap (LOH/Gen 3).




On Wed, Jul 29, 2009 at 1:06 PM, Greg Young <[email protected]> =
wrote:

> Yes with ants you have to wait for the problem to happen.
>
> if you google SOS you can find some tutorials on it (it is a separate
> download from windbg ... it can also be used in visual studio!). John
> Robbins also has a great book covering the subject that I highly
> recommend.
>
> Cheers,
>
> greg
>
> On Wed, Jul 29, 2009 at 1:01 PM, Eddie Lascu<[email protected]> =
wrote:
> > Shoot, I didn't know SOS can do that for you. Can you share more =
about
> this
> > tool (a link to download it)? Is it part of the WinDbg tool that =
comes
> from
> > Microsoft? It's true that with ANTS I am at the mercy of the system,
> hoping
> > it will go nuts again and will have the memory size shut up crazy =
high.
> >
> > Cheers,
> > Eddie
> >
> >
> >
> > -----Original Message-----
> > From: Discussion of advanced .NET topics.
> > [mailto:[email protected]] On Behalf Of Greg =
Young
> > Sent: Wednesday, July 29, 2009 12:46 PM
> > To: [email protected]
> > Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a =
process
> >
> > The wonderful thing about SOS is you dont need to run a profiler ...
> > when it happens take a memory dump, look whats actually happening in
> > the running app, is your LOH fragmented? Otherwise you are just
> > sticking your thumb up your butt and saying it will rain next
> > thursday.
> >
> > On Wed, Jul 29, 2009 at 12:43 PM, Eddie Lascu<[email protected]>
> wrote:
> >> I am using Reg Gate ANTS to profile the app, but it may take too =
much
to
> >> happen again. Before the spike I saw last weekend, the processes =
ran
for
> > 3-4
> >> months with no significant change in the size of memory used.
> >> Will look into Process.WorkingSet.
> >>
> >> Thanks,
> >> Eddie
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: Discussion of advanced .NET topics.
> >> [mailto:[email protected]] On Behalf Of Greg =
Young
> >> Sent: Wednesday, July 29, 2009 12:38 PM
> >> To: [email protected]
> >> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a =
process
> >>
> >> Task manager shows you working set not total memory used but yes =
the
> >> process class can help you with that (Process.WorkingSet as an
> >> example).
> >>
> >> Have you verified the LOH being the problem by using SOS?
> >>
> >> Cheers,
> >>
> >> Greg
> >>
> >> On Wed, Jul 29, 2009 at 12:33 PM, Eddie Lascu<[email protected]>
> wrote:
> >>> Hi guys,
> >>>
> >>> In light of this article:
> >>>
> >>>
> >>
> >
>
http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large=
-ob
> >> ject-heap/
> >>>
> >>> I am no longer sure what I have is a memory leak. I could see how =
some
> of
> >> the things that happen in my application match the scenarios =
described
> in
> >> the above article. Hence, I am ready to think about a workaround. =
My
> >> services are set up to run automatically and restart in case they
crash.
> >> While they never crash, I could leverage that feature and use it to =
fix
> > this
> >> excess memory usage that I experience every once in a blue moon. =
What I
> >> thought was to monitor the memory used and terminate the =
application in
> > case
> >> it exceeds a set threshold. The Windows Service Manager will then
> restart
> >> the service and all the memory will be reclaimed by the OS.
> >>>
> >>> My question at this point is whether the .NET Framework has some
metric
> >> classes that can help monitor the total amount of memory allocated
> within
> > a
> >> process. Surely System.Diagnostics must have something like that. =
Is
> > anyone
> >> aware of a way to find out what memory is used by a process, just =
like
> the
> >> Task Manager does?
> >>>
> >>> Cheers,
> >>> Eddie
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: Discussion of advanced .NET topics.
> >> [mailto:[email protected]] On Behalf Of Shawn
> > Wildermuth
> >>> Sent: Monday, July 27, 2009 12:49 PM
> >>> To: [email protected]
> >>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
process
> >>>
> >>> Also, your 'using' statement stuff should help, but are you sure =
that
> all
> >>> your object (XML classes and db connection classes) that support
> >> IDisposable
> >>> are being cleaned up?
> >>>
> >>> Thanks,
> >>>
> >>> Shawn Wildermuth
> >>> http://wildermuth.com
> >>> https://agilitrain.com
> >>> Microsoft MVP (C#), MCSD.NET, Author and Speaker
> >>>
> >>> The Silverlight Tour is coming to a city near you!
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: Discussion of advanced .NET topics.
> >>> [mailto:[email protected]] On Behalf Of John =
Warner
> >>> Sent: Monday, July 27, 2009 12:28 PM
> >>> To: [email protected]
> >>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
process
> >>>
> >>> I don't disagree with you just note that before 'some change' =
working
> OK
> >>> no memory leaks, after change, memory leak. We both know your code =
did
> > not
> >>> recompile itself to include a memory leak ...
> >>> That reminds has the framework/runtimes been updated/patched?
> >>>
> >>> John Warner
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Discussion of advanced .NET topics. [mailto:ADVANCED-
> >>>> [email protected]] On Behalf Of Eddie Lascu
> >>>> Sent: Monday, July 27, 2009 12:00 PM
> >>>> To: [email protected]
> >>>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
> >>>> process
> >>>>
> >>>> Hi John,
> >>>>
> >>>> I could check that with our system administrators. Do you think =
that
> can
> >>> be
> >>>> a possible explanation?
> >>>> I just talked with one of the DB Admin and she said she made some
> >>> changes
> >>>> on
> >>>> some Stored Produces, but nothing as dramatic as a patch update. =
She
> >>> also
> >>>> said that the drivers for the network cards were updated last =
week,
> but
> >>> I
> >>>> can't see how that can affect the memory used by my processes.
> >>>>
> >>>> Thanks,
> >>>> Eddie
> >>>>
> >>>>
> >>>> -----Original Message-----
> >>>> From: Discussion of advanced .NET topics.
> >>>> [mailto:[email protected]] On Behalf Of
> >>>> John Warner
> >>>> Sent: Monday, July 27, 2009 11:49 AM
> >>>> To: [email protected]
> >>>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
> >>>> process
> >>>>
> >>>> Not what you are seeking, but has anything on the server changed? =
An
> >>>> Oracle patch or something of that nature?
> >>>>
> >>>> John Warner
> >>>>
> >>>>
> >>>> > -----Original Message-----
> >>>> > From: Discussion of advanced .NET topics. [mailto:ADVANCED-
> >>>> > [email protected]] On Behalf Of Eddie Lascu
> >>>> > Sent: Monday, July 27, 2009 11:33 AM
> >>>> > To: [email protected]
> >>>> > Subject: [ADVANCED-DOTNET] Excess memory usage growth for a
> >>>> process
> >>>> >
> >>>> > Hi guys,
> >>>> >
> >>>> >
> >>>> >
> >>>> > I need your help with an issue I am having right now. I have
several
> >>>> windows
> >>>> > services deployed. These services gather some XML files from =
the
> >>>> internet
> >>>> > and then parse the content and save the information in the
database.
> >>> The
> >>>> > database is Oracle 11g, so the access is done through ODP.NET. =
I
> had
> >>>> some
> >>>> > issues before (and some of you with good memory may recall my
> previous
> >>>> > questions) where I noticed the memory used by these services
growing
> >>>> very
> >>>> > high. At the time, I understood that some classes in the =
ODP.NET
> >>> library
> >>>> use
> >>>> > some JAVA code and they are known for not cleaning properly =
after
> >>> them.
> >>>> As a
> >>>> > result I rewrote my code in such a way that every dynamic
allocation
> >>> of
> >>>> an
> >>>> > object from the ODP.NET library was encompassed in an "using"
> >>> statement,
> >>>> > like this:
> >>>> >
> >>>> >
> >>>> >
> >>>> > using (OracleCommand objDbCommand =3D new
> >>>> > OracleCommand(strSqlSelectStatement,
> >>>> > objConnection))
> >>>> >
> >>>> >
> >>>> >
> >>>> > All was fine for a very long time. All my services kept a =
constant
> >>> size
> >>>> when
> >>>> > it came to the total memory used, a size that hovered around =
55-60
> MB.
> >>>> This
> >>>> > was the case for months. At the beginning I checked the size =
almost
> on
> >>> a
> >>>> > daily basis. After a while, my confidence grew and I only =
checked
it
> >>>> once a
> >>>> > week or so. Every time I looked at the size of all my processes =
(I
> had
> >>> 4
> >>>> > that followed pretty much the same approach on dealing with =
those
> >>> Oracle
> >>>> > objects) the size was around that 55-60 MB mark.
> >>>> >
> >>>> >
> >>>> >
> >>>> > Now, all of a sudden, this last weekend, the size for some of =
the
> >>>> services
> >>>> > shut up to about 690 MB, despite the fact the service was =
running
> with
> >>>> the
> >>>> > exact same code as it did a month ago. This represents a =
potential
> >>> fatal
> >>>> > problem for the server that can get to a point where it can no
> longer
> >>>> > allocate memory, so I need to find a way to understand what the
heck
> >>> is
> >>>> > going on. Unfortunately, I do not have any metrics implemented =
in
my
> >>>> > application to log the total amount of memory used. That would =
have
> >>> been
> >>>> a
> >>>> > very good indication on when exactly did this surge started to
> happen.
> >>> I
> >>>> > could have checked the Event Viewer to see if the server got =
hit by
> a
> >>>> > nuclear bomb or something on that precise moment. I really =
don't
get
> >>>> this. I
> >>>> > mean that's why we moved to .NET in the first place, to be rid =
of
> such
> >>>> > concerns, like making sure all dynamically allocated objects =
are
> then
> >>>> > destroyed.
> >>>> >
> >>>> >
> >>>> >
> >>>> > Can anyone give me an idea on how to investigate this issue? =
Are
> there
> >>>> tools
> >>>> > out there that could be used to pick at the memory allocated by =
a
> >>>> process
> >>>> > and see what are those objects that take up so much  memory? =
For
the
> >>>> time
> >>>> > being, I have the processes running, but two of them are at or
close
> >>> to
> >>>> 690
> >>>> > MB and I don't know how long will I be able to keep them =
running.
> >>>> >
> >>>> >
> >>>> >
> >>>> > If you had an issue like this, what would you try to do?
> >>>> >
> >>>> >
> >>>> >
> >>>> > Any suggestion will be greatly appreciated.
> >>>> >
> >>>> >
> >>>> >
> >>>> > TIA,
> >>>> >
> >>>> > Eddie
> >>>> >
> >>>> >
> >>>> >
> >>>> >
> >>>> > =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >>>> > View archives and manage your subscription(s) at
> >>>> > http://peach.ease.lsoft.com/archives
> >>>>
> >>>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >>>> View archives and manage your subscription(s) at
> >>>> http://peach.ease.lsoft.com/archives
> >>>>
> >>>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >>>> View archives and manage your subscription(s) at
> >>>> http://peach.ease.lsoft.com/archives
> >>>
> >>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >>> View archives and manage your subscription(s) at
> >>> http://peach.ease.lsoft.com/archives
> >>>
> >>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >>> View archives and manage your subscription(s) at
> >> http://peach.ease.lsoft.com/archives
> >>>
> >>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >>> View archives and manage your subscription(s) at
> >> http://peach.ease.lsoft.com/archives
> >>>
> >>
> >>
> >>
> >> --
> >> Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour =
m'assurer
> >> de votre attention
> >>
> >> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >> View archives and manage your subscription(s) at
> >> http://peach.ease.lsoft.com/archives
> >>
> >> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >> View archives and manage your subscription(s) at
> > http://peach.ease.lsoft.com/archives
> >>
> >
> >
> >
> > --
> > Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour =
m'assurer
> > de votre attention
> >
> > =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > View archives and manage your subscription(s) at
> > http://peach.ease.lsoft.com/archives
> >
> > =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > View archives and manage your subscription(s) at
> http://peach.ease.lsoft.com/archives
> >
>
>
>
> --
> Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour =
m'assurer
> de votre attention
>
> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> View archives and manage your subscription(s) at
> http://peach.ease.lsoft.com/archives
>

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives