Re: Excess memory usage growth for a process

Thank you, Guys. I will look into that as well. Do I need a special =
start of
the process if I am to use ADPlus or I can attach to the process later?


-----Original Message-----
From: Discussion of advanced .NET topics.
[mailto:[email protected]] On Behalf Of Greg Young
Sent: Thursday, July 30, 2009 1:52 PM
To: [email protected]
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process

Yep exactly what ADPlus is meant for! you can even do this right
within ADPlus :)

On Thu, Jul 30, 2009 at 1:49 PM, Wilson, =
Phil<[email protected]>
wrote:
> Not answering this particular question, but I believe you could hook =
all
this together to automatically get that dump when the memory gets large =
if
it's rare enough that you don't want to sit and look at it all day. =
Perfmon
has triggers that can fire off a script, and you can fire off the =
debugging
script ADPlus to dump your process with the -hang option.
>
> Phil Wilson
>
>
> -----Original Message-----
> From: Discussion of advanced .NET topics.
[mailto:[email protected]] On Behalf Of Eddie Lascu
> Sent: Wednesday, July 29, 2009 11:40 AM
> To: [email protected]
> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a =
process
>
> Based on my preliminary reading of what this has to offer, it seems =
you
can
> integrate it with your VS IDE (not an option for me as my processes =
are
> already deployed) or use it via WinDbg, but you need to look at the =
debug
> version or the runtime version that has the .pdb file (not an option
either
> as I only have the sole runtime file deployed, but much easily =
fixable).
Is
> this statement accurate?
>
> Thanks,
> Eddie
>
> -----Original Message-----
> From: Discussion of advanced .NET topics.
> [mailto:[email protected]] On Behalf Of Adam =
Tuliper
> Sent: Wednesday, July 29, 2009 1:24 PM
> To: [email protected]
> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a =
process
>
>
http://blogs.msdn.com/alejacma/archive/2009/06/30/sos-cheat-sheet-net-2-0=
-3-
> 0-3-5.aspx
>
> !DumpHeap -min 85000
> !EEHeap -GC; !DumpHeap <LOH begin> <LOH allocated>
> =A0 =A0-> Shows all objects in Large Object Heap (LOH/Gen 3).
>
>
>
>
> On Wed, Jul 29, 2009 at 1:06 PM, Greg Young <[email protected]>
wrote:
>
>> Yes with ants you have to wait for the problem to happen.
>>
>> if you google SOS you can find some tutorials on it (it is a separate
>> download from windbg ... it can also be used in visual studio!). John
>> Robbins also has a great book covering the subject that I highly
>> recommend.
>>
>> Cheers,
>>
>> greg
>>
>> On Wed, Jul 29, 2009 at 1:01 PM, Eddie Lascu<[email protected]> =
wrote:
>> > Shoot, I didn't know SOS can do that for you. Can you share more =
about
>> this
>> > tool (a link to download it)? Is it part of the WinDbg tool that =
comes
>> from
>> > Microsoft? It's true that with ANTS I am at the mercy of the =
system,
>> hoping
>> > it will go nuts again and will have the memory size shut up crazy =
high.
>> >
>> > Cheers,
>> > Eddie
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Discussion of advanced .NET topics.
>> > [mailto:[email protected]] On Behalf Of Greg =
Young
>> > Sent: Wednesday, July 29, 2009 12:46 PM
>> > To: [email protected]
>> > Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a =
process
>> >
>> > The wonderful thing about SOS is you dont need to run a profiler =
...
>> > when it happens take a memory dump, look whats actually happening =
in
>> > the running app, is your LOH fragmented? Otherwise you are just
>> > sticking your thumb up your butt and saying it will rain next
>> > thursday.
>> >
>> > On Wed, Jul 29, 2009 at 12:43 PM, Eddie Lascu<[email protected]>
>> wrote:
>> >> I am using Reg Gate ANTS to profile the app, but it may take too =
much
> to
>> >> happen again. Before the spike I saw last weekend, the processes =
ran
> for
>> > 3-4
>> >> months with no significant change in the size of memory used.
>> >> Will look into Process.WorkingSet.
>> >>
>> >> Thanks,
>> >> Eddie
>> >>
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Discussion of advanced .NET topics.
>> >> [mailto:[email protected]] On Behalf Of Greg =
Young
>> >> Sent: Wednesday, July 29, 2009 12:38 PM
>> >> To: [email protected]
>> >> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
process
>> >>
>> >> Task manager shows you working set not total memory used but yes =
the
>> >> process class can help you with that (Process.WorkingSet as an
>> >> example).
>> >>
>> >> Have you verified the LOH being the problem by using SOS?
>> >>
>> >> Cheers,
>> >>
>> >> Greg
>> >>
>> >> On Wed, Jul 29, 2009 at 12:33 PM, Eddie Lascu<[email protected]>
>> wrote:
>> >>> Hi guys,
>> >>>
>> >>> In light of this article:
>> >>>
>> >>>
>> >>
>> >
>>
>
http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large=
-ob
>> >> ject-heap/
>> >>>
>> >>> I am no longer sure what I have is a memory leak. I could see how
some
>> of
>> >> the things that happen in my application match the scenarios =
described
>> in
>> >> the above article. Hence, I am ready to think about a workaround. =
My
>> >> services are set up to run automatically and restart in case they
> crash.
>> >> While they never crash, I could leverage that feature and use it =
to
fix
>> > this
>> >> excess memory usage that I experience every once in a blue moon. =
What
I
>> >> thought was to monitor the memory used and terminate the =
application
in
>> > case
>> >> it exceeds a set threshold. The Windows Service Manager will then
>> restart
>> >> the service and all the memory will be reclaimed by the OS.
>> >>>
>> >>> My question at this point is whether the .NET Framework has some
> metric
>> >> classes that can help monitor the total amount of memory allocated
>> within
>> > a
>> >> process. Surely System.Diagnostics must have something like that. =
Is
>> > anyone
>> >> aware of a way to find out what memory is used by a process, just =
like
>> the
>> >> Task Manager does?
>> >>>
>> >>> Cheers,
>> >>> Eddie
>> >>>
>> >>>
>> >>> -----Original Message-----
>> >>> From: Discussion of advanced .NET topics.
>> >> [mailto:[email protected]] On Behalf Of Shawn
>> > Wildermuth
>> >>> Sent: Monday, July 27, 2009 12:49 PM
>> >>> To: [email protected]
>> >>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
> process
>> >>>
>> >>> Also, your 'using' statement stuff should help, but are you sure =
that
>> all
>> >>> your object (XML classes and db connection classes) that support
>> >> IDisposable
>> >>> are being cleaned up?
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Shawn Wildermuth
>> >>> http://wildermuth.com
>> >>> https://agilitrain.com
>> >>> Microsoft MVP (C#), MCSD.NET, Author and Speaker
>> >>>
>> >>> The Silverlight Tour is coming to a city near you!
>> >>>
>> >>>
>> >>> -----Original Message-----
>> >>> From: Discussion of advanced .NET topics.
>> >>> [mailto:[email protected]] On Behalf Of John
Warner
>> >>> Sent: Monday, July 27, 2009 12:28 PM
>> >>> To: [email protected]
>> >>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
> process
>> >>>
>> >>> I don't disagree with you just note that before 'some change' =
working
>> OK
>> >>> no memory leaks, after change, memory leak. We both know your =
code
did
>> > not
>> >>> recompile itself to include a memory leak ...
>> >>> That reminds has the framework/runtimes been updated/patched?
>> >>>
>> >>> John Warner
>> >>>
>> >>>
>> >>>> -----Original Message-----
>> >>>> From: Discussion of advanced .NET topics. [mailto:ADVANCED-
>> >>>> [email protected]] On Behalf Of Eddie Lascu
>> >>>> Sent: Monday, July 27, 2009 12:00 PM
>> >>>> To: [email protected]
>> >>>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
>> >>>> process
>> >>>>
>> >>>> Hi John,
>> >>>>
>> >>>> I could check that with our system administrators. Do you think =
that
>> can
>> >>> be
>> >>>> a possible explanation?
>> >>>> I just talked with one of the DB Admin and she said she made =
some
>> >>> changes
>> >>>> on
>> >>>> some Stored Produces, but nothing as dramatic as a patch update. =
She
>> >>> also
>> >>>> said that the drivers for the network cards were updated last =
week,
>> but
>> >>> I
>> >>>> can't see how that can affect the memory used by my processes.
>> >>>>
>> >>>> Thanks,
>> >>>> Eddie
>> >>>>
>> >>>>
>> >>>> -----Original Message-----
>> >>>> From: Discussion of advanced .NET topics.
>> >>>> [mailto:[email protected]] On Behalf Of
>> >>>> John Warner
>> >>>> Sent: Monday, July 27, 2009 11:49 AM
>> >>>> To: [email protected]
>> >>>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
>> >>>> process
>> >>>>
>> >>>> Not what you are seeking, but has anything on the server =
changed? An
>> >>>> Oracle patch or something of that nature?
>> >>>>
>> >>>> John Warner
>> >>>>
>> >>>>
>> >>>> > -----Original Message-----
>> >>>> > From: Discussion of advanced .NET topics. [mailto:ADVANCED-
>> >>>> > [email protected]] On Behalf Of Eddie Lascu
>> >>>> > Sent: Monday, July 27, 2009 11:33 AM
>> >>>> > To: [email protected]
>> >>>> > Subject: [ADVANCED-DOTNET] Excess memory usage growth for a
>> >>>> process
>> >>>> >
>> >>>> > Hi guys,
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > I need your help with an issue I am having right now. I have
> several
>> >>>> windows
>> >>>> > services deployed. These services gather some XML files from =
the
>> >>>> internet
>> >>>> > and then parse the content and save the information in the
> database.
>> >>> The
>> >>>> > database is Oracle 11g, so the access is done through ODP.NET. =
I
>> had
>> >>>> some
>> >>>> > issues before (and some of you with good memory may recall my
>> previous
>> >>>> > questions) where I noticed the memory used by these services
> growing
>> >>>> very
>> >>>> > high. At the time, I understood that some classes in the =
ODP.NET
>> >>> library
>> >>>> use
>> >>>> > some JAVA code and they are known for not cleaning properly =
after
>> >>> them.
>> >>>> As a
>> >>>> > result I rewrote my code in such a way that every dynamic
> allocation
>> >>> of
>> >>>> an
>> >>>> > object from the ODP.NET library was encompassed in an "using"
>> >>> statement,
>> >>>> > like this:
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > using (OracleCommand objDbCommand =3D new
>> >>>> > OracleCommand(strSqlSelectStatement,
>> >>>> > objConnection))
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > All was fine for a very long time. All my services kept a =
constant
>> >>> size
>> >>>> when
>> >>>> > it came to the total memory used, a size that hovered around =
55-60
>> MB.
>> >>>> This
>> >>>> > was the case for months. At the beginning I checked the size
almost
>> on
>> >>> a
>> >>>> > daily basis. After a while, my confidence grew and I only =
checked
> it
>> >>>> once a
>> >>>> > week or so. Every time I looked at the size of all my =
processes (I
>> had
>> >>> 4
>> >>>> > that followed pretty much the same approach on dealing with =
those
>> >>> Oracle
>> >>>> > objects) the size was around that 55-60 MB mark.
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > Now, all of a sudden, this last weekend, the size for some of =
the
>> >>>> services
>> >>>> > shut up to about 690 MB, despite the fact the service was =
running
>> with
>> >>>> the
>> >>>> > exact same code as it did a month ago. This represents a =
potential
>> >>> fatal
>> >>>> > problem for the server that can get to a point where it can no
>> longer
>> >>>> > allocate memory, so I need to find a way to understand what =
the
> heck
>> >>> is
>> >>>> > going on. Unfortunately, I do not have any metrics implemented =
in
> my
>> >>>> > application to log the total amount of memory used. That would
have
>> >>> been
>> >>>> a
>> >>>> > very good indication on when exactly did this surge started to
>> happen.
>> >>> I
>> >>>> > could have checked the Event Viewer to see if the server got =
hit
by
>> a
>> >>>> > nuclear bomb or something on that precise moment. I really =
don't
> get
>> >>>> this. I
>> >>>> > mean that's why we moved to .NET in the first place, to be rid =
of
>> such
>> >>>> > concerns, like making sure all dynamically allocated objects =
are
>> then
>> >>>> > destroyed.
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > Can anyone give me an idea on how to investigate this issue? =
Are
>> there
>> >>>> tools
>> >>>> > out there that could be used to pick at the memory allocated =
by a
>> >>>> process
>> >>>> > and see what are those objects that take up so much =A0memory? =
For
> the
>> >>>> time
>> >>>> > being, I have the processes running, but two of them are at or
> close
>> >>> to
>> >>>> 690
>> >>>> > MB and I don't know how long will I be able to keep them =
running.
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > If you had an issue like this, what would you try to do?
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > Any suggestion will be greatly appreciated.
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > TIA,
>> >>>> >
>> >>>> > Eddie
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >>>> > View archives and manage your subscription(s) at
>> >>>> > http://peach.ease.lsoft.com/archives
>> >>>>
>> >>>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >>>> View archives and manage your subscription(s) at
>> >>>> http://peach.ease.lsoft.com/archives
>> >>>>
>> >>>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >>>> View archives and manage your subscription(s) at
>> >>>> http://peach.ease.lsoft.com/archives
>> >>>
>> >>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >>> View archives and manage your subscription(s) at
>> >>> http://peach.ease.lsoft.com/archives
>> >>>
>> >>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >>> View archives and manage your subscription(s) at
>> >> http://peach.ease.lsoft.com/archives
>> >>>
>> >>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >>> View archives and manage your subscription(s) at
>> >> http://peach.ease.lsoft.com/archives
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour =
m'assurer
>> >> de votre attention
>> >>
>> >> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> View archives and manage your subscription(s) at
>> >> http://peach.ease.lsoft.com/archives
>> >>
>> >> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> View archives and manage your subscription(s) at
>> > http://peach.ease.lsoft.com/archives
>> >>
>> >
>> >
>> >
>> > --
>> > Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour =
m'assurer
>> > de votre attention
>> >
>> > =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> > View archives and manage your subscription(s) at
>> > http://peach.ease.lsoft.com/archives
>> >
>> > =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> > View archives and manage your subscription(s) at
>> http://peach.ease.lsoft.com/archives
>> >
>>
>>
>>
>> --
>> Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour =
m'assurer
>> de votre attention
>>
>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> View archives and manage your subscription(s) at
>> http://peach.ease.lsoft.com/archives
>>
>
> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> View archives and manage your subscription(s) at
> http://peach.ease.lsoft.com/archives
>
> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives
>
> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives
>



--=20
Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour =
m'assurer
de votre attention

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives