Re: Excess memory usage growth for a process

> I believe this clearly indicates that there is no memory leak per se, but
> rather is just that the large objects heap is growing w/o control.
> Is my assessment correct?

Private bytes is just how much memory your application is using that
can't be accessed by other applications.

Why does this clearly indicate anything? much less that its an "out of
control LOH". What is the size of your LOH in your memory dump? What
objects are on it? How are those objects rooted?

Cheers,

Greg

On Thu, Aug 6, 2009 at 3:35 PM, Eddie Lascu<[email protected]> wrote:
> Hi Folks,
>
> So I have been running this Red Gate Memory Profiler for more than 8 days
> now and I can see some very interesting developments. The Profiler is
> monitoring some of the Performance Counters. One in particular has caught=
 my
> interest and that is the size of Private Bytes. Although steady for the
> first two days at 48 MB, after that it started to grow slowly and what wa=
s a
> flat line before, turned into a ascending incline ramp. At the moment I t=
ype
> this message, the size of the Private Bytes is 855 MB. In the task manage=
r,
> my process is indicated as having the Virtual Memory size up to 876 MB wh=
ile
> the Mem Usage is only 64 MB.
> I believe this clearly indicates that there is no memory leak per se, but
> rather is just that the large objects heap is growing w/o control.
> Is my assessment correct?
>
> Regards,
> Eddie
>
>
> -----Original Message-----
> From: Discussion of advanced .NET topics.
> [mailto:[email protected]] On Behalf Of Trey Nash
> Sent: Thursday, July 30, 2009 2:24 PM
> To: [email protected]
> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
>
> Hi,
>
> Yes, ADPlus is really just a complex VB script that drive windb(a)g. =A0I=
f you
> feed it the -? option, it should spew out a lot of helpful info on the
> various ways you can use it.
>
> I wrote a blog post on our team blog not too long ago regarding ADPlus th=
at
> may be helpful to become more familiar with it:
>
> http://blogs.msdn.com/ntdebugging/archive/2009/05/18/capturing-adplus-clr=
-cr
> ashes.aspx
>
> I hope this helps,
>
> =A0 =A0 =A0 =A0-Trey
>
>> -----Original Message-----
>> From: Discussion of advanced .NET topics. [mailto:ADVANCED-
>> [email protected]] On Behalf Of Eddie Lascu
>> Sent: Thursday, July 30, 2009 1:04 PM
>> To: [email protected]
>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
>>
>> Thank you, Guys. I will look into that as well. Do I need a special
>> start of
>> the process if I am to use ADPlus or I can attach to the process later?
>>
>>
>> -----Original Message-----
>> From: Discussion of advanced .NET topics.
>> [mailto:[email protected]] On Behalf Of Greg Young
>> Sent: Thursday, July 30, 2009 1:52 PM
>> To: [email protected]
>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
>>
>> Yep exactly what ADPlus is meant for! you can even do this right
>> within ADPlus :)
>>
>> On Thu, Jul 30, 2009 at 1:49 PM, Wilson,
>> Phil<[email protected]>
>> wrote:
>> > Not answering this particular question, but I believe you could hook
>> all
>> this together to automatically get that dump when the memory gets large
>> if
>> it's rare enough that you don't want to sit and look at it all day.
>> Perfmon
>> has triggers that can fire off a script, and you can fire off the
>> debugging
>> script ADPlus to dump your process with the -hang option.
>> >
>> > Phil Wilson
>> >
>> >
>> > -----Original Message-----
>> > From: Discussion of advanced .NET topics.
>> [mailto:[email protected]] On Behalf Of Eddie Lascu
>> > Sent: Wednesday, July 29, 2009 11:40 AM
>> > To: [email protected]
>> > Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
>> process
>> >
>> > Based on my preliminary reading of what this has to offer, it seems
>> you
>> can
>> > integrate it with your VS IDE (not an option for me as my processes
>> are
>> > already deployed) or use it via WinDbg, but you need to look at the
>> debug
>> > version or the runtime version that has the .pdb file (not an option
>> either
>> > as I only have the sole runtime file deployed, but much easily
>> fixable).
>> Is
>> > this statement accurate?
>> >
>> > Thanks,
>> > Eddie
>> >
>> > -----Original Message-----
>> > From: Discussion of advanced .NET topics.
>> > [mailto:[email protected]] On Behalf Of Adam
>> Tuliper
>> > Sent: Wednesday, July 29, 2009 1:24 PM
>> > To: [email protected]
>> > Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
>> process
>> >
>> >
>> http://blogs.msdn.com/alejacma/archive/2009/06/30/sos-cheat-sheet-net-
>> 2-0-3-
>> > 0-3-5.aspx
>> >
>> > !DumpHeap -min 85000
>> > !EEHeap -GC; !DumpHeap <LOH begin> <LOH allocated>
>> > =A0 =A0-> Shows all objects in Large Object Heap (LOH/Gen 3).
>> >
>> >
>> >
>> >
>> > On Wed, Jul 29, 2009 at 1:06 PM, Greg Young <[email protected]>
>> wrote:
>> >
>> >> Yes with ants you have to wait for the problem to happen.
>> >>
>> >> if you google SOS you can find some tutorials on it (it is a
>> separate
>> >> download from windbg ... it can also be used in visual studio!).
>> John
>> >> Robbins also has a great book covering the subject that I highly
>> >> recommend.
>> >>
>> >> Cheers,
>> >>
>> >> greg
>> >>
>> >> On Wed, Jul 29, 2009 at 1:01 PM, Eddie Lascu<[email protected]>
>> wrote:
>> >> > Shoot, I didn't know SOS can do that for you. Can you share more
>> about
>> >> this
>> >> > tool (a link to download it)? Is it part of the WinDbg tool that
>> comes
>> >> from
>> >> > Microsoft? It's true that with ANTS I am at the mercy of the
>> system,
>> >> hoping
>> >> > it will go nuts again and will have the memory size shut up crazy
>> high.
>> >> >
>> >> > Cheers,
>> >> > Eddie
>> >> >
>> >> >
>> >> >
>> >> > -----Original Message-----
>> >> > From: Discussion of advanced .NET topics.
>> >> > [mailto:[email protected]] On Behalf Of Greg
>> Young
>> >> > Sent: Wednesday, July 29, 2009 12:46 PM
>> >> > To: [email protected]
>> >> > Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
>> process
>> >> >
>> >> > The wonderful thing about SOS is you dont need to run a profiler
>> ...
>> >> > when it happens take a memory dump, look whats actually happening
>> in
>> >> > the running app, is your LOH fragmented? Otherwise you are just
>> >> > sticking your thumb up your butt and saying it will rain next
>> >> > thursday.
>> >> >
>> >> > On Wed, Jul 29, 2009 at 12:43 PM, Eddie Lascu<[email protected]>
>> >> wrote:
>> >> >> I am using Reg Gate ANTS to profile the app, but it may take too
>> much
>> > to
>> >> >> happen again. Before the spike I saw last weekend, the processes
>> ran
>> > for
>> >> > 3-4
>> >> >> months with no significant change in the size of memory used.
>> >> >> Will look into Process.WorkingSet.
>> >> >>
>> >> >> Thanks,
>> >> >> Eddie
>> >> >>
>> >> >>
>> >> >>
>> >> >> -----Original Message-----
>> >> >> From: Discussion of advanced .NET topics.
>> >> >> [mailto:[email protected]] On Behalf Of Greg
>> Young
>> >> >> Sent: Wednesday, July 29, 2009 12:38 PM
>> >> >> To: [email protected]
>> >> >> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
>> process
>> >> >>
>> >> >> Task manager shows you working set not total memory used but yes
>> the
>> >> >> process class can help you with that (Process.WorkingSet as an
>> >> >> example).
>> >> >>
>> >> >> Have you verified the LOH being the problem by using SOS?
>> >> >>
>> >> >> Cheers,
>> >> >>
>> >> >> Greg
>> >> >>
>> >> >> On Wed, Jul 29, 2009 at 12:33 PM, Eddie
>> Lascu<[email protected]>
>> >> wrote:
>> >> >>> Hi guys,
>> >> >>>
>> >> >>> In light of this article:
>> >> >>>
>> >> >>>
>> >> >>
>> >> >
>> >>
>> >
>> http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-
>> large-ob
>> >> >> ject-heap/
>> >> >>>
>> >> >>> I am no longer sure what I have is a memory leak. I could see
>> how
>> some
>> >> of
>> >> >> the things that happen in my application match the scenarios
>> described
>> >> in
>> >> >> the above article. Hence, I am ready to think about a workaround.
>> My
>> >> >> services are set up to run automatically and restart in case they
>> > crash.
>> >> >> While they never crash, I could leverage that feature and use it
>> to
>> fix
>> >> > this
>> >> >> excess memory usage that I experience every once in a blue moon.
>> What
>> I
>> >> >> thought was to monitor the memory used and terminate the
>> application
>> in
>> >> > case
>> >> >> it exceeds a set threshold. The Windows Service Manager will then
>> >> restart
>> >> >> the service and all the memory will be reclaimed by the OS.
>> >> >>>
>> >> >>> My question at this point is whether the .NET Framework has some
>> > metric
>> >> >> classes that can help monitor the total amount of memory
>> allocated
>> >> within
>> >> > a
>> >> >> process. Surely System.Diagnostics must have something like that.
>> Is
>> >> > anyone
>> >> >> aware of a way to find out what memory is used by a process, just
>> like
>> >> the
>> >> >> Task Manager does?
>> >> >>>
>> >> >>> Cheers,
>> >> >>> Eddie
>> >> >>>
>> >> >>>
>> >> >>> -----Original Message-----
>> >> >>> From: Discussion of advanced .NET topics.
>> >> >> [mailto:[email protected]] On Behalf Of Shawn
>> >> > Wildermuth
>> >> >>> Sent: Monday, July 27, 2009 12:49 PM
>> >> >>> To: [email protected]
>> >> >>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
>> > process
>> >> >>>
>> >> >>> Also, your 'using' statement stuff should help, but are you sure
>> that
>> >> all
>> >> >>> your object (XML classes and db connection classes) that support
>> >> >> IDisposable
>> >> >>> are being cleaned up?
>> >> >>>
>> >> >>> Thanks,
>> >> >>>
>> >> >>> Shawn Wildermuth
>> >> >>> http://wildermuth.com
>> >> >>> https://agilitrain.com
>> >> >>> Microsoft MVP (C#), MCSD.NET, Author and Speaker
>> >> >>>
>> >> >>> The Silverlight Tour is coming to a city near you!
>> >> >>>
>> >> >>>
>> >> >>> -----Original Message-----
>> >> >>> From: Discussion of advanced .NET topics.
>> >> >>> [mailto:[email protected]] On Behalf Of John
>> Warner
>> >> >>> Sent: Monday, July 27, 2009 12:28 PM
>> >> >>> To: [email protected]
>> >> >>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
>> > process
>> >> >>>
>> >> >>> I don't disagree with you just note that before 'some change'
>> working
>> >> OK
>> >> >>> no memory leaks, after change, memory leak. We both know your
>> code
>> did
>> >> > not
>> >> >>> recompile itself to include a memory leak ...
>> >> >>> That reminds has the framework/runtimes been updated/patched?
>> >> >>>
>> >> >>> John Warner
>> >> >>>
>> >> >>>
>> >> >>>> -----Original Message-----
>> >> >>>> From: Discussion of advanced .NET topics. [mailto:ADVANCED-
>> >> >>>> [email protected]] On Behalf Of Eddie Lascu
>> >> >>>> Sent: Monday, July 27, 2009 12:00 PM
>> >> >>>> To: [email protected]
>> >> >>>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
>> >> >>>> process
>> >> >>>>
>> >> >>>> Hi John,
>> >> >>>>
>> >> >>>> I could check that with our system administrators. Do you think
>> that
>> >> can
>> >> >>> be
>> >> >>>> a possible explanation?
>> >> >>>> I just talked with one of the DB Admin and she said she made
>> some
>> >> >>> changes
>> >> >>>> on
>> >> >>>> some Stored Produces, but nothing as dramatic as a patch
>> update. She
>> >> >>> also
>> >> >>>> said that the drivers for the network cards were updated last
>> week,
>> >> but
>> >> >>> I
>> >> >>>> can't see how that can affect the memory used by my processes.
>> >> >>>>
>> >> >>>> Thanks,
>> >> >>>> Eddie
>> >> >>>>
>> >> >>>>
>> >> >>>> -----Original Message-----
>> >> >>>> From: Discussion of advanced .NET topics.
>> >> >>>> [mailto:[email protected]] On Behalf Of
>> >> >>>> John Warner
>> >> >>>> Sent: Monday, July 27, 2009 11:49 AM
>> >> >>>> To: [email protected]
>> >> >>>> Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a
>> >> >>>> process
>> >> >>>>
>> >> >>>> Not what you are seeking, but has anything on the server
>> changed? An
>> >> >>>> Oracle patch or something of that nature?
>> >> >>>>
>> >> >>>> John Warner
>> >> >>>>
>> >> >>>>
>> >> >>>> > -----Original Message-----
>> >> >>>> > From: Discussion of advanced .NET topics. [mailto:ADVANCED-
>> >> >>>> > [email protected]] On Behalf Of Eddie Lascu
>> >> >>>> > Sent: Monday, July 27, 2009 11:33 AM
>> >> >>>> > To: [email protected]
>> >> >>>> > Subject: [ADVANCED-DOTNET] Excess memory usage growth for a
>> >> >>>> process
>> >> >>>> >
>> >> >>>> > Hi guys,
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > I need your help with an issue I am having right now. I have
>> > several
>> >> >>>> windows
>> >> >>>> > services deployed. These services gather some XML files from
>> the
>> >> >>>> internet
>> >> >>>> > and then parse the content and save the information in the
>> > database.
>> >> >>> The
>> >> >>>> > database is Oracle 11g, so the access is done through
>> ODP.NET. I
>> >> had
>> >> >>>> some
>> >> >>>> > issues before (and some of you with good memory may recall my
>> >> previous
>> >> >>>> > questions) where I noticed the memory used by these services
>> > growing
>> >> >>>> very
>> >> >>>> > high. At the time, I understood that some classes in the
>> ODP.NET
>> >> >>> library
>> >> >>>> use
>> >> >>>> > some JAVA code and they are known for not cleaning properly
>> after
>> >> >>> them.
>> >> >>>> As a
>> >> >>>> > result I rewrote my code in such a way that every dynamic
>> > allocation
>> >> >>> of
>> >> >>>> an
>> >> >>>> > object from the ODP.NET library was encompassed in an "using"
>> >> >>> statement,
>> >> >>>> > like this:
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > using (OracleCommand objDbCommand =3D new
>> >> >>>> > OracleCommand(strSqlSelectStatement,
>> >> >>>> > objConnection))
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > All was fine for a very long time. All my services kept a
>> constant
>> >> >>> size
>> >> >>>> when
>> >> >>>> > it came to the total memory used, a size that hovered around
>> 55-60
>> >> MB.
>> >> >>>> This
>> >> >>>> > was the case for months. At the beginning I checked the size
>> almost
>> >> on
>> >> >>> a
>> >> >>>> > daily basis. After a while, my confidence grew and I only
>> checked
>> > it
>> >> >>>> once a
>> >> >>>> > week or so. Every time I looked at the size of all my
>> processes (I
>> >> had
>> >> >>> 4
>> >> >>>> > that followed pretty much the same approach on dealing with
>> those
>> >> >>> Oracle
>> >> >>>> > objects) the size was around that 55-60 MB mark.
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > Now, all of a sudden, this last weekend, the size for some of
>> the
>> >> >>>> services
>> >> >>>> > shut up to about 690 MB, despite the fact the service was
>> running
>> >> with
>> >> >>>> the
>> >> >>>> > exact same code as it did a month ago. This represents a
>> potential
>> >> >>> fatal
>> >> >>>> > problem for the server that can get to a point where it can
>> no
>> >> longer
>> >> >>>> > allocate memory, so I need to find a way to understand what
>> the
>> > heck
>> >> >>> is
>> >> >>>> > going on. Unfortunately, I do not have any metrics
>> implemented in
>> > my
>> >> >>>> > application to log the total amount of memory used. That
>> would
>> have
>> >> >>> been
>> >> >>>> a
>> >> >>>> > very good indication on when exactly did this surge started
>> to
>> >> happen.
>> >> >>> I
>> >> >>>> > could have checked the Event Viewer to see if the server got
>> hit
>> by
>> >> a
>> >> >>>> > nuclear bomb or something on that precise moment. I really
>> don't
>> > get
>> >> >>>> this. I
>> >> >>>> > mean that's why we moved to .NET in the first place, to be
>> rid of
>> >> such
>> >> >>>> > concerns, like making sure all dynamically allocated objects
>> are
>> >> then
>> >> >>>> > destroyed.
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > Can anyone give me an idea on how to investigate this issue?
>> Are
>> >> there
>> >> >>>> tools
>> >> >>>> > out there that could be used to pick at the memory allocated
>> by a
>> >> >>>> process
>> >> >>>> > and see what are those objects that take up so much =A0memory?
>> For
>> > the
>> >> >>>> time
>> >> >>>> > being, I have the processes running, but two of them are at
>> or
>> > close
>> >> >>> to
>> >> >>>> 690
>> >> >>>> > MB and I don't know how long will I be able to keep them
>> running.
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > If you had an issue like this, what would you try to do?
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > Any suggestion will be greatly appreciated.
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > TIA,
>> >> >>>> >
>> >> >>>> > Eddie
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> >>>> > View archives and manage your subscription(s) at
>> >> >>>> > http://peach.ease.lsoft.com/archives
>> >> >>>>
>> >> >>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> >>>> View archives and manage your subscription(s) at
>> >> >>>> http://peach.ease.lsoft.com/archives
>> >> >>>>
>> >> >>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> >>>> View archives and manage your subscription(s) at
>> >> >>>> http://peach.ease.lsoft.com/archives
>> >> >>>
>> >> >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> >>> View archives and manage your subscription(s) at
>> >> >>> http://peach.ease.lsoft.com/archives
>> >> >>>
>> >> >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> >>> View archives and manage your subscription(s) at
>> >> >> http://peach.ease.lsoft.com/archives
>> >> >>>
>> >> >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> >>> View archives and manage your subscription(s) at
>> >> >> http://peach.ease.lsoft.com/archives
>> >> >>>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour
>> m'assurer
>> >> >> de votre attention
>> >> >>
>> >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> >> View archives and manage your subscription(s) at
>> >> >> http://peach.ease.lsoft.com/archives
>> >> >>
>> >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> >> View archives and manage your subscription(s) at
>> >> > http://peach.ease.lsoft.com/archives
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour
>> m'assurer
>> >> > de votre attention
>> >> >
>> >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> > View archives and manage your subscription(s) at
>> >> > http://peach.ease.lsoft.com/archives
>> >> >
>> >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> > View archives and manage your subscription(s) at
>> >> http://peach.ease.lsoft.com/archives
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour
>> m'assurer
>> >> de votre attention
>> >>
>> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> >> View archives and manage your subscription(s) at
>> >> http://peach.ease.lsoft.com/archives
>> >>
>> >
>> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> > View archives and manage your subscription(s) at
>> > http://peach.ease.lsoft.com/archives
>> >
>> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> > View archives and manage your subscription(s) at
>> http://peach.ease.lsoft.com/archives
>> >
>> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> > View archives and manage your subscription(s) at
>> http://peach.ease.lsoft.com/archives
>> >
>>
>>
>>
>> --
>> Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour m'assur=
er
>> de votre attention
>>
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> View archives and manage your subscription(s) at
>> http://peach.ease.lsoft.com/archives
>>
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> View archives and manage your subscription(s) at
>> http://peach.ease.lsoft.com/archives
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> View archives and manage your subscription(s) at
> http://peach.ease.lsoft.com/archives
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> View archives and manage your subscription(s) at http://peach.ease.lsoft.=
com/archives
>



--=20
Les erreurs de grammaire et de syntaxe ont =E9t=E9 incluses pour m'assurer
de votre attention

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives