Re: Gradual Growth of Memory Use?

Heh! 

Many of the collisions are coming from the eventual JOIN of a FORK’ed series of Tasks. That might be expected in some cases.

But a huge number of collisions come from the collision collector itself. Talk about an intrusive telemetry system - self defeating. 

This might be one place for a MP:LOCK.

> On Mar 19, 2025, at 08:53, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote:
> 
> I was keeping the collision messages, which themselves contain references to the delivery target Actor. This is useful for tracking down the source of collisions.
> 
> 
> 
>> On Mar 19, 2025, at 08:51, David McClain <[email protected]> wrote:
>> 
>> AHA!!
>> 
>> I found the culprit. I had a telemetry collection down in the Actor Dispatch mechanism to detect and count the number of BECOME collisions - parallel tasks attempting to BECOME inside of any one Actor behavior, which allows only one of them to succeed and forces a retry of message delivery for the losers. 
>> 
>> This is an interesting problem for parallel concurrency. Some Actor systems allow only one thread of execution inside any one Actor behavior. Mine allows full parallel execution.
>> 
>> At any rate, I zapped the telemetry queue, did another (GC-GENERATION 7), and now Tim Dump shows only 588 Actors. That sounds more reasonable.
>> 
>> - DM
>> 
>> 
>> 
>>> On Mar 19, 2025, at 08:44, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote:
>>> 
>>> Here is the Tim Dump after (GC-GENERATION 7) and now having run for 7 hours.
>>> 
>>> Now shows 650,000 Actors accumulated !!?? Something is clearly not working as I would have expected….
>>> 
>>> (#S(CLASS-COUNTER :NAME CONS
>>>                :COUNT 11538098
>>>                :TOTAL-SIZE 184609568
>>>                :CUMULATIVE-SIZE 409114392)
>>> #S(CLASS-COUNTER :NAME FUNCTION
>>>                :COUNT 1984077
>>>                :TOTAL-SIZE 88266264
>>>                :CUMULATIVE-SIZE 224504824)
>>> #S(CLASS-COUNTER :NAME SIMPLE-VECTOR
>>>                :COUNT 2981570
>>>                :TOTAL-SIZE 88202744
>>>                :CUMULATIVE-SIZE 136238560)
>>> #S(CLASS-COUNTER :NAME COM.RAL.ACTORS:ACTOR
>>>                :COUNT 651265
>>>                :TOTAL-SIZE 15630360
>>>                :CUMULATIVE-SIZE 48035816)
>>> #S(CLASS-COUNTER :NAME CLOS::SYSTEM-OBJECT
>>>                :COUNT 125668
>>>                :TOTAL-SIZE 7476728
>>>                :CUMULATIVE-SIZE 32405456)
>>> #S(CLASS-COUNTER :NAME SIMPLE-STRING
>>>                :COUNT 22364
>>>                :TOTAL-SIZE 5244112
>>>                :CUMULATIVE-SIZE 24928728)
>>> #S(CLASS-COUNTER :NAME VECTOR
>>>                :COUNT 3100
>>>                :TOTAL-SIZE 5132048
>>>                :CUMULATIVE-SIZE 19684616)
>>> #S(CLASS-COUNTER :NAME SYMBOL
>>>                :COUNT 103353
>>>                :TOTAL-SIZE 4960944
>>>                :CUMULATIVE-SIZE 14552568)
>>> #S(CLASS-COUNTER :NAME SIMPLE-BASE-STRING
>>>                :COUNT 126148
>>>                :TOTAL-SIZE 3816784
>>>                :CUMULATIVE-SIZE 9591624)
>>> #S(CLASS-COUNTER :NAME EDITOR::TEXT-PROPERTY-POINT
>>>                :COUNT 8710
>>>                :TOTAL-SIZE 696800
>>>                :CUMULATIVE-SIZE 5774840)
>>> #S(CLASS-COUNTER :NAME DOUBLE-FLOAT
>>>                :COUNT 41393
>>>                :TOTAL-SIZE 662288
>>>                :CUMULATIVE-SIZE 5078040)
>>> 
>>> 
>>>> On Mar 19, 2025, at 08:24, Martin Simmons <[email protected]> wrote:
>>>> 
>>>> It would be useful to try again after (GC-GENERATION 7), because that can
>>>> sometimes collect more objects in generation 3 than (GC-GENERATION 3) can.
>>>> 
>>>> -- 
>>>> Martin Simmons
>>>> LispWorks Ltd
>>>> http://www.lispworks.com/
>>>> 
>>>> 
>>>> 
>>>>>>>>> On Wed, 19 Mar 2025 06:50:20 -0700, David McClain said:
>>>>> 
>>>>> I just did a (GC-GENERATION 3), about half-hour after the last dump. Here is
>>>>> the new dump: (Now shows 460,000 Actors, up from 342,000 !!??)
>>>>> 
>>>>> (#S(CLASS-COUNTER :NAME CONS
>>>>>               :COUNT 8821834
>>>>>               :TOTAL-SIZE 141149344
>>>>>               :CUMULATIVE-SIZE 324811016)
>>>>> #S(CLASS-COUNTER :NAME FUNCTION
>>>>>               :COUNT 1456012
>>>>>               :TOTAL-SIZE 71371256
>>>>>               :CUMULATIVE-SIZE 183661672)
>>>>> #S(CLASS-COUNTER :NAME SIMPLE-VECTOR
>>>>>               :COUNT 2221920
>>>>>               :TOTAL-SIZE 69226000
>>>>>               :CUMULATIVE-SIZE 112290416)
>>>>> #S(CLASS-COUNTER :NAME COM.RAL.ACTORS:ACTOR
>>>>>               :COUNT 459690
>>>>>               :TOTAL-SIZE 11032560
>>>>>               :CUMULATIVE-SIZE 43064416)
>>>>> #S(CLASS-COUNTER :NAME CLOS::SYSTEM-OBJECT
>>>>>               :COUNT 127772
>>>>>               :TOTAL-SIZE 7416080
>>>>>               :CUMULATIVE-SIZE 32031856)
>>>>> #S(CLASS-COUNTER :NAME VECTOR
>>>>>               :COUNT 3110
>>>>>               :TOTAL-SIZE 5701968
>>>>>               :CUMULATIVE-SIZE 24615776)
>>>>> #S(CLASS-COUNTER :NAME SIMPLE-STRING
>>>>>               :COUNT 22319
>>>>>               :TOTAL-SIZE 5269464
>>>>>               :CUMULATIVE-SIZE 18913808)
>>>>> #S(CLASS-COUNTER :NAME SYMBOL
>>>>>               :COUNT 103353
>>>>>               :TOTAL-SIZE 4960944
>>>>>               :CUMULATIVE-SIZE 13644344)
>>>>> #S(CLASS-COUNTER :NAME SIMPLE-BASE-STRING
>>>>>               :COUNT 126192
>>>>>               :TOTAL-SIZE 3817624
>>>>>               :CUMULATIVE-SIZE 8683400)
>>>>> #S(CLASS-COUNTER :NAME DOUBLE-FLOAT
>>>>>               :COUNT 27586
>>>>>               :TOTAL-SIZE 441376
>>>>>               :CUMULATIVE-SIZE 4865776)
>>>>> 
>>>>> 
>>>>>> On Mar 19, 2025, at 06:43, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On Mar 19, 2025, at 06:02, Martin Simmons <[email protected]> wrote:
>>>>>>> 
>>>>>>> I suggest comparing those counters with another call to size-by-class/loop
>>>>>>> when the image has grown some more.  After that, you could do (gc-generation
>>>>>>> 3) and compare again to see if the objects were really live or just stuck in
>>>>>>> generation 3.
>>>>>> 
>>>>>> 
>>>>>> So, from your suggestion, it would make most sense to perform (gc-generation 3) before doing the memory dump from Tim? Allow the system to discard the accumulated garbage first? Then compare dumps from two separated times.
>>>>>> 
>>>>>> My experience running (gc-generation 3) in the past has shown that the memory image is indeed increasing. The results of the call do keep increasing, which would indicate memory actually in-use.
>>>> 
>>>> _______________________________________________
>>>> Lisp Hug - the mailing list for LispWorks users
>>>> [email protected]
>>>> http://www.lispworks.com/support/lisp-hug.html
>>> 
>>> 
>>> _______________________________________________
>>> Lisp Hug - the mailing list for LispWorks users
>>> [email protected]
>>> http://www.lispworks.com/support/lisp-hug.html
>> 
> 
> 
> _______________________________________________
> Lisp Hug - the mailing list for LispWorks users
> [email protected]
> http://www.lispworks.com/support/lisp-hug.html


_______________________________________________
Lisp Hug - the mailing list for LispWorks users
[email protected]
http://www.lispworks.com/support/lisp-hug.html