Re: Gradual Growth of Memory Use?
"David McClain (as dbm at refined-audiometrics dot com)" <[email protected]>
| Newsgroups | gmane.lisp.lispworks.general |
|---|---|
| Message-ID | <[email protected]> |
Memory use has now leveled off to around 180-200 MB. Explorer.exe continues to climb. That is a known Win/11 irritation. > On Mar 19, 2025, at 09:23, David McClain <[email protected]> wrote: > > I replaced the telemetry with a simple SYS:ATOMIC-INCF. Now seeing about 1.5 collisions per sec, or 50K per hour. A completely acceptable level. > > > >> On Mar 19, 2025, at 09:00, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote: >> >> Heh! >> >> Many of the collisions are coming from the eventual JOIN of a FORK’ed series of Tasks. That might be expected in some cases. >> >> But a huge number of collisions come from the collision collector itself. Talk about an intrusive telemetry system - self defeating. >> >> This might be one place for a MP:LOCK. >> >>> On Mar 19, 2025, at 08:53, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote: >>> >>> I was keeping the collision messages, which themselves contain references to the delivery target Actor. This is useful for tracking down the source of collisions. >>> >>> >>> >>>> On Mar 19, 2025, at 08:51, David McClain <[email protected]> wrote: >>>> >>>> AHA!! >>>> >>>> I found the culprit. I had a telemetry collection down in the Actor Dispatch mechanism to detect and count the number of BECOME collisions - parallel tasks attempting to BECOME inside of any one Actor behavior, which allows only one of them to succeed and forces a retry of message delivery for the losers. >>>> >>>> This is an interesting problem for parallel concurrency. Some Actor systems allow only one thread of execution inside any one Actor behavior. Mine allows full parallel execution. >>>> >>>> At any rate, I zapped the telemetry queue, did another (GC-GENERATION 7), and now Tim Dump shows only 588 Actors. That sounds more reasonable. >>>> >>>> - DM >>>> >>>> >>>> >>>>> On Mar 19, 2025, at 08:44, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote: >>>>> >>>>> Here is the Tim Dump after (GC-GENERATION 7) and now having run for 7 hours. >>>>> >>>>> Now shows 650,000 Actors accumulated !!?? Something is clearly not working as I would have expected…. >>>>> >>>>> (#S(CLASS-COUNTER :NAME CONS >>>>> :COUNT 11538098 >>>>> :TOTAL-SIZE 184609568 >>>>> :CUMULATIVE-SIZE 409114392) >>>>> #S(CLASS-COUNTER :NAME FUNCTION >>>>> :COUNT 1984077 >>>>> :TOTAL-SIZE 88266264 >>>>> :CUMULATIVE-SIZE 224504824) >>>>> #S(CLASS-COUNTER :NAME SIMPLE-VECTOR >>>>> :COUNT 2981570 >>>>> :TOTAL-SIZE 88202744 >>>>> :CUMULATIVE-SIZE 136238560) >>>>> #S(CLASS-COUNTER :NAME COM.RAL.ACTORS:ACTOR >>>>> :COUNT 651265 >>>>> :TOTAL-SIZE 15630360 >>>>> :CUMULATIVE-SIZE 48035816) >>>>> #S(CLASS-COUNTER :NAME CLOS::SYSTEM-OBJECT >>>>> :COUNT 125668 >>>>> :TOTAL-SIZE 7476728 >>>>> :CUMULATIVE-SIZE 32405456) >>>>> #S(CLASS-COUNTER :NAME SIMPLE-STRING >>>>> :COUNT 22364 >>>>> :TOTAL-SIZE 5244112 >>>>> :CUMULATIVE-SIZE 24928728) >>>>> #S(CLASS-COUNTER :NAME VECTOR >>>>> :COUNT 3100 >>>>> :TOTAL-SIZE 5132048 >>>>> :CUMULATIVE-SIZE 19684616) >>>>> #S(CLASS-COUNTER :NAME SYMBOL >>>>> :COUNT 103353 >>>>> :TOTAL-SIZE 4960944 >>>>> :CUMULATIVE-SIZE 14552568) >>>>> #S(CLASS-COUNTER :NAME SIMPLE-BASE-STRING >>>>> :COUNT 126148 >>>>> :TOTAL-SIZE 3816784 >>>>> :CUMULATIVE-SIZE 9591624) >>>>> #S(CLASS-COUNTER :NAME EDITOR::TEXT-PROPERTY-POINT >>>>> :COUNT 8710 >>>>> :TOTAL-SIZE 696800 >>>>> :CUMULATIVE-SIZE 5774840) >>>>> #S(CLASS-COUNTER :NAME DOUBLE-FLOAT >>>>> :COUNT 41393 >>>>> :TOTAL-SIZE 662288 >>>>> :CUMULATIVE-SIZE 5078040) >>>>> >>>>> >>>>>> On Mar 19, 2025, at 08:24, Martin Simmons <[email protected]> wrote: >>>>>> >>>>>> It would be useful to try again after (GC-GENERATION 7), because that can >>>>>> sometimes collect more objects in generation 3 than (GC-GENERATION 3) can. >>>>>> >>>>>> -- >>>>>> Martin Simmons >>>>>> LispWorks Ltd >>>>>> http://www.lispworks.com/ >>>>>> >>>>>> >>>>>> >>>>>>>>>>> On Wed, 19 Mar 2025 06:50:20 -0700, David McClain said: >>>>>>> >>>>>>> I just did a (GC-GENERATION 3), about half-hour after the last dump. Here is >>>>>>> the new dump: (Now shows 460,000 Actors, up from 342,000 !!??) >>>>>>> >>>>>>> (#S(CLASS-COUNTER :NAME CONS >>>>>>> :COUNT 8821834 >>>>>>> :TOTAL-SIZE 141149344 >>>>>>> :CUMULATIVE-SIZE 324811016) >>>>>>> #S(CLASS-COUNTER :NAME FUNCTION >>>>>>> :COUNT 1456012 >>>>>>> :TOTAL-SIZE 71371256 >>>>>>> :CUMULATIVE-SIZE 183661672) >>>>>>> #S(CLASS-COUNTER :NAME SIMPLE-VECTOR >>>>>>> :COUNT 2221920 >>>>>>> :TOTAL-SIZE 69226000 >>>>>>> :CUMULATIVE-SIZE 112290416) >>>>>>> #S(CLASS-COUNTER :NAME COM.RAL.ACTORS:ACTOR >>>>>>> :COUNT 459690 >>>>>>> :TOTAL-SIZE 11032560 >>>>>>> :CUMULATIVE-SIZE 43064416) >>>>>>> #S(CLASS-COUNTER :NAME CLOS::SYSTEM-OBJECT >>>>>>> :COUNT 127772 >>>>>>> :TOTAL-SIZE 7416080 >>>>>>> :CUMULATIVE-SIZE 32031856) >>>>>>> #S(CLASS-COUNTER :NAME VECTOR >>>>>>> :COUNT 3110 >>>>>>> :TOTAL-SIZE 5701968 >>>>>>> :CUMULATIVE-SIZE 24615776) >>>>>>> #S(CLASS-COUNTER :NAME SIMPLE-STRING >>>>>>> :COUNT 22319 >>>>>>> :TOTAL-SIZE 5269464 >>>>>>> :CUMULATIVE-SIZE 18913808) >>>>>>> #S(CLASS-COUNTER :NAME SYMBOL >>>>>>> :COUNT 103353 >>>>>>> :TOTAL-SIZE 4960944 >>>>>>> :CUMULATIVE-SIZE 13644344) >>>>>>> #S(CLASS-COUNTER :NAME SIMPLE-BASE-STRING >>>>>>> :COUNT 126192 >>>>>>> :TOTAL-SIZE 3817624 >>>>>>> :CUMULATIVE-SIZE 8683400) >>>>>>> #S(CLASS-COUNTER :NAME DOUBLE-FLOAT >>>>>>> :COUNT 27586 >>>>>>> :TOTAL-SIZE 441376 >>>>>>> :CUMULATIVE-SIZE 4865776) >>>>>>> >>>>>>> >>>>>>>> On Mar 19, 2025, at 06:43, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Mar 19, 2025, at 06:02, Martin Simmons <[email protected]> wrote: >>>>>>>>> >>>>>>>>> I suggest comparing those counters with another call to size-by-class/loop >>>>>>>>> when the image has grown some more. After that, you could do (gc-generation >>>>>>>>> 3) and compare again to see if the objects were really live or just stuck in >>>>>>>>> generation 3. >>>>>>>> >>>>>>>> >>>>>>>> So, from your suggestion, it would make most sense to perform (gc-generation 3) before doing the memory dump from Tim? Allow the system to discard the accumulated garbage first? Then compare dumps from two separated times. >>>>>>>> >>>>>>>> My experience running (gc-generation 3) in the past has shown that the memory image is indeed increasing. The results of the call do keep increasing, which would indicate memory actually in-use. >>>>>> >>>>>> _______________________________________________ >>>>>> Lisp Hug - the mailing list for LispWorks users >>>>>> [email protected] >>>>>> http://www.lispworks.com/support/lisp-hug.html >>>>> >>>>> >>>>> _______________________________________________ >>>>> Lisp Hug - the mailing list for LispWorks users >>>>> [email protected] >>>>> http://www.lispworks.com/support/lisp-hug.html >>>> >>> >>> >>> _______________________________________________ >>> Lisp Hug - the mailing list for LispWorks users >>> [email protected] >>> http://www.lispworks.com/support/lisp-hug.html >> >> >> _______________________________________________ >> Lisp Hug - the mailing list for LispWorks users >> [email protected] >> http://www.lispworks.com/support/lisp-hug.html > _______________________________________________ Lisp Hug - the mailing list for LispWorks users [email protected] http://www.lispworks.com/support/lisp-hug.html