Re: Gradual Growth of Memory Use?
"David McClain (as dbm at refined-audiometrics dot com)" <[email protected]>
| Newsgroups | gmane.lisp.lispworks.general |
|---|---|
| Message-ID | <[email protected]> |
The old chart recorder code was taking a quad of measurements and timestamp as a &REST list, and pushing onto the telemetry list. Lists of lists of numbers. So pretty inefficient. Now we have 4 parallel arrays using fill-pointers and the data values are appended where the fill-pointer sits. So no more need to REVERSE anything, no more need to SECOND, THIRD, etc to reach individual values. Still have to SUBSEQ for most plotting (no need to see everything unless specifically requested with SHOW-ALL or REGRESS-ALL). > On Mar 13, 2025, at 12:59, David McClain <[email protected]> wrote: > > My analysis processing times have dropped pretty significantly. I wrote the chart recorder code maybe 10 years ago, and while rewriting for the quad arrays I noticed that the old code was fetching data by first REVERSE of the accumulated list, then coercing to vector a bunch of subselections of the list. So lots of messy CONSing and list ops that are no longer needed. > > All that extra nonsense is now gone from the analysis timings. > > > <PastedGraphic-1.png> > >> On Mar 13, 2025, at 12:55, David McClain <[email protected]> wrote: >> >> Not sure I can answer that. I use a lot of (tons of vectors) ephemeral arrays. The GC probably allows the system to grow with garbage until some threshold is passed? >> >> After 35 minutes we are now passing 196 MB of allocation to LWW. In-use memory shows 5 GB (0 compressed) out of 32 GB total. >> >> >>> On Mar 13, 2025, at 12:38, Yuri Davidovsky (as work at disclosure dot ie) <[email protected]> wrote: >>> >>> >>> >>>> On 13 Mar 2025, at 20:32, David McClain <[email protected]> wrote: >>>> >>>> Made the change from a list of quads (timestamp, val, filtered val, snr) to 4 typed arrays with a fill-pointer. Maximal size prealloc of 8192 points (about 34 hours). >>> >>> It doesn’t look like it would need that much memory altogether. >>> >>>> So far so good. Memory use has climbed from about 120 MB to now 144 MB after about 12 minutes. >>> >>> So what exactly is using this amount of 24 megabytes after 12 minutes? >>> >>> _______________________________________________ >>> Lisp Hug - the mailing list for LispWorks users >>> [email protected] >>> http://www.lispworks.com/support/lisp-hug.html >> > _______________________________________________ Lisp Hug - the mailing list for LispWorks users [email protected] http://www.lispworks.com/support/lisp-hug.html