Re: Tracking down memory usage (leak?) in Pike program

I notice that num_objects is WAY smaller in your newly started program.
That means that your program leaks objects somehow.  I suspect that those
objects also have some external data, such as UI elements or events
associated with them, which would explain why you don't see all the data in
object_bytes.  Of course, by "leaked" I mean that they are somehow still
referenced, but not on the screen. This can happen if UI elements have
callbacks to themselves for instance. It's possible that adding descruct()
calls when you remove ui elements could resolve this problem.

The next step is to walk all the objects in memory, get their programs and
see how many instances you have of each program.

    /Hubbe


On Tue, Jul 19, 2016 at 12:23 PM, Chris Angelico <[email protected]> wrote:

> On Wed, Jul 20, 2016 at 4:28 AM, Fredrik Hubinette <[email protected]>
> wrote:
> > First thing might be to compare these values to a program that's been
> > running for an hour or so. You seem to have a very large number of
> objects,
> > is that normal?
>
> Yes, it's a pretty big program. Here's memory_usage immediately after
> startup (and settling down):
>
> ([ /* 50 elements */
>   "array_bytes": 665536,
>   "ba_mixed_frame_bytes": 216,
>   "call_out_bytes": 1184,
>   "callable_bytes": 27896,
>   "callback_bytes": 4328,
>   "catch_context_bytes": 7488,
>   "compat_cb_box_bytes": 216,
>   "destroy_called_mark_bytes": 216,
>   "free_block_bytes": 560688,
>   "gc_rec_frame_bytes": 216,
>   "malloc_block_bytes": 14823424,
>   "malloc_bytes": 11669968,
>   "mapping_bytes": 802327,
>   "marker_bytes": 216,
>   "mc_marker_bytes": 216,
>   "multiset_bytes": 40672,
>   "node_s_bytes": 216,
>   "num_arrays": 4944,
>   "num_ba_mixed_frames": 0,
>   "num_call_outs": 2,
>   "num_callables": 256,
>   "num_callbacks": 5,
>   "num_catch_contexts": 2,
>   "num_compat_cb_boxs": 0,
>   "num_destroy_called_marks": 0,
>   "num_free_blocks": 1,
>   "num_gc_rec_frames": 0,
>   "num_malloc": 26,
>   "num_malloc_blocks": 5,
>   "num_malloced_strings": 8165,
>   "num_mappings": 908,
>   "num_markers": 0,
>   "num_mc_markers": 0,
>   "num_multisets": 90,
>   "num_node_ss": 0,
>   "num_objects": 1439,
>   "num_pike_frames": 272,
>   "num_pike_list_nodes": 0,
>   "num_pike_types": 9640,
>   "num_programs": 1041,
>   "num_short_strings": 17551,
>   "num_static_strings": 1043,
>   "num_strings": 28753,
>   "num_substrings": 1954,
>   "object_bytes": 218904,
>   "pike_frame_bytes": 32768,
>   "pike_list_node_bytes": 216,
>   "pike_type_bytes": 650568,
>   "program_bytes": 2622416,
>   "string_bytes": 4925892
> ])
>
> Two orders of magnitude less malloc_bytes/malloc_block_bytes than the
> 5GB process.
>
> > Since I don't remember: Is malloc_bytes a subset of malloc_block_bytes,
> or
> > not?
>
> No idea, but it looks like it.
>
> > Second thing would be to run everything through valgrind and see if it
> says
> > anything interesting.
>
> Will have to try that some time... I hope valgrind doesn't slow the
> program down much, as it's one I use extensively through the day.
>
> > You can also check the memory map with pmap(1) to see if maybe there are
> > some large mmaps or something using up memory.
>
> The only one of any appreciable size is this:
>
> rosuav@sikorsky:~$ pmap -x 2279
> 2279:   pike gypsum
> Address           Kbytes     RSS   Dirty Mode  Mapping
> ...
> 000000000277c000 5204304 5192596 5192596 rw---   [ anon ]
> 000000000277c000       0       0       0 rw---   [ anon ]
> ...
> ---------------- ------- ------- -------
> total kB         5704440 5287820 5271004
>
> Looks like that's actual RAM allocation as done by the language
> interpreter, I think.
>
> Thanks for the tips.
>
> ChrisA
>
>
>