Re: Tracking down memory usage (leak?) in Pike program

On Wed, Jul 20, 2016 at 4:28 AM, Fredrik Hubinette <[email protected]> wrote:
> First thing might be to compare these values to a program that's been
> running for an hour or so. You seem to have a very large number of objects,
> is that normal?

Yes, it's a pretty big program. Here's memory_usage immediately after
startup (and settling down):

([ /* 50 elements */
  "array_bytes": 665536,
  "ba_mixed_frame_bytes": 216,
  "call_out_bytes": 1184,
  "callable_bytes": 27896,
  "callback_bytes": 4328,
  "catch_context_bytes": 7488,
  "compat_cb_box_bytes": 216,
  "destroy_called_mark_bytes": 216,
  "free_block_bytes": 560688,
  "gc_rec_frame_bytes": 216,
  "malloc_block_bytes": 14823424,
  "malloc_bytes": 11669968,
  "mapping_bytes": 802327,
  "marker_bytes": 216,
  "mc_marker_bytes": 216,
  "multiset_bytes": 40672,
  "node_s_bytes": 216,
  "num_arrays": 4944,
  "num_ba_mixed_frames": 0,
  "num_call_outs": 2,
  "num_callables": 256,
  "num_callbacks": 5,
  "num_catch_contexts": 2,
  "num_compat_cb_boxs": 0,
  "num_destroy_called_marks": 0,
  "num_free_blocks": 1,
  "num_gc_rec_frames": 0,
  "num_malloc": 26,
  "num_malloc_blocks": 5,
  "num_malloced_strings": 8165,
  "num_mappings": 908,
  "num_markers": 0,
  "num_mc_markers": 0,
  "num_multisets": 90,
  "num_node_ss": 0,
  "num_objects": 1439,
  "num_pike_frames": 272,
  "num_pike_list_nodes": 0,
  "num_pike_types": 9640,
  "num_programs": 1041,
  "num_short_strings": 17551,
  "num_static_strings": 1043,
  "num_strings": 28753,
  "num_substrings": 1954,
  "object_bytes": 218904,
  "pike_frame_bytes": 32768,
  "pike_list_node_bytes": 216,
  "pike_type_bytes": 650568,
  "program_bytes": 2622416,
  "string_bytes": 4925892
])

Two orders of magnitude less malloc_bytes/malloc_block_bytes than the
5GB process.

> Since I don't remember: Is malloc_bytes a subset of malloc_block_bytes, or
> not?

No idea, but it looks like it.

> Second thing would be to run everything through valgrind and see if it says
> anything interesting.

Will have to try that some time... I hope valgrind doesn't slow the
program down much, as it's one I use extensively through the day.

> You can also check the memory map with pmap(1) to see if maybe there are
> some large mmaps or something using up memory.

The only one of any appreciable size is this:

rosuav@sikorsky:~$ pmap -x 2279
2279:   pike gypsum
Address           Kbytes     RSS   Dirty Mode  Mapping
...
000000000277c000 5204304 5192596 5192596 rw---   [ anon ]
000000000277c000       0       0       0 rw---   [ anon ]
...
---------------- ------- ------- -------
total kB         5704440 5287820 5271004

Looks like that's actual RAM allocation as done by the language
interpreter, I think.

Thanks for the tips.

ChrisA