Re: OCaml Memory Management Best Practices

"Gerd Stolpmann [email protected] [ocaml_beginners]" <[email protected]>
Newsgroups gmane.comp.lang.ocaml.beginners
Message-ID <[email protected]>
Am Sonntag, den 28.06.2015, 17:56 +0000 schrieb Kenneth Miller
[email protected] [ocaml_beginners]:
>   
> Everything that you've said sounds very counter-intuitive to what I've
> read and learned so far. If there's nothing I can do about my ocaml
> program running until there is no more memory or swap before dying
> that would be abysmally sad.

I don't know from where you have your knowledge, but it is definitely
wrong. Java has always had a fully automatic garbage collector: An
object is collected when it cannot be reached by any pointer. The same
holds for OCaml, only substitute the word "value" for "object".

And this is not "sad". You have the guarantee that all unused memory is
really reclaimed. Why do you want to improve something that is already
perfect?

>  From what I understand about garbage collection in the general
> problem, it's undecidable; 

I guess you also don't know what the word "undecidable" means. (A
problem is called undecidable when there is no algorithm that can answer
the problem positively or negatively in a finite number of steps - note
that there is also a strict formal definition).

> garbage collectors aim just for optimal performance and good behavior
> in most cases. 

No. There is a class known as precise collectors, and these find all
unused memory and reclaim it. It is true that there also imprecise
collectors (e.g. using reference counting or conservative collection),
but these are not state of the art anymore.

> So then if there are even memory profilers for ocaml there must be
> some corrective action that can be taken, that's what I need to know.

The memory profilers do not exist for finding leaks. There are no leaks
in the usual sense that memory is completely forgotten about.
Nevertheless, you sometimes want to know how much bytes you need for
your data structures, and if you want to measure this after running a
complex program for a while a memory profiler can be handy.

>  The pervasives library just logged a leak-they will do something
> about that. There has to be some kind of general practice that will
> allow me to keep moving forward.

I don't know to which bug you refer. The pervasives library has also a
part implemented in C.

> Some of the other stuff about the JVM's GC you say is not what I've
> heard so far (JVM GC use mark and sweep?), so wouldn't dispute what
> algorithm it uses until I've read about it in depth. Java not using
> reference counting is precisely the opposite of what I've learned.

Just google for it. E.g. I found
http://www.cubrid.org/blog/dev-platform/understanding-java-garbage-collection/

>  Suppose I have an array which stores null or a reference to an
> object. If I finish working with an object, do I not mark any
> reference null in order to ensure that it gets garbage collected?

Well, the reason for this is not to ensure _that_ it is garbage
collected, but to get more control _when_ it is collected. If you leave
the unused objects in the array, they will first get unreachable when
the pointers are overwritten in the array or when the array is
completely dropped. I'd bother with this only in special situations, in
particular when the array is long-living and you'd spend a lot of memory
for too long.

As your initial question was about OCaml lists, and List.map. As this
function always creates a new list, the old list gets out of use
immediately, and will be collected soon. This is very different from a
long-living array.

Gerd

>  I had an assignment once where we used a tree-set data structure to
> manage indexes to another reference array for fast lookup. We had to
> do exactly this.
> 
> 
> 
> 
> On Sunday, June 28, 2015 9:17 AM, "Gerd Stolpmann
> [email protected] [ocaml_beginners]"
> <[email protected]> wrote:
> 
> 
> 
>   
> 
> Am Samstag, den 27.06.2015, 17:06 +0000 schrieb Kenneth Miller
> [email protected] [ocaml_beginners]:
> > 
> > For operations like List.map, I would imagine that the what is
> > actually occurring, is a new list of equal size is being constructed
> > where an operation on the old map is happening for every element. I
> > would like to know what the real implications are for the garbage
> > collector, and how I can minimize the performance and memory impact.
> 
> You cannot do anything, and you need not.
> 
> > Things like List.append calculate a new list by constructing a copy.
> > That's incredibly inefficient to me, both in the sense that each
> > element afterward (time) needs to be copied, and in that more memory
> > (space) is needed.
> > 
> > 
> > In addition, is there a set of advice points that can be given to
> make
> > sure that the garbage collector will appropriately see what it needs
> > to in order to free items? In java, everything is reference counted,
> 
> No, this is not true. Java, like OCaml, uses at least a mark-and-sweep
> collector (of course, this depends on the JVM used, and there is
> normally even a generational collector). Reference-counting is a bad
> choice for any higher-level language because it cannot detect cyclic
> pointer chains, and these remain uncollected.
> 
> > so if you want an object to be freed and you're maintaining it in a
> > tree the lack of use of that object is better treated by going to
> the
> > tree and updating the node that references it to mark it null. In
> this
> > way, the garbage collector can collect the item that you were
> pointing
> > to.
> 
> There is no need to do this (breaking cycles) in Java, and no need to
> do
> this in OCaml.
> 
> > OCaml is a different animal though, and it doesn't use reference
> > counting. There doesn't seem to me to be any simple go-to practice
> in
> > order to ensure that the GC will in fact collect an item that I know
> > at the moment because it's operation is so different.
> 
> The OCaml collector is fully automatic like the one in Java. Cycles
> are
> collected when the whole cycle becomes unreachable.
> 
> Gerd
> 
> [Non-text portions of this message have been removed]
> 
> 
> 
> 
> 
> 
>
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.