Re: MLton and Valgrind

On Thu, Mar 1, 2012 at 4:24 PM, David Hansel
<[email protected]> wrote:
> I've been trying to use Valgrind (valgrind.org) to try to find out if
> there are any memory errors in FFI code called by our MLton program
> (this is on Ubuntu 10.04).

I've not tried Valgrind with MLton; and while I've heard the
1-sentence summary of Valgrind, I've never really worked with the
tools.

> Running any MLton compiled program under Valgrind at first gave me a
> large number of errors, until I tried compiling with the C codegen
> (instead of the native).  After changing that the errors seem to be
> gone.

That makes some sense, as the native codegen doesn't follow C
conventions, either statically or dynamically.  Statically, that means
that dumping the referenced labels in native codegen object file may
not be interpretable by Valgrind as functions with well-defined
boundaries.  Dynamically, that means that if Valgrind is inspecting a
running process, then $esp isn't pointing to the top of the C stack
when executing within a native codegen object file.  On the other
hand, if Valgrind only inspects the program state at calls to C
functions (e.g., malloc/free/...), then the dynamic program state
(e.g., $esp) should look reasonably like a C program at those points.

> However, after adding FFI code to the MLton program I do occasionally
> get memory errors reported by Valgrind, presumably during garbage
> collection.  Note that the program runs fine and does not crash (with or
> without Valgrind). The errors reported look like this:
>
> ==19049== Invalid write of size 1
> ==19049==    at 0x4027448: memset (mc_replace_strmem.c:602)
> ==19049==    by 0x805157C: setCardMapAndCrossMap (in /home/hansel/test1)
> ==19049==  Address 0x467b000 is not stack'd, malloc'd or (recently) free'd
>
> ==19049== Invalid read of size 1
> ==19049==    at 0x4026970: memcpy (mc_replace_strmem.c:497)
> ==19049==    by 0x805183D: forwardObjptr (in /home/hansel/test1)
> ==19049==  Address 0x470a5ce is not stack'd, malloc'd or (recently) free'd

I agree with Wesley that these are true statements.  The memory
obtained for the ML heap (which includes the card map and cross map)
is obtained my mmap (and resized by mremap and released by unmap).
So, it is true that any manipulation of addresses within the ML heap
is "no stack'd, malloc'd or (recently) free'd".  On the other hand, I
would imagine that mmap is a sufficiently widespread idiom for
obtaining large blocks of memory that Valgrind would recognize it; but
I may be wrong.  Of course, even if Valgrind recognized mmap obtained
memory, it is quite true that the GC does mem* operations within the
heap long after it was initially obtained.

> I'm not an expert with Valgrind and I guess it is possible that these
> are somehow wrongfully detected and there is no problem with MLton
> itself.  Has anybody here tried using Valgrind with MLton?  Is there
> something that I should be aware of?  Should I just ignore these errors
> or are they maybe an indication of something going wrong in MLton?
>
> I can provide some example code which produces the errors mentioned
> when running under Valgrind.  For the same code the errors go away if
> I remove the FFI functions from the .mlb file (the FFI functions are
> not actually called by the code,  it seems that just loading the .so
> file creates the problem).

That is a little bit strange.  I suppose the shared library could have
some initialization code that triggers something in Valgrind, but it
is curious that it manifests itself with warnings about the ML heap.
The shared library ought not to have any reference to the ML heap.

> If you are interested, please let me know where I can upload
> the (2KB) .zip file.

You can upload files too large for the mailing list to:
  http://mlton.org/TemporaryUpload

------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing 
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/