[rvm-research] binding all memory to a single socket

Hi All,

I am working on NUMA-awareness in the Jikes RVM. As a start, I am trying to isolate all the memory the Java program and the Jikes RVM creates on a single socket of a multi-socket machine. My approach is to call mbind() after each malloc and mmap call in Jikes. Then, I run a simple test and use the pcm-memory.x performance counter utility from Intel to observe the traffic on the memory channels in the two sockets that my test machine has.

Mmap and malloc is called in JikesRVM from sys.C, RunBootImage.C, libvm.c and in classpath from the native jni code.

So far, I have added mbind to the appropriate locations in Jikes. JNI locations are trickier to instrument since not all allocations are at a page boundary.

I wrote a simple test to stress the heap allocation. Elements are continuously added to a vector and removed, and elements are written once. 

When I run this test program, I observe that, traffic on the socket where I bind the memory allocation is high, say around 1 GB/s for my test program. But I also observe up to 50 MB/s on the other socket. My question is what is happening on the other socket? For this simple test, could it be the allocations in the classpath? Are there other allocations in Jikes RVM that I am missing? Have I forgotten to instrument allocations in the boot loader code? In libvm.c, there is mapImgeFile that I instrumented. In RunBootImage, there is a call to malloc() for storing args, that is unlikely to make any difference.

Thanks in advance. A wild guess is as welcome as an informed opinion :)

Regards,
Shoaib
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot