[rvm-research] Bugs with nursery creation

UGAWA Tomoharu <[email protected]>
Newsgroups gmane.comp.java.jikes.rvm.devel
Message-ID <[email protected]> 
Hi

I found that nursery space is reserved even for non-generational collectors
in my environment.  Also, I am afraid that there is a possibility that
normal spaces (e.g., small object space, LOS, and so on) may be expanded such
that they overlap with nursery regardless of whether the collector is
generational or not.

My environment is:
> Linux enju 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Javac is:
> javac 1.6.0_31
Jikes RVM version is:
> changeset:   10617:65aaf607fd8d
> tag:         tip

(1) Nursery is reserved for non-generational collectors

This is caused by a (possibly redundant) class dependency.

org.mmtk.utility.options.PretenureThresholdFraction depends on
org.mmtk.plan.generational.Gen as it refers to DEFAULT_PRETENURE_THRESHOLD_FRACTION
defined in Gen for the default value of this command line option.
Thus, javac recognises this dependency, and Gen is put in the primordial
list (Primordial.txt).

The class PretenureThresholdFraction is instantiated in the constructor of
org.mmtk.plan.Plan.  Thus this class is instantiated regardless of whether
the collector is generational or not.

I checked this with the configuration FastAdaptiveMarkSweep.
Executing vm with -X:gc:verbose=3 option, I observed there is nursery.

For the time being, I removed the reference to the constant defined in Gen from
the PretenureThresholdFraction class.  (I attached the patch).

(2) Overlapping normal space with nursery

This is caused by the execution order of two initialisers:
- the constructor of org.mmtk.plan.Plan
- the class initialiser of org.mmtk.plan.generational.Gen

In the constructor of Plan, the free list of chunks (Map.regionMap) is created
by calling the Map.finalizeStaticSpaceMap() method, and the address range available
for normal space is fixed.  If the nursery space is created beforehand, the
address ranged for the nursery is excluded from the available address range.
However, if the class initialiser of Gen is executed after the constructor of
Plan is executed, the address range for the nursery is not taken into account when
computing the available address range because the nursery is created in the class
initialiser of Gen.

For generational collectors, this seems to rarely happen because of the class
hierarchy.  Loading Plan and Gen is usually triggered by the compilation of expression
> VM.activePlan.global()
in the boot image compilation.
This expression triggers class loading of the active plan, org.mmtk.plan.GenImmix
for example.  GenImmix extends Gen, which is a subclass of Plan.
Thus, when GenImmix is loaded, Plan and Gen are also loaded, and class initialiser of
Gen is executed prior to the constructor of Plan.
However, if compiler happens to compile an expression that triggers class loading
of Plan before Gen is loaded, the nursery is not taken into account when computing
the available address range for normal spaces.

For a non-generational collector, say MS for mark sweep, this is more likely to occur
unless the problem (1) is fixed.  MS is not a subclass of Gen.  Thus, loading MS class
does not trigger loading Gen.

The following program fills up the memory and is expected to die with an out of memory
exception.  However, when I executed with FastAdaptiveMarkSweep with more than 850 MB
of heap (-Xmx850m), it crashed outputting a log of a VM internal bug saying:
> Conflicting virtual address request for space "ms" at 0xa4c00000
I quote the complete crash log following to the program.

public class Test {
    static class List {
	List next;
    }
    public static void main(String[] args) {
	List head = null;
	while (true) {
	    List p = new List();
	    p.next = head;
	    head = p;
	}
    }
}

---- crash log ----
Conflicting virtual address request for space "ms" at 0xa4c00000
Key: (I)mmortal (N)onmoving (D)iscontiguous (E)xtent (F)raction
     HEAP_START 0x60000000
AVAILABLE_START 0x65800000
       boot IN  0x60000000->0x6fffffff E 0x10000000
   immortal IND []
       meta  ND []
        los  ND [0x70c00000->0x70ffffff]
     sanity  ND []
 non-moving  ND [0x70000000->0x703fffff]
    sm-code  ND [0x70800000->0x70bfffff]
    lg-code  ND []
         ms  ND [0xa4800000->0xa4bfffff, 0xa4400000->0xa47fffff, 0xa4000000->0xa43fffff, 0xa3c00000->0xa3ffffff, 0xa3800000->0xa3bfffff, 0xa3400000->0xa37fffff, 0xa3000000->0xa33fffff, 0xa2c00000->0xa2ffffff, 0xa2800000->0xa2bfffff, 0xa2400000->0xa27fffff, 0xa2000000->0xa23fffff, 0xa1c00000->0xa1ffffff, 0xa1800000->0xa1bfffff, 0xa1400000->0xa17fffff, 0xa1000000->0xa13fffff, 0xa0c00000->0xa0ffffff, 0xa0800000->0xa0bfffff, 0xa0400000->0xa07fffff, 0xa0000000->0xa03fffff, 0x9fc00000->0x9fffffff, 0x9f800000->0x9fbfffff, 0x9f400000->0x9f7fffff, 0x9f000000->0x9f3fffff, 0x9ec00000->0x9effffff, 0x9e800000->0x9ebfffff, 0x9e400000->0x9e7fffff, 0x9e000000->0x9e3fffff, 0x9dc00000->0x9dffffff, 0x9d800000->0x9dbfffff, 0x9d400000->0x9d7fffff, 0x9d000000->0x9d3fffff, 0x9cc00000->0x9cffffff, 0x9c800000->0x9cbfffff, 0x9c400000->0x9c7fffff, 0x9c000000->0x9c3fffff, 0x9bc00000->0x9bffffff, 0x9b800000->0x9bbfffff, 0x9b400000->0x9b7fffff, 0x9b000000->0x9b3fffff, 0x9ac00000->0x9affffff, 0x9a800000->0x9abfffff, 0x9a400000->0x9a7fffff, 0x9a000000->0x9a3fffff, 0x99c00000->0x99ffffff, 0x99800000->0x99bfffff, 0x99400000->0x997fffff, 0x99000000->0x993fffff, 0x98c00000->0x98ffffff, 0x98800000->0x98bfffff, 0x98400000->0x987fffff, 0x98000000->0x983fffff, 0x97c00000->0x97ffffff, 0x97800000->0x97bfffff, 0x97400000->0x977fffff, 0x97000000->0x973fffff, 0x96c00000->0x96ffffff, 0x96800000->0x96bfffff, 0x96400000->0x967fffff, 0x96000000->0x963fffff, 0x95c00000->0x95ffffff, 0x95800000->0x95bfffff, 0x95400000->0x957fffff, 0x95000000->0x953fffff, 0x94c00000->0x94ffffff, 0x94800000->0x94bfffff, 0x94400000->0x947fffff, 0x94000000->0x943fffff, 0x93c00000->0x93ffffff, 0x93800000->0x93bfffff, 0x93400000->0x937fffff, 0x93000000->0x933fffff, 0x92c00000->0x92ffffff, 0x92800000->0x92bfffff, 0x92400000->0x927fffff, 0x92000000->0x923fffff, 0x91c00000->0x91ffffff, 0x91800000->0x91bfffff, 0x91400000->0x917fffff, 0x91000000->0x913fffff, 0x90c00000->0x90ffffff, 0x90800000->0x90bfffff, 0x90400000->0x907fffff, 0x90000000->0x903fffff, 0x8fc00000->0x8fffffff, 0x8f800000->0x8fbfffff, 0x8f400000->0x8f7fffff, 0x8f000000->0x8f3fffff, 0x8ec00000->0x8effffff, 0x8e800000->0x8ebfffff, 0x8e400000->0x8e7fffff, 0x8e000000->0x8e3fffff, 0x8dc00000->0x8dffffff, 0x8d800000->0x8dbfffff, 0x8d400000->0x8d7fffff, 0x8d000000->0x8d3fffff, 0x8cc00000->0x8cffffff, 0x8c800000->0x8cbfffff, 0x8c400000->0x8c7fffff, 0x8c000000->0x8c3fffff, 0x8bc00000->0x8bffffff, 0x8b800000->0x8bbfffff, 0x8b400000->0x8b7fffff, 0x8b000000->0x8b3fffff, 0x8ac00000->0x8affffff, 0x8a800000->0x8abfffff, 0x8a400000->0x8a7fffff, 0x8a000000->0x8a3fffff, 0x89c00000->0x89ffffff, 0x89800000->0x89bfffff, 0x89400000->0x897fffff, 0x89000000->0x893fffff, 0x88c00000->0x88ffffff, 0x88800000->0x88bfffff, 0x88400000->0x887fffff, 0x88000000->0x883fffff, 0x87c00000->0x87ffffff, 0x87800000->0x87bfffff, 0x87400000->0x877fffff, 0x87000000->0x873fffff, 0x86c00000->0x86ffffff, 0x86800000->0x86bfffff, 0x86400000->0x867fffff, 0x86000000->0x863fffff, 0x85c00000->0x85ffffff, 0x85800... WARNING: Text truncated.
    nursery     0xa4c00000->0xafffffff F 0.15
  AVAILABLE_END 0xb0000000
       HEAP_END 0xb0000000
used = 214097 pgs = boot 0 pgs + immortal 0 pgs + meta 0 pgs + los 937 pgs + sanity 0 pgs + non-moving 113 pgs + sm-code 42 pgs + lg-code 0 pgs + ms 213005 pgs + nursery 0 pgs
used = 836.31 Mb = boot 0.00 Mb + immortal 0.00 Mb + meta 0.00 Mb + los 3.66 Mb + sanity 0.00 Mb + non-moving 0.44 Mb + sm-code 0.16 Mb + lg-code 0.00 Mb + ms 832.05 Mb + nursery 0.00 Mb
exiting
Thread #19
-- Stack --
(0x70fb4b94 7556)   at [0x70fb4b94, 0x647da6e7] Lorg/jikesrvm/VM; sysFail(Ljava/lang/String;)V at line 2324
(0x70fb4be4 19234)   at [0x70fb4be4, 0x647da202] Lorg/jikesrvm/mm/mmtk/Assert; fail(Ljava/lang/String;)V at line 41
   at [0x70fb4be4, 0x647da202] Lorg/mmtk/utility/heap/Map; insert(Lorg/vmmagic/unboxed/Address;Lorg/vmmagic/unboxed/Extent;ILorg/mmtk/policy/Space;)V at line 104
(0x70fb4c38 19235)   at [0x70fb4c38, 0x64602c5a] Lorg/mmtk/utility/heap/Map; allocateContiguousChunks(ILorg/mmtk/policy/Space;ILorg/vmmagic/unboxed/Address;)Lorg/vmmagic/unboxed/Address; at line 140
(0x70fb4ca0 13365)   at [0x70fb4ca0, 0x64603854] Lorg/mmtk/policy/Space; growDiscontiguousSpace(I)Lorg/vmmagic/unboxed/Address; at line 453
   at [0x70fb4ca0, 0x64603854] Lorg/mmtk/utility/heap/FreeListPageResource; allocateContiguousChunks(I)I at line 266
(0x70fb4d10 13361)   at [0x70fb4d10, 0x64605b4b] Lorg/mmtk/utility/heap/FreeListPageResource; allocPages(IIZ)Lorg/vmmagic/unboxed/Address; at line 149
(0x70fb4d58 5439)   at [0x70fb4d58, 0x64937ed4] Lorg/mmtk/utility/heap/PageResource; getNewPages(IIZ)Lorg/vmmagic/unboxed/Address; at line 241
   at [0x70fb4d58, 0x64937ed4] Lorg/mmtk/policy/Space; acquire(I)Lorg/vmmagic/unboxed/Address; at line 430
(0x70fb4d8c 11195)   at [0x70fb4d8c, 0x645a9a1f] Lorg/mmtk/utility/alloc/BlockAllocator; alloc(Lorg/mmtk/policy/Space;I)Lorg/vmmagic/unboxed/Address; at line 80
(0x70fb4dec 10955)   at [0x70fb4dec, 0x645aaef9] Lorg/mmtk/policy/SegregatedFreeListSpace; expandSizeClass(ILorg/vmmagic/unboxed/AddressArray;)Lorg/vmmagic/unboxed/Address; at line 216
   at [0x70fb4dec, 0x645aaef9] Lorg/mmtk/policy/SegregatedFreeListSpace; getAllocationBlock(ILorg/vmmagic/unboxed/AddressArray;)Lorg/vmmagic/unboxed/Address; at line 199
(0x70fb4e2c 854)   at [0x70fb4e2c, 0x645aa916] Lorg/mmtk/utility/alloc/SegregatedFreeListLocal; allocSlowOnce(III)Lorg/vmmagic/unboxed/Address; at line 123
(0x70fb4e74 553)   at [0x70fb4e74, 0x7083047b] Lorg/mmtk/utility/alloc/Allocator; allocSlowInline(III)Lorg/vmmagic/unboxed/Address; at line 258
   at [0x70fb4e74, 0x7083047b] Lorg/mmtk/utility/alloc/Allocator; allocSlow(III)Lorg/vmmagic/unboxed/Address; at line 237
(0x70fb4ea0 21707)   at [0x70fb4ea0, 0x64a3baee] Lorg/mmtk/utility/alloc/SegregatedFreeList; alloc(III)Lorg/vmmagic/unboxed/Address; at line 88
   at [0x70fb4ea0, 0x64a3baee] Lorg/mmtk/plan/marksweep/MSMutator; alloc(IIIII)Lorg/vmmagic/unboxed/Address; at line 64
   at [0x70fb4ea0, 0x64a3baee] Lorg/jikesrvm/mm/mminterface/MemoryManager; allocateSpace(Lorg/jikesrvm/mm/mminterface/Selected$Mutator;IIIII)Lorg/vmmagic/unboxed/Address; at line 578
   at [0x70fb4ea0, 0x64a3baee] Lorg/jikesrvm/mm/mminterface/MemoryManager; allocateScalar(ILorg/jikesrvm/objectmodel/TIB;IIII)Ljava/lang/Object; at line 487
   at [0x70fb4ea0, 0x64a3baee] Lorg/jikesrvm/runtime/RuntimeEntrypoints; resolvedNewScalar(ILorg/jikesrvm/objectmodel/TIB;ZIIII)Ljava/lang/Object; at line 326
   at [0x70fb4ea0, 0x64a3baee] LTest; main([Ljava/lang/String;)V at line 9
(0x70fb4eb8 -1)   at [0x70fb4eb8, 0x6484895e] <invisible method>
(0x70fb4f3c 17432)   at [0x70fb4f3c, 0x6470d15a] Lorg/jikesrvm/runtime/Reflection; outOfLineInvoke(Lorg/jikesrvm/classloader/RVMMethod;Ljava/lang/Object;[Ljava/lang/Object;Z)Ljava/lang/Object; at line 194
(0x70fb4f90 18558)   at [0x70fb4f90, 0x648d7680] Lorg/jikesrvm/runtime/Reflection; invoke(Lorg/jikesrvm/classloader/RVMMethod;Lorg/jikesrvm/runtime/ReflectionBase;Ljava/lang/Object;[Ljava/lang/Object;Z)Ljava/lang/Object; at line 76
   at [0x70fb4f90, 0x648d7680] Lorg/jikesrvm/scheduler/MainThread; run()V at line 201
(0x70fb4fc0 20752)   at [0x70fb4fc0, 0x64b3cd98] Lorg/jikesrvm/scheduler/RVMThread; run()V at line 2561
(0x70fb4ff8 20839)   at [0x70fb4ff8, 0x0804fe56] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 2608
********************************************************************************
*                      Abnormal termination of Jikes RVM                       *
* Jikes RVM terminated abnormally indicating a problem in the virtual machine. *
* Jikes RVM relies on community support to get debug information. Help improve *
* Jikes RVM for everybody by reporting this error. Please see:                 *
*                      http://jikesrvm.org/Reporting+Bugs                      *
********************************************************************************

Kind regards
Tomoharu

------------------------------------------------------------------------------
Own the Future-Intel&reg; Level Up Game Demo Contest 2013
Rise to greatness in Intel's independent game demo contest.
Compete for recognition, cash, and the chance to get your game 
on Steam. $5K grand prize plus 10 genre and skill prizes. 
Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d

_______________________________________________
Jikesrvm-researchers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers
nursery.patch (application/octet-stream, 4.8 KB) 
diff --git a/MMTk/src/org/mmtk/plan/copyms/CopyMS.java b/MMTk/src/org/mmtk/plan/copyms/CopyMS.java
index 0e70092..6a88ed5 100644
--- a/MMTk/src/org/mmtk/plan/copyms/CopyMS.java
+++ b/MMTk/src/org/mmtk/plan/copyms/CopyMS.java
@@ -50,6 +50,7 @@ public class CopyMS extends StopTheWorld {
   /****************************************************************************
    * Constants
    */
+  static final float DEFAULT_PRETENURE_THRESHOLD_FRACTION = 1.5f;
 
   /****************************************************************************
    * Class variables
@@ -83,6 +84,7 @@ public class CopyMS extends StopTheWorld {
  */
   public CopyMS() {
     trace = new Trace(metaDataSpace);
+    Options.pretenureThresholdFraction.setDefaultValue(DEFAULT_PRETENURE_THRESHOLD_FRACTION);
   }
 
 
diff --git a/MMTk/src/org/mmtk/plan/generational/Gen.java b/MMTk/src/org/mmtk/plan/generational/Gen.java
index be4c9ce..2541bf1 100644
--- a/MMTk/src/org/mmtk/plan/generational/Gen.java
+++ b/MMTk/src/org/mmtk/plan/generational/Gen.java
@@ -150,6 +150,10 @@ public abstract class Gen extends StopTheWorld {
       nurseryCons = null;
     }
   }
+  
+  public Gen() {
+    Options.pretenureThresholdFraction.setDefaultValue(DEFAULT_PRETENURE_THRESHOLD_FRACTION);
+  }
 
   /*****************************************************************************
    *
diff --git a/MMTk/src/org/mmtk/plan/refcount/generational/GenRC.java b/MMTk/src/org/mmtk/plan/refcount/generational/GenRC.java
index 8186eb0..65a9844 100644
--- a/MMTk/src/org/mmtk/plan/refcount/generational/GenRC.java
+++ b/MMTk/src/org/mmtk/plan/refcount/generational/GenRC.java
@@ -26,6 +26,8 @@ import org.vmmagic.unboxed.ObjectReference;
 @Uninterruptible
 public class GenRC extends RCBase {
 
+  static final float DEFAULT_PRETENURE_THRESHOLD_FRACTION = 1.5f;
+
   public static final int ALLOC_NURSERY = ALLOC_DEFAULT;
   public static final int ALLOC_RC      = RCBase.ALLOCATORS + 1;
 
@@ -34,6 +36,10 @@ public class GenRC extends RCBase {
 
   public static final int NURSERY = nurserySpace.getDescriptor();
 
+  public GenRC() {
+    Options.pretenureThresholdFraction.setDefaultValue(DEFAULT_PRETENURE_THRESHOLD_FRACTION);
+  }
+  
   /*****************************************************************************
    *
    * Collection
diff --git a/MMTk/src/org/mmtk/policy/Space.java b/MMTk/src/org/mmtk/policy/Space.java
index 7b03902..89237e8 100644
--- a/MMTk/src/org/mmtk/policy/Space.java
+++ b/MMTk/src/org/mmtk/policy/Space.java
@@ -165,6 +165,7 @@ public abstract class Space implements Constants {
         VM.assertions.fail(name + " starting on non-aligned boundary: " + start.toLong() + " bytes");
       }
     } else if (vmRequest.top) {
+      if (Map.isFinalized()) VM.assertions.fail("heap is narrowed after regionMap is finalized: "+ name);
       heapLimit = heapLimit.minus(extent);
       start = heapLimit;
     } else {
diff --git a/MMTk/src/org/mmtk/utility/heap/Map.java b/MMTk/src/org/mmtk/utility/heap/Map.java
index 6f86580..0a0586f 100644
--- a/MMTk/src/org/mmtk/utility/heap/Map.java
+++ b/MMTk/src/org/mmtk/utility/heap/Map.java
@@ -53,6 +53,8 @@ public class Map {
   private static final FreeListPageResource[] sharedFLMap;
   private static int totalAvailableDiscontiguousChunks = 0;
 
+  private static boolean finalized = false;
+
   private static final Lock lock = VM.newLock("Map lock");
 
   /****************************************************************************
@@ -277,8 +279,14 @@ public class Map {
       if (VM.VERIFY_ASSERTIONS) VM.assertions._assert(allocedPages == firstPage);
       firstPage += Space.PAGES_IN_CHUNK;
     }
+    
+    finalized  = true;
   }
 
+  public static boolean isFinalized() {
+    return finalized;
+  }
+  
   /**
    * Return the ordinal number for some free list space wishing to share a discontiguous region.
    * @return The ordinal number for a free list space wishing to share a discontiguous region
diff --git a/MMTk/src/org/mmtk/utility/options/PretenureThresholdFraction.java b/MMTk/src/org/mmtk/utility/options/PretenureThresholdFraction.java
index c2ae9e8..7b274e5 100644
--- a/MMTk/src/org/mmtk/utility/options/PretenureThresholdFraction.java
+++ b/MMTk/src/org/mmtk/utility/options/PretenureThresholdFraction.java
@@ -12,8 +12,6 @@
  */
 package org.mmtk.utility.options;
 
-import static org.mmtk.plan.generational.Gen.DEFAULT_PRETENURE_THRESHOLD_FRACTION;
-
 public class PretenureThresholdFraction extends org.vmutil.options.FloatOption {
   /**
    * Create the option.
@@ -21,7 +19,7 @@ public class PretenureThresholdFraction extends org.vmutil.options.FloatOption {
   public PretenureThresholdFraction() {
     super(Options.set, "Pretenure Threshold Fraction",
           "Objects larger than this fraction of the remaining nursery will be allocated directly into the LOS.",
-          DEFAULT_PRETENURE_THRESHOLD_FRACTION);
+          1.0f);
   }
 
   /**
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.