Re: [rvm-research] Failure implementing a Mark-Sweep plus Semispace composite
Shoaib Akram <[email protected]> Fri, 25 Nov 2016 22:40:37 +0100
| Newsgroups | gmane.comp.java.jikes.rvm.devel |
|---|---|
| Message-ID | <[email protected]> |
Thanks Robin and Mike for the ideas. To narrow down the problem, I created the collector Mike described: a non-generational collector with a nursery consisting of two semispaces (A and B) and a mark-sweep mature space (MS). Every collection is full-heap so no need to maintain remsets. Mutator first allocates into the nursery space A. After the first collection, live objects are moved to MS and the mutator’s allocator is rebound to B. Then the mutator allocates into B. The next collection will take live objects out of both MS and B and place them in A. The mutator then allocates into A where the collector left off. I experienced the same problems with this collector as with the generational variant. After some debugging, I narrowed the problem to a bunch of problematic objects whose moving from MS to A or B causes JikesRVM to hang. These include objects from classes loaded by the bootstrap classloader (Lgnu, Ljava, Lorg/jikesrvm, Lorg/vmmagic). Also some other classes such as Ljavax create the same problem. After pinning these objects in MS, a selection of dacapo benchmarks finish execution properly. I wonder why moving these objects out of a CopySpace is not a problem but moving them out of mark-sweep space is (?) There is a utility function in Space.java isMovable() which another part of Jikes can use to inquire whether objects in a particular space can move or not. For my implementation of mark-sweep space, I override this function to return true. I did not check my generational variant yet. But I hope if remsets were a problem, enabling the default sanity checker in MMTk would have reported errors (?) Thanks again for your thoughts. > On Nov 21, 2016, at 4:16 PM, Michael Bond <[email protected]> wrote: > > Hi Shoaib, > > What happens if you get rid of the nursery entirely, i.e., just have full-heap collector that has a MarkSweep space and (two?) SemiSpace spaces, and support copying between the two types of spaces? That might help narrow down whether this is a generational GC issue or not. > > Cheers, > Mike > > On 11/21/2016 09:25 AM, Shoaib Akram wrote: >> Thanks Robin for replying. I didn’t change the way remsets work. By default in the generational collectors in JikesRVM, the remsets are processed as part of the tracing procedure, and handled the same way as any root edge. So, making every collection din’t help. By triggering early (full-heap) collection, I get a different error (below). I presume CopyMS does not move objects from mark-sweep to nursery space. That is the path giving me problems. So basically my mark-sweep space at the end of full-heap collection has both forwarded and unforwarded (marked) objects. My suspicion is that I am moving something out of the mark-sweep space without the new reference getting updated somewhere. Although my debug logs (and results of sanity checker) show that all pointers are being forwarded properly. >> >> xception in thread "MainThread" java.lang.IllegalStateException: ZipFile has closed: /home/sakram/jikesrvm-3.1.3/dist/FastAdaptiveGenCopyBarriers_x86_64-linux/rvmrt.jar >> at java.lang.Exception.<init>(Exception.java:78) >> at java.lang.RuntimeException.<init>(RuntimeException.java:76) >> at java.lang.IllegalStateException.<init>(IllegalStateException.java:79) >> at java.util.zip.ZipFile.checkClosed(ZipFile.java:219) >> at java.util.zip.ZipFile.getEntry(ZipFile.java:377) >> at java.util.jar.JarFile.readSignatures(JarFile.java:591) >> at java.util.jar.JarFile.getEntry(JarFile.java:420) >> at gnu.java.net.protocol.jar.Connection.connect(Connection.java:147) >> at gnu.java.net.protocol.jar.Connection.getInputStream(Connection.java:159) >> at java.net.URL.openStream(URL.java:737) >> at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:733) >> at java.util.ResourceBundle.tryBundle(ResourceBundle.java:481) >> at java.util.ResourceBundle.tryBundle(ResourceBundle.java:550) >> at java.util.ResourceBundle.getBundle(ResourceBundle.java:400) >> at java.text.DecimalFormatSymbols.<init>(DecimalFormatSymbols.java:166) >> at java.text.DecimalFormat.<clinit>(DecimalFormat.java:97) >> at org.jikesrvm.classloader.RVMClass.initialize(RVMClass.java:1542) >> at org.jikesrvm.runtime.RuntimeEntrypoints.initializeClassForDynamicLink(RuntimeEntrypoints.java:610) >> at org.jikesrvm.runtime.RuntimeEntrypoints.unresolvedNewScalar(RuntimeEntrypoints.java:268) >> at org.dacapo.harness.TestHarness.twoDecimalPlaces(TestHarness.java:234) >> at org.dacapo.harness.TestHarness.<clinit>(TestHarness.java:48) >> at org.jikesrvm.classloader.RVMClass.initialize(RVMClass.java:1542) >> at org.jikesrvm.runtime.RuntimeEntrypoints.initializeClassForDynamicLink(RuntimeEntrypoints.java:610) >> at org.jikesrvm.classloader.TableBasedDynamicLinker.resolveMember(TableBasedDynamicLinker.java:76) >> at org.jikesrvm.classloader.TableBasedDynamicLinker.resolveMember(TableBasedDynamicLinker.java:65) >> at Harness.main(Harness.java:17) >> >> >>> On Nov 21, 2016, at 1:00 PM, Robin Garner <[email protected] <mailto:[email protected]>> wrote: >>> >>> How do your remsets work ? >>> >>> Can you get your collector to work if you ignore remsets and do a full-heap collection every time (a-la CopyMS ?) (CopyMS exists because it separates out the remsets from the GC mechanisms for debugging purposes). >>> >>> On Mon, Nov 21, 2016 at 10:45 AM, Shoaib Akram <[email protected] <mailto:[email protected]>> wrote: >>> Thanks Mike for replying. As I understand it, whether objects can move from a space or not is controlled by some flags. I did modify the associated variables to reflect the correct flags. Regarding trying a different space, my plan does work correctly if I use two separate mature spaces that are both copying i.e. space A consist of two semispaces A0 and A1, whereas space B consists of two semispaces B0 and B1. Objects can move from A0 to B0 or B1 and vice-versa. Trouble occurs in case one space is mark-sweep and objects move from mark-sweep to a copy space. >>> >>> > On Nov 17, 2016, at 5:03 PM, Michael Bond <[email protected] <mailto:[email protected]>> wrote: >>> > >>> > Hi Shoaib, >>> > >>> > I believe that the implementation of MarkSweep assumes that objects are >>> > non-movable (i.e., can't be moved within the space or to another space). >>> > There are some methods that the MarkSweepSpace (or maybe the collector >>> > or *Constraints classe(s)?) override/implement that specify whether the >>> > space and/or objects are movable; at a minimum, you'll need to change >>> > the implementation of these methods to specify that objects are now >>> > movable in a MarkSweepSpace. >>> > >>> > There might also be assumptions for a MarkSweepSpace that are not >>> > explicitly specified but that would effectively prohibit moving objects. >>> > But maybe not; I think it's worth a try. :) MMTk developers, what do you >>> > think? >>> > >>> > Have you considered using a different kind of mature space, such as >>> > ImmixSpace, that supports moving objects? >>> > >>> > Regarding debugging: you could turn on GC assertions in order to >>> > hopefully get an earlier / more relevant error for runs that fail. >>> > >>> > Cheers, >>> > Mike >>> > >>> > On 11/14/2016 02:28 PM, Shoaib Akram wrote: >>> >> Hi All, >>> >> >>> >> I am having problems with a new collector I am trying to build that extends the traditional generational semispace collector in JikesRVM with another mature space that is an instance of the MarkSweepSpace (MS). The plan is as follows. Nursery survivors always evacuate to the active mature semispace. Objects are copied between the two mature semispaces as usual. From time to time, a mature collection copies a selection of the live objects out of the mature semispace to MS and vice-versa. A live object in MS is guaranteed to be either marked or forwarded during a mature collection. >>> >> >>> >> Everything works fine as long as there are no copies from MS to the semispace. But if objects are copied from MS to the semispace, then after a few collections, JikesRVM crashes. It appears to me that crash always occur during a collection phase. >>> >> >>> >> I have taken care that the GC header has both forwarding bits and mark bits. The GC header is initialized properly when the object is copied to a space. I updated the masks in MarkSweepSpace appropriately. Also note that the copies between the different spaces always succeed. >>> >> >>> >> Logically, I am quite convinced that my plan should work correctly. Something is probably missing in the implementation. >>> >> >>> >> Anyone who tired similar things: is there any fundamental limitation to copying objects from a MarkSweep space to a copy space? Other things that I need to take care to make this plan work correctly? Should I provide more details? >>> >> >>> >> A stack dump from one run of a benchmark is appended. Any help or suggestion is greatly appreciated! >>> >> >>> >> Regards, >>> >> Shoaib. >>> >> >>> >> scanning 6 >>> >> scanning 2 >>> >> scanning 3 >>> >> scanning 1 >>> >> hardwareTrapHandler: thread = 0xf5fffb40 >>> >> JikesRVM: TROUBLE. Got a signal (Segmentation fault; #11) from outside the VM's address space in thread 0xf5fffb40. >>> >> JikesRVM: UNRECOVERABLE trapped signal 11 (Segmentation fault) >>> >> handler stack 5cb700 >>> >> si->si_addr 0x5cb700 >>> >> cs 0x00000023 >>> >> ds 0x0000002b >>> >> es 0x0000002b >>> >> fs 0x00000000 >>> >> gs 0x00000063 >>> >> ss 0x0000002b >>> >> edi 0x7001d020 >>> >> esi -- PR/VP 0x70008414 >>> >> ebp 0x71011b1c >>> >> esp -- SP 0x70c9dc48 >>> >> ebx 0x74b3a2f3 >>> >> edx 0x71011b1c >>> >> ecx 0x7380a050 >>> >> eax 0x71011b1c >>> >> eip 0x005cb700 >>> >> trapno 0x0000000e >>> >> err 0x00000014 >>> >> eflags 0x00010203 >>> >> fpregs f5604150 >>> >> oldmask 0x00020000 >>> >> cr2 0x005cb700 >>> >> fp0 0x00000000000000000000 >>> >> fp1 0x00000000000000000000 >>> >> fp2 0x00000000000000000000 >>> >> fp3 0x00000000000000000000 >>> >> fp4 0x00000000000000000000 >>> >> fp5 0x00000000000080004005 >>> >> fp6 0x00000000000080004019 >>> >> fp7 0x00000000e30085fc4016 >>> >> JikesRVM: internal error >>> >> -- Stack -- >>> >> (0x70c9dcb8 19460) at [0x70c9dcb8, 0x647a9bc2] Lorg/jikesrvm/mm/mmtk/ScanThread; scanFrame(I)Lorg/vmmagic/unboxed/Address; at line 367 >>> >> (0x70c9dd04 19457) at [0x70c9dd04, 0x64a207c7] Lorg/jikesrvm/mm/mmtk/ScanThread; scanThreadInternal(Lorg/vmmagic/unboxed/Address;I)V at line 279 >>> >> (0x70c9dd60 19477) at [0x70c9dd60, 0x64a209ef] Lorg/jikesrvm/mm/mmtk/ScanThread; startScan(Lorg/mmtk/plan/TraceLocal;ZLorg/jikesrvm/scheduler/RVMThread;Lorg/vmmagic/unboxed/Address;Lorg/vmmagic/unboxed/Address;Lorg/vmmagic/unboxed/Address;Lorg/vmmagic/unboxed/Address;Lorg/vmmagic/unboxed/Address;)V at line 229 >>> >> at [0x70c9dd60, 0x64a209ef] Lorg/jikesrvm/mm/mmtk/ScanThread; scanThread(Lorg/jikesrvm/scheduler/RVMThread;Lorg/mmtk/plan/TraceLocal;ZLorg/vmmagic/unboxed/Address;Lorg/vmmagic/unboxed/Address;)V at line 195 >>> >> (0x70c9ddd0 19475) at [0x70c9ddd0, 0x6497ad0a] Lorg/jikesrvm/mm/mmtk/ScanThread; scanThread(Lorg/jikesrvm/scheduler/RVMThread;Lorg/mmtk/plan/TraceLocal;Z)V at line 147 >>> >> (0x70c9ddfc 20814) at [0x70c9ddfc, 0x6476db36] Lorg/jikesrvm/mm/mmtk/Scanning; computeThreadRoots(Lorg/mmtk/plan/TraceLocal;)V at line 189 >>> >> (0x70c9de80 9495) at [0x70c9de80, 0x64790484] Lorg/mmtk/plan/SimpleCollector; collectionPhase(SZ)V at line 62 >>> >> at [0x70c9de80, 0x64790484] Lorg/mmtk/plan/generational/GenCollector; collectionPhase(SZ)V at line 126 >>> >> (0x70c9ded4 9501) at [0x70c9ded4, 0x64ce1810] Lorg/mmtk/plan/generational/copying/GenCopyCollector; collectionPhase(SZ)V at line 158 >>> >> (0x70c9df40 7462) at [0x70c9df40, 0x64ce2549] Lorg/mmtk/plan/Phase; processPhaseStack(Z)V at line 474 >>> >> (0x70c9df70 7460) at [0x70c9df70, 0x6479053b] Lorg/mmtk/plan/Phase; beginNewPhaseStack(I)V at line 385 >>> >> (0x70c9df84 9490) at [0x70c9df84, 0x6475bcc3] Lorg/mmtk/plan/StopTheWorldCollector; collect()V at line 40 >>> >> (0x70c9dfa4 9478) at [0x70c9dfa4, 0x647aa268] Lorg/mmtk/plan/ParallelCollector; run()V at line 50 >>> >> (0x70c9dfc4 15787) at [0x70c9dfc4, 0x64e5df4a] Lorg/jikesrvm/mm/mminterface/CollectorThread; run()V at line 80 >>> >> (0x70c9dffc 21056) at [0x70c9dffc, 0x0804f795] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 2606 >>> >> JikesRVM: exit 124 >>> >> ------------------------------------------------------------------------------ >>> >> _______________________________________________ >>> >> Jikesrvm-researchers mailing list >>> >> [email protected] <mailto:[email protected]> >>> >> https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers <https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers> >>> > >>> > >>> > ------------------------------------------------------------------------------ >>> > _______________________________________________ >>> > Jikesrvm-researchers mailing list >>> > [email protected] <mailto:[email protected]> >>> > https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers <https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers> >>> >>> >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> Jikesrvm-researchers mailing list >>> [email protected] <mailto:[email protected]> >>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers <https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers> >>> >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> Jikesrvm-researchers mailing list >>> [email protected] <mailto:[email protected]> >>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers <https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers> >> >> >> >> ------------------------------------------------------------------------------ >> >> >> _______________________________________________ >> Jikesrvm-researchers mailing list >> [email protected] <mailto:[email protected]> >> https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers <https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers> > > ------------------------------------------------------------------------------ > _______________________________________________ > Jikesrvm-researchers mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers ------------------------------------------------------------------------------ _______________________________________________ Jikesrvm-researchers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers