Re: Problem in Thread.c
"Sébastien Adam" <[email protected]> Mon, 17 Apr 2006 15:42:05 +0000
| Newsgroups | gmane.comp.java.vm.sablevm.devel |
|---|---|
| Message-ID | <[email protected]> |
--===============0300591828== Content-Type: multipart/alternative; boundary="----=_Part_14356_14475745.1145288525949" ------=_Part_14356_14475745.1145288525949 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline > ./configure --with-gc=3Dgencopy ./configure --with-gc=3Dgc > You shouldn't need to "trace" anything in the older generation if the write barrier is set up correctly. A correct write barrier is necessary regardless of where large objects are initially allocated. The remembered set needs to be updated with any potential intergenerational references. When a large object is first allocated, none of its fields will have valid references (all zero/NULL). They will only be updated through later mutator actions that should be caught by a write barrier. Right... my mistake... I need to take a break to think about that problem..= . for now... I don't have time... But if you have some hints... don't hesitat= e to let me know... Actually, you can see a big difference in the gc time when you allocate the large object in the oldest generation. With SableCC 2.18, many big objects die young. If these objects are allocated in the oldest generation, they ar= e collected only when a full gc happens. This increase significantly the gc time. Maybe I can increase the threshold between large and small objects (1024 -> 8096 or even more). But the problem happens when the threshold is 1024 bytes. On 4/17/06, Carl Lebsack <[email protected]> wrote: > > On 4/17/06, S=E9bastien Adam <[email protected]> wrote: > > > I am curious. What type of generational collector are you writing? > > Have you looked at the one included with SableVM currently? If so, > > what are the major differences? > > > > Yes... I have looked at the one included with SableVM but it doesn't > work. I > > build my generational gc from scrash. It is quite different from the > other. > > The write barrier is not the same, the algorithm to select the > collection > > (minor, major, full) is not the same... In fact, pratically everything > is > > different. > > > > I am writing a report. Maybe I will publish it as soon as possible. > > > > > > ./configure --with-gc=3Dgencopy > > This doesn't work? How so? I built the SDK just last week and ran it > on several benchmarks. What problem do you see, and do you have a > test case that produces a problem? > > > > If putting your larger objects into the nursery fixed your problem, i= t > > sounds like you may have a write barrier leak. What large object > > threshold size were you using? Sounds like the problem goes away > > because of recursive tracing of the fields when placed in the nursery. > > If large objects are allocated directly in the old generation, then > > updates to their fields need to be tracked by a write barrier. > > > > No... I don't recursively trace the heap but I trace the fields of the > moved > > object. I use a modified write barrier to update these fields. > > > > Actually, I have found the problem. My large object fields may > reference > > the younger generations and these references are not in the remembered > set. > > Since the oldest generation is never traced (we use the remembered set)= , > > some objects die prematuraly. > > This is what I meant. If the objects are allocated in the nursery > they will get traced. > > > > > I must trace the fields of my large objects when I allocate them in the > > oldest generation (only when the first gc after the creation happens). = I > > don't do that by now... there is the problem. > > You shouldn't need to "trace" anything in the older generation if the > write barrier is set up correctly. A correct write barrier is > necessary regardless of where large objects are initially allocated. > The remembered set needs to be updated with any potential > intergenerational references. When a large object is first allocated, > none of its fields will have valid references (all zero/NULL). They > will only be updated through later mutator actions that should be > caught by a write barrier. > > Carl Lebsack > > > > > > > > You change may not have actually fixed the problem, but simply hidden > it > > in your current test case. > > > > I know... but for now... I have a report and some results to produce... > I > > will fix the problem this week... > > > > Thanks for your help... > > > > Seb > > > > > > On 4/17/06, Carl Lebsack <[email protected]> wrote: > > > I am curious. What type of generational collector are you writing? > > > Have you looked at the one included with SableVM currently? If so, > > > what are the major differences? > > > > > > If putting your larger objects into the nursery fixed your problem, i= t > > > sounds like you may have a write barrier leak. What large object > > > threshold size were you using? Sounds like the problem goes away > > > because of recursive tracing of the fields when placed in the nursery= . > > > If large objects are allocated directly in the old generation, then > > > updates to their fields need to be tracked by a write barrier. You > > > change may not have actually fixed the problem, but simply hidden it > > > in your current test case. > > > > > > Carl Lebsack > > > > > > On 4/17/06, S=E9bastien Adam < [email protected]> wrote: > > > > > This doesn't imply the bug is in SableVM. > > > > > > > > I agree with you... It's why I said "maybe". > > > > > > > > > > > > > If a single reference isn't updated correctly, or a reference is > > updated > > > > incorrectly the VM can crash. > > > > > > > > Right.... I met this problem a couple days ago. > > > > > > > > > > > > > Changing the heap size would allow some runs to complete > successfully > > > > while others would crash. > > > > > > > > I allocated the larger objects in the oldest generation. I think > this is > > the > > > > source of the problem. Since I allocate these objects in the > nursery, > > > > everything works perfectly... and for all heap size... > > > > > > > > > > > > > > > > > > > > On 4/17/06, Carl Lebsack <[email protected]> wrote: > > > > > This doesn't imply the bug is in SableVM. Garbage collectors are > > > > > extremely sensitive to correctness. If a single reference isn't > > > > > updated correctly, or a reference is updated incorrectly the VM > can > > > > > crash. However, the crash (segfault etc.) does not have to appea= r > at > > > > > the source of the problem, but at some point after the > problem. The > > > > > fact that changing the heap size has an effect probably means > there is > > > > > still an issue with garbage collection and the GC trigger point i= s > > > > > what causes the variation. This is a guess based on my own > experience > > > > > writing a generational collector for SableVM. Missing a single > write > > > > > barrier location caused a host of bizarre problems to manifest in > > > > > various parts of the VM. Changing the heap size would allow some > runs > > > > > to complete successfully while others would crash. > > > > > > > > > > Good luck. > > > > > > > > > > Carl Lebsack > > > > > > > > > > On 4/16/06, S=E9bastien Adam <[email protected]> wrote: > > > > > > Finally, maybe the bug is in sablevm. When I set my heap size > with > > the > > > > > > following value: > > > > > > > > > > > > #define SVM_DEFAULT_GENERATION_SIZE SVM_DEFAULT_FRAME_SIZE * > 1536 > > > > > > > > > > > > Everything works perfectly... > > > > > > > > > > > > So, depending of the heap size sablevm will produce the good > result > > or > > > > > > not... > > > > > > > > > > > > > > > > > > > > > > > > On 4/16/06, Etienne Gagnon < [email protected]> wrote: > > > > > > > > > > > > > Actually, the bug is possibly in sablevm... Look at: > > > > > > http://sablevm.org/bugs/172 > > > > > > > > > > > > Also, note that you should be able to change such value using a > > runtime > > > > > > system property, like > > > > > > > > > > > > sablevm --preperty=3D sablevm.heap.increment=3D1024 > > > > > > or > > > > > > sablevm > > > > --preperty=3Dsablevm.heap.default.generation.size=3D... > > > > > > > > > > > > look in vm_args.m4.c > > > > > > > > > > > > Etienne > > > > > > > > > > > > S=E9bastien Adam wrote: > > > > > > > Last news... > > > > > > > > > > > > > > When I change the heap size... for example: > > > > > > > > > > > > > > In gc_gencopy.c, > > > > > > > > > > > > > > we replace: > > > > > > > #define SVM_DEFAULT_GENERATION_SIZE SVM_DEFAULT_FRAME_SIZE * > 1024 > > > > > > > with: > > > > > > > #define SVM_DEFAULT_GENERATION_SIZE SVM_DEFAULT_FRAME_SIZE * > 512 > > > > > > > > > > > > > > I got the following error: > > > > > > > sablevm: gc_gencopy.c:455: _svmf_copy_object: Assertion > `((void *) > > > > > > > obj->lockword) >=3D heap->to_generation-> space.start && ((vo= id > *) > > > > > > > obj->lockword) < heap->to_generation-> alloc.start' failed. > > > > > > > > > > > > > > hummmm... this is really bad... It seems to be a problem in m= y > gc > > :( > > > > > > > > > > > > -- > > > > > > Etienne M. Gagnon, Ph.D. > > > > > > http://www.info2.uqam.ca/~egagnon/ > > > > > > SableVM: > > > > > > http://www.sablevm.org/ > > > > > > SableCC: > > > > > > http://www.sablecc.org/ > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > SableVM-devel mailing list > > > > > > [email protected] > > > > > > > > http://sablevm.org/lists/control/listinfo/sablevm-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > SableVM-devel mailing list > > > > > > [email protected] > > > > > > > > http://sablevm.org/lists/control/listinfo/sablevm-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > SableVM-devel mailing list > > > > > [email protected] > > > > > > > http://sablevm.org/lists/control/listinfo/sablevm-devel > > > > > > > > > > > > > > > > > _______________________________________________ > > > > SableVM-devel mailing list > > > > [email protected] > > > > http://sablevm.org/lists/control/listinfo/sablevm-devel > > > > > > > > > > > > > > > > > > _______________________________________________ > > > SableVM-devel mailing list > > > [email protected] > > > http://sablevm.org/lists/control/listinfo/sablevm-devel > > > > > > > > > _______________________________________________ > > SableVM-devel mailing list > > [email protected] > > http://sablevm.org/lists/control/listinfo/sablevm-devel > > > > > > > > _______________________________________________ > SableVM-devel mailing list > [email protected] > http://sablevm.org/lists/control/listinfo/sablevm-devel > ------=_Part_14356_14475745.1145288525949 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline > ./configure --with-gc=3Dgencopy<br> <br> ./configure --with-gc=3Dgc<br> <br> > You shouldn't need to "trace" anything in the older generati= on if the<br> write barrier is set up correctly. A correct write barrier is<br= > necessary regardless of where large objects are initially allocated.<br> The remembered set needs to be updated with any potential<br> intergenerational references. When a large object is first alloc= ated,<br> none of its fields will have valid references (all zero/NULL). T= hey<br> will only be updated through later mutator actions that should be<br> caught by a write barrier.<br><br> Right... my mistake... I need to take a break to think about that problem... for now... I don't have time... But if you have some hints... don't hesitate to let me know...<br> <br> Actually, you can see a big difference in the gc time when you allocate the large object in the oldest generation. With SableCC 2.18, many big objects die young. If these objects are allocated in the oldest generation, they are collected only when a full gc happens. This increase significantly the gc time. Maybe I can increase the threshold between large and small objects (1024 -> 8096 or even more). But the problem happens when the threshold is 1024 bytes.<br= > <br> <br> <br> <br> <br> <br><div><span class=3D"gmail_quote">On 4/17/06, <b class=3D"gmail_senderna= me">Carl Lebsack</b> <<a href=3D"mailto:[email protected]">lebsack@ias= tate.edu</a>> wrote:</span><blockquote class=3D"gmail_quote" style=3D"bo= rder-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding= -left: 1ex;"> On 4/17/06, S=E9bastien Adam <<a href=3D"mailto:[email protected]= ">[email protected]</a>> wrote:<br>> > I am curious. &= nbsp;What type of generational collector are you writing?<br>> Have you = looked at the one included with SableVM currently? If so, <br>> what are the major differences?<br>><br>> Yes... I have look= ed at the one included with SableVM but it doesn't work. I<br>> build my= generational gc from scrash. It is quite different from the other.<br> > The write barrier is not the same, the algorithm to select the collect= ion<br>> (minor, major, full) is not the same... In fact, pratically eve= rything is<br>> different.<br>><br>> I am writing a report. &= nbsp;Maybe I will publish it as soon as possible. <br>><br>><br><br>./configure --with-gc=3Dgencopy<br><br>This doesn't= work? How so? I built the SDK just last week and ran= it<br>on several benchmarks. What problem do you see, and do yo= u have a<br>test case that produces a problem? <br><br>> > If putting your larger objects into the nursery fixed you= r problem, it<br>> sounds like you may have a write barrier leak. &= nbsp;What large object<br>> threshold size were you using? So= unds like the problem goes away <br>> because of recursive tracing of the fields when placed in the nurs= ery.<br>> If large objects are allocated directly in the old generation,= then<br>> updates to their fields need to be tracked by a write barrier= . <br>><br>> No... I don't recursively trace the heap but I trace the f= ields of the moved<br>> object. I use a modified write barrier to update= these fields.<br>><br>> Actually, I have found the problem. &nb= sp;My large object fields may reference <br>> the younger generations and these references are not in the rememb= ered set.<br>> Since the oldest generation is never traced (we use the r= emembered set),<br>> some objects die prematuraly.<br><br>This is what I= meant. If the objects are allocated in the nursery <br>they will get traced.<br><br>><br>> I must trace the fields of my= large objects when I allocate them in the<br>> oldest generation (only = when the first gc after the creation happens). I<br>> don't do that by n= ow... there is the problem. <br><br>You shouldn't need to "trace" anything in the older gener= ation if the<br>write barrier is set up correctly. A correct wri= te barrier is<br>necessary regardless of where large objects are initially = allocated. <br>The remembered set needs to be updated with any potential<br>intergener= ational references. When a large object is first allocated,<br>n= one of its fields will have valid references (all zero/NULL). Th= ey<br>will only be updated through later mutator actions that should be <br>caught by a write barrier.<br><br>Carl Lebsack<br><br>><br>><br>&= gt; > You change may not have actually fixed the problem, but simply hid= den it<br>> in your current test case.<br>><br>> I know... but for= now... I have a report and some results to produce... I <br>> will fix the problem this week...<br>><br>> Thanks for your = help...<br>><br>> Seb<br>><br>><br>> On 4/17/06, Carl Lebsac= k <<a href=3D"mailto:[email protected]">[email protected]</a>> wr= ote: <br>> > I am curious. What type of generational collector = are you writing?<br>> > Have you looked at the one included with Sabl= eVM currently? If so,<br>> > what are the major difference= s?<br>> ><br> > > If putting your larger objects into the nursery fixed your proble= m, it<br>> > sounds like you may have a write barrier leak. &nbs= p;What large object<br>> > threshold size were you using? = Sounds like the problem goes away <br>> > because of recursive tracing of the fields when placed in the= nursery.<br>> > If large objects are allocated directly in the old g= eneration, then<br>> > updates to their fields need to be tracked by = a write barrier. You <br>> > change may not have actually fixed the problem, but simply hi= dden it<br>> > in your current test case.<br>> ><br>> > C= arl Lebsack<br>> ><br>> > On 4/17/06, S=E9bastien Adam < <a = href=3D"mailto:[email protected]"> [email protected]</a>> wrote:<br>> > > > This doesn't= imply the bug is in SableVM.<br>> > ><br>> > > I agree w= ith you... It's why I said "maybe".<br>> > ><br>> >= ; > <br>> > > > If a single reference isn't updated correctly, or a= reference is<br>> updated<br>> > > incorrectly the VM can cras= h.<br>> > ><br>> > > Right.... I met this problem a coupl= e days ago. <br>> > ><br>> > ><br>> > > > Changing the he= ap size would allow some runs to complete successfully<br>> > > wh= ile others would crash.<br>> > ><br>> > > I allocated the= larger objects in the oldest generation. I think this is <br>> the<br>> > > source of the problem. Since I allocate thes= e objects in the nursery,<br>> > > everything works perfectly... a= nd for all heap size...<br>> > ><br>> > ><br>> > &g= t; <br>> > ><br>> > > On 4/17/06, Carl Lebsack <<a href= =3D"mailto:[email protected]">[email protected]</a>> wrote:<br>> = > > > This doesn't imply the bug is in SableVM. Garbage= collectors are <br>> > > > extremely sensitive to correctness. If a= single reference isn't<br>> > > > updated correctly, or a refe= rence is updated incorrectly the VM can<br>> > > > crash. = However, the crash (segfault etc.) does not have to appear at <br>> > > > the source of the problem, but at some point after = the problem. The<br>> > > > fact that changing the h= eap size has an effect probably means there is<br>> > > > still= an issue with garbage collection and the GC trigger point is <br>> > > > what causes the variation. This is a gue= ss based on my own experience<br>> > > > writing a generational= collector for SableVM. Missing a single write<br>> > >= > barrier location caused a host of bizarre problems to manifest in <br>> > > > various parts of the VM. Changing the he= ap size would allow some runs<br>> > > > to complete successful= ly while others would crash.<br>> > > ><br>> > > > = Good luck. <br>> > > ><br>> > > > Carl Lebsack<br>> > &g= t; ><br>> > > > On 4/16/06, S=E9bastien Adam <<a href=3D"= mailto:[email protected]">[email protected]</a>> wrote:<br= > > > > > > Finally, maybe the bug is in sablevm. When I set m= y heap size with<br>> the<br>> > > > > following value:<b= r>> > > > ><br>> > > > > #define SVM_DEFAULT_= GENERATION_SIZE SVM_DEFAULT_FRAME_SIZE * 1536 <br>> > > > ><br>> > > > > Everything works p= erfectly...<br>> > > > ><br>> > > > > So, dep= ending of the heap size sablevm will produce the good result<br>> or <br>> > > > > not...<br>> > > > ><br>> >= ; > > ><br>> > > > ><br>> > > > > On= 4/16/06, Etienne Gagnon < <a href=3D"mailto:[email protected]">egagno= [email protected] </a>> wrote:<br>> > > > > ><br>> > > > >= ; Actually, the bug is possibly in sablevm... Look at:<br>> &= gt; > > > <a href=3D"http://sablevm.org/bugs/172">http://sablevm.o= rg/bugs/172 </a><br>> > > > ><br>> > > > > Also, note tha= t you should be able to change such value using a<br>> runtime<br>> &= gt; > > > system property, like<br>> > > > ><br> > > > > > sablevm --preperty=3D sablevm.heap.increment=3D102= 4<br>> > > > > or<br>> > > > > sablevm<br>>= ; > > --preperty=3Dsablevm.heap.default.generation.size=3D...<br>>= > > > > <br>> > > > > look in vm_args.m4.c<br>> > > > &g= t;<br>> > > > > Etienne<br>> > > > ><br>> = > > > > S=E9bastien Adam wrote:<br>> > > > > >= ; Last news... <br>> > > > > ><br>> > > > > > When I c= hange the heap size... for example:<br>> > > > > ><br>>= ; > > > > > In gc_gencopy.c,<br>> > > > > >= ; <br>> > > > > > we replace:<br>> > > > > &= gt; #define SVM_DEFAULT_GENERATION_SIZE SVM_DEFAULT_FRAME_SIZE * 1024<br>&g= t; > > > > > with:<br>> > > > > > #define = SVM_DEFAULT_GENERATION_SIZE SVM_DEFAULT_FRAME_SIZE * 512 <br>> > > > > ><br>> > > > > > I got th= e following error:<br>> > > > > > sablevm: gc_gencopy.c:4= 55: _svmf_copy_object: Assertion `((void *)<br>> > > > > >= ; obj->lockword) >=3D heap->to_generation->=20 space.start && ((void *)<br>> > > > > > obj->l= ockword) < heap->to_generation-> alloc.start' failed.<br>> >= > > > ><br>> > > > > > hummmm... this is rea= lly bad... It seems to be a problem in my gc <br>> :(<br>> > > > ><br>> > > > > --<br>&= gt; > > > > Etienne M. Gagnon, Ph.D.<br>> > > > >= ; <a href=3D"http://www.info2.uqam.ca/~egagnon/">http://www.info2.uqam.ca/~= egagnon/ </a><br>> > > > > SableVM:<br>> > > > > <a hr= ef=3D"http://www.sablevm.org/">http://www.sablevm.org/</a><br>> > >= ; > > SableCC:<br>> > > > > <a href=3D"http://www.sabl= ecc.org/"> http://www.sablecc.org/</a><br>> > > > ><br>> > > &= gt; ><br>> > > > ><br>> > > > > __________= _____________________________________<br>> > > > ><br>> &= gt; > > > SableVM-devel mailing list <br>> > > > > <a href=3D"mailto:[email protected]">S= [email protected]</a><br>> > > > ><br>> <a href=3D= "http://sablevm.org/lists/control/listinfo/sablevm-devel">http://sablevm.or= g/lists/control/listinfo/sablevm-devel </a><br>> > > > ><br>> > > > ><br>> > &= gt; > ><br>> > > > ><br>> > > > ><br>&g= t; > > > ><br>> > > > > ________________________= _______________________ <br>> > > > > SableVM-devel mailing list<br>> > > &= gt; > <a href=3D"mailto:[email protected]">SableVM-devel@sablevm= .org</a><br>> > > > ><br>> <a href=3D"http://sablevm.org/= lists/control/listinfo/sablevm-devel"> http://sablevm.org/lists/control/listinfo/sablevm-devel</a><br>> > &g= t; > ><br>> > > > ><br>> > > > ><br>>= ; > > ><br>> > > > ___________________________________= ____________ <br>> > > > SableVM-devel mailing list<br>> > > > <= a href=3D"mailto:[email protected]">[email protected]</a><b= r>> > > ><br>> <a href=3D"http://sablevm.org/lists/control/l= istinfo/sablevm-devel"> http://sablevm.org/lists/control/listinfo/sablevm-devel</a><br>> > &g= t; ><br>> > ><br>> > ><br>> > > _____________= __________________________________<br>> > > SableVM-devel mailing = list <br>> > > <a href=3D"mailto:[email protected]">SableVM-dev= [email protected]</a><br>> > > <a href=3D"http://sablevm.org/lists/co= ntrol/listinfo/sablevm-devel">http://sablevm.org/lists/control/listinfo/sab= levm-devel </a><br>> > ><br>> > ><br>> > ><br>> ><br>= > > _______________________________________________<br>> > Sabl= eVM-devel mailing list<br>> > <a href=3D"mailto:SableVM-devel@sablevm= .org"> [email protected]</a><br>> > <a href=3D"http://sablevm.org/li= sts/control/listinfo/sablevm-devel">http://sablevm.org/lists/control/listin= fo/sablevm-devel</a><br>> ><br>><br>><br>> _________________= ______________________________ <br>> SableVM-devel mailing list<br>> <a href=3D"mailto:SableVM-devel= @sablevm.org">[email protected]</a><br>> <a href=3D"http://sable= vm.org/lists/control/listinfo/sablevm-devel">http://sablevm.org/lists/contr= ol/listinfo/sablevm-devel </a><br>><br>><br>><br><br>_______________________________________= ________<br>SableVM-devel mailing list<br><a href=3D"mailto:SableVM-devel@s= ablevm.org">[email protected]</a><br><a href=3D"http://sablevm.org/= lists/control/listinfo/sablevm-devel"> http://sablevm.org/lists/control/listinfo/sablevm-devel</a><br></blockquote= ></div><br> ------=_Part_14356_14475745.1145288525949-- --===============0300591828== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ SableVM-devel mailing list [email protected] http://sablevm.org/lists/control/listinfo/sablevm-devel --===============0300591828==--