Re: beam core file R17
Matthew Evans <[email protected]> Sun, 29 May 2016 21:32:58 -0400
| Newsgroups | gmane.comp.lang.erlang.bugs,gmane.comp.lang.erlang.general |
|---|---|
| Message-ID | <[email protected]> |
--===============8577750819219721991== Content-Type: multipart/alternative; boundary="_92ab3938-119c-46dc-8435-73aaedc147c6_" --_92ab3938-119c-46dc-8435-73aaedc147c6_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Thanks=2C Fortunately our latest release has moved all the NIF logic to a separate CN= ODE based process. I am leaning towards the NIF as the cause since the VM i= tself has proven to be very stable. Date: Sun=2C 29 May 2016 21:23:15 +0200 Subject: Re: [erlang-bugs] beam core file R17 From: [email protected] To: [email protected] CC: [email protected] sweep_off_heap is run when collecting refc binaries (among other things)=2C= so if you by mistake have decremented the reference count of a binary in a= nif too much=2C this error will happen when the GC is inspecting the binar= y. I'd recommend looking for something like that in any nifs you have. On Sun=2C May 29=2C 2016 at 6:04 PM=2C Matthew Evans <mattevans123@hotmail.= com> wrote: Thanks=2C this software does use nifs =0A= =0A= Sent from my iPhone =0A= =0A= > On May 29=2C 2016=2C at 11:56 AM=2C Mikael Pettersson <[email protected]= om> wrote: =0A= > =0A= > Matthew Evans writes: =0A= >> Hi=2C =0A= >> This core was found on a live system (R17): =0A= >> =0A= >> =0A= >> =0A= >> =0A= >> =0A= >> =0A= >> =0A= >> =0A= >> 10:16:38:# erl =0A= >> Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] = [hipe] [kernel-poll:false] =0A= >> =0A= >> =0A= >> Eshell V6.2 (abort with ^G) =0A= >> 1> =0A= >> =0A= >> ........ =0A= >> =0A= >> =0A= >> =0A= >> =0A= >> =0A= >> =0A= >> =0A= >> =0A= >> May 28 22:18:56 [info ] plexxi kernel: [1235119.885465] beam.smp[2267]= general protection ip:4b698a sp:7faeb6a7d650 error:0 in beam.smp[400000+1a= c000] =0A= >> warning: Can't read pathname for load map: Input/output error.[Thread de= bugging using libthread_db enabled]Using host libthread_db library "/lib/li= bthread_db.so.1".Core was generated by `/usr/lib/erlang/erts-6.2/bin/beam.s= mp -K true -A 24 -P 350000 -- -root /usr/lib'.Program terminated with signa= l 11=2C Segmentation fault.#0 0x00000000004b698a in sweep_off_heap ()(gdb)= bt#0 0x00000000004b698a in sweep_off_heap ()#1 0x00000000004b77d1 in do_= minor ()#2 0x00000000004b8479 in erts_garbage_collect ()#3 0x00000000004e= 1374 in process_main ()#4 0x000000000048071d in sched_thread_func ()#5 0x= 0000000000549f89 in thr_wrapper ()#6 0x00007faeba32ba30 in start_thread ()= from /lib/libpthread.so.0#7 0x00007faeb9e8a53d in clone () from /lib/libc= .so.6 =0A= > =0A= > It looks like your beam.smp binary lacks debugging information=2C so we o= nly know the general =0A= > area where it crashed (sweep_off_heap() as called from do_minor()). Cras= hes here would usually =0A= > be due to memory corruption=2C which could be caused by: =0A= > - a bug in the VM =0A= > - a bug in a NIF =0A= > - a bug in HiPE =0A= > - a bug in the C compiler used to compile the VM (I've seen that happen a= t least 3 times) =0A= > - a HW error (though you'd then also find e.g. machine check events logge= d) =0A= > =0A= > If you want to debug this=2C you should first ensure that your beam.smp g= ets built and installed =0A= > with full debugging information (just attach gdb=2C bt=2C and list to ver= ify). You should also try =0A= > without NIFs or native code=2C if those are used and you can configure th= em not to be. =0A= _______________________________________________ =0A= erlang-bugs mailing list =0A= [email protected] =0A= http://erlang.org/mailman/listinfo/erlang-bugs =0A= = --_92ab3938-119c-46dc-8435-73aaedc147c6_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html> <head> <style><!-- .hmmessage P { margin:0px=3B padding:0px } body.hmmessage { font-size: 12pt=3B font-family:Calibri } --></style></head> <body class=3D'hmmessage'><div dir=3D'ltr'><br>Thanks=2C<div><br></div><div= >Fortunately our latest release has moved all the NIF logic to a separate C= NODE based process. I am leaning towards the NIF as the cause since the VM = itself has proven to be very stable.</div><div><br></div><div><br><div><hr = id=3D"stopSpelling">Date: Sun=2C 29 May 2016 21:23:15 +0200<br>Subject: Re:= [erlang-bugs] beam core file R17<br>From: [email protected]<br>To: matte= [email protected]<br>CC: [email protected]<br><br><div dir=3D"l= tr">sweep_off_heap is run when collecting refc binaries (among other things= )=2C so if you by mistake have decremented the reference count of a binary = in a nif too much=2C this error will happen when the GC is inspecting the b= inary. I'd recommend looking for something like that in any nifs you have.<= br><div class=3D"ecxgmail_extra"><br><div class=3D"ecxgmail_quote">On Sun= =2C May 29=2C 2016 at 6:04 PM=2C Matthew Evans <span dir=3D"ltr"><=3B<a h= ref=3D"mailto:[email protected]" target=3D"_blank">mattevans123@hotm= ail.com</a>>=3B</span> wrote:<br><blockquote class=3D"ecxgmail_quote" sty= le=3D"border-left:1px #ccc solid=3Bpadding-left:1ex=3B">Thanks=2C this soft= ware does use nifs<br>=0A= <br>=0A= Sent from my iPhone<br>=0A= <div class=3D"ecxHOEnZb"><div class=3D"h5"><br>=0A= >=3B On May 29=2C 2016=2C at 11:56 AM=2C Mikael Pettersson <=3B<a href= =3D"mailto:[email protected]">[email protected]</a>>=3B wrote:<br>= =0A= >=3B<br>=0A= >=3B Matthew Evans writes:<br>=0A= >=3B>=3B Hi=2C<br>=0A= >=3B>=3B This core was found on a live system (R17):<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B 10:16:38:# erl<br>=0A= >=3B>=3B Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-th= reads:10] [hipe] [kernel-poll:false]<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B Eshell V6.2 =3B (abort with ^G)<br>=0A= >=3B>=3B 1>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B ........<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B<br>=0A= >=3B>=3B May 28 22:18:56 [info =3B  =3B] plexxi kernel: [123511= 9.885465] beam.smp[2267] general protection ip:4b698a sp:7faeb6a7d650 error= :0 in beam.smp[400000+1ac000]<br>=0A= >=3B>=3B warning: Can't read pathname for load map: Input/output error.= [Thread debugging using libthread_db enabled]Using host libthread_db librar= y "/lib/libthread_db.so.1".Core was generated by `/usr/lib/erlang/erts-6.2/= bin/beam.smp -K true -A 24 -P 350000 -- -root /usr/lib'.Program terminated = with signal 11=2C Segmentation fault.#0 =3B 0x00000000004b698a in sweep= _off_heap ()(gdb) bt#0 =3B 0x00000000004b698a in sweep_off_heap ()#1&nb= sp=3B 0x00000000004b77d1 in do_minor ()#2 =3B 0x00000000004b8479 in ert= s_garbage_collect ()#3 =3B 0x00000000004e1374 in process_main ()#4 = =3B 0x000000000048071d in sched_thread_func ()#5 =3B 0x0000000000549f89= in thr_wrapper ()#6 =3B 0x00007faeba32ba30 in start_thread () from /li= b/libpthread.so.0#7 =3B 0x00007faeb9e8a53d in clone () from /lib/libc.s= o.6<br>=0A= >=3B<br>=0A= >=3B It looks like your beam.smp binary lacks debugging information=2C so= we only know the general<br>=0A= >=3B area where it crashed (sweep_off_heap() as called from do_minor()).&= nbsp=3B Crashes here would usually<br>=0A= >=3B be due to memory corruption=2C which could be caused by:<br>=0A= >=3B - a bug in the VM<br>=0A= >=3B - a bug in a NIF<br>=0A= >=3B - a bug in HiPE<br>=0A= >=3B - a bug in the C compiler used to compile the VM (I've seen that hap= pen at least 3 times)<br>=0A= >=3B - a HW error (though you'd then also find e.g. machine check events = logged)<br>=0A= >=3B<br>=0A= >=3B If you want to debug this=2C you should first ensure that your beam.= smp gets built and installed<br>=0A= >=3B with full debugging information (just attach gdb=2C bt=2C and list t= o verify). =3B You should also try<br>=0A= >=3B without NIFs or native code=2C if those are used and you can configu= re them not to be.<br>=0A= _______________________________________________<br>=0A= erlang-bugs mailing list<br>=0A= <a href=3D"mailto:[email protected]">[email protected]</a><br>=0A= <a href=3D"http://erlang.org/mailman/listinfo/erlang-bugs" rel=3D"noreferre= r" target=3D"_blank">http://erlang.org/mailman/listinfo/erlang-bugs</a><br>= =0A= </div></div></blockquote></div><br></div></div></div></div> </di= v></body> </html>= --_92ab3938-119c-46dc-8435-73aaedc147c6_-- --===============8577750819219721991== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ erlang-bugs mailing list [email protected] http://erlang.org/mailman/listinfo/erlang-bugs --===============8577750819219721991==--