Re: PostgreSQL 12 crash with segmentation violation in heap_freetuple

Marcel Ruff <[email protected]> Tue, 3 Dec 2019 11:12:36 +0100
Newsgroups gmane.comp.db.postgresql.novice
Organization Netwake GmbH
Message-ID <[email protected]>
This is a multi-part message in MIME format.
--------------BA51159822242DA14E190FE8
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

FYI

with the self compiled release 12.1 the crash has disappeared,

thank you for your world leading DB

Marcel

06.11.19 um 07:48 Marcel Ruff:
>
> Hi,
>
> here is the crash dump, it is the current postgresql 12 release (not
> yet a current git clone) on Debian Linux with 40G RAM and AMD Opteron
> 23xx (Gen 3 Class Opteron)
>
>     ./configure --build=x86_64-linux-gnu --host=x86_64-linux-gnu
> --target=x86_64-linux-gnu --with-pam --with-openssl --with-libxml
> –with-libxslt –with-llvm --enable-debug
>     make -j 8
>     make install
>     cd contrib
>     make -j 8
>     make install
>
> and the default postgresql.conf but with jit = false (it also happens
> with jit = on):
>
> --------------
> [New LWP 21468]
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> Core was generated by `postgres: watchee watchee 127.'.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0  0x000055ceb9518cc3 in GetMemoryChunkContext (pointer=0x0) at
> ../../../../src/include/utils/memutils.h:127
> 127             context = *(MemoryContext *) (((char *) pointer) -
> sizeof(void *));
> (gdb) where
> #0  0x000055ceb9518cc3 in GetMemoryChunkContext (pointer=0x0) at
> ../../../../src/include/utils/memutils.h:127
> #1  pfree (pointer=0x0) at mcxt.c:1033
> #2  0x000055ceb90fcfe5 in heap_freetuple (htup=<optimized out>) at
> heaptuple.c:1340
> #3  0x000055ceb92931a1 in tts_buffer_heap_clear (slot=0x55ceba097a18)
> at execTuples.c:652
> #4  0x000055ceb9293b05 in ExecClearTuple (slot=0x55ceba097a18) at
> ../../../src/include/executor/tuptable.h:428
> #5  ExecForceStoreHeapTuple (tuple=0x55ceb9aad760,
> slot=0x55ceba097a18, shouldFree=<optimized out>) at execTuples.c:1446
> #6  0x000055ceb926b851 in ExecBRUpdateTriggers
> (estate=estate@entry=0x55ceb9a4e090,
> epqstate=epqstate@entry=0x55ceb9a4ed28,
> relinfo=relinfo@entry=0x55ceb9a4e300,
>     tupleid=tupleid@entry=0x7fff3cbcb92a,
> fdw_trigtuple=fdw_trigtuple@entry=0x0,
> newslot=newslot@entry=0x55ceba097a18) at trigger.c:3109
> #7  0x000055ceb92ac464 in ExecUpdate
> (mtstate=mtstate@entry=0x55ceb9a4ec30, tupleid=0x7fff3cbcb92a,
> oldtuple=0x0, slot=0x55ceba097a18, planSlot=0x55ceba0978b8,
>     epqstate=epqstate@entry=0x55ceb9a4ed28, estate=0x55ceb9a4e090,
> canSetTag=true) at nodeModifyTable.c:1072
> #8  0x000055ceb92ad862 in ExecModifyTable (pstate=0x55ceb9a4ec30) at
> nodeModifyTable.c:2223
> #9  0x000055ceb928860b in ExecProcNode (node=0x55ceb9a4ec30) at
> ../../../src/include/executor/executor.h:239
> #10 ExecutePlan (execute_once=<optimized out>, dest=0x55ceb977fb60
> <donothingDR>, direction=<optimized out>, numberTuples=0,
> sendTuples=<optimized out>, operation=CMD_UPDATE,
>     use_parallel_mode=<optimized out>, planstate=0x55ceb9a4ec30,
> estate=0x55ceb9a4e090) at execMain.c:1646
> #11 standard_ExecutorRun (queryDesc=0x55ceba065378,
> direction=<optimized out>, count=0, execute_once=<optimized out>) at
> execMain.c:364
> #12 0x000055ceb93d9140 in ProcessQuery (plan=<optimized out>,
>     sourceText=0x55ceba065140 "update Location set enmea=$1, guid=$2,
> latDeci=$3, lonDeci=$4, modifiedByLoginName=$5, poiObjectId=$6,
> timestamp=$7, TRACK_ID=$8 where LOCATION_ID=$9",
> params=0x55ceba065268, queryEnv=0x0, dest=0x55ceb977fb60
> <donothingDR>, completionTag=0x7fff3cbcbdd0 "") at pquery.c:161
> #13 0x000055ceb93d9370 in PortalRunMulti
> (portal=portal@entry=0x55ceb9a31750, isTopLevel=isTopLevel@entry=true,
> setHoldSnapshot=setHoldSnapshot@entry=false,
>     dest=0x55ceb977fb60 <donothingDR>, dest@entry=0x55ceb99cd950,
> altdest=0x55ceb977fb60 <donothingDR>, altdest@entry=0x55ceb99cd950,
>     completionTag=completionTag@entry=0x7fff3cbcbdd0 "") at pquery.c:1283
> #14 0x000055ceb93d9e51 in PortalRun
> (portal=portal@entry=0x55ceb9a31750, count=count@entry=1,
> isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=false,
>     dest=dest@entry=0x55ceb99cd950,
> altdest=altdest@entry=0x55ceb99cd950, completionTag=0x7fff3cbcbdd0 "")
> at pquery.c:796
> #15 0x000055ceb93d7557 in exec_execute_message (max_rows=1,
> portal_name=0x55ceb99cd540 "") at postgres.c:2090
> #16 PostgresMain (argc=<optimized out>,
> argv=argv@entry=0x55ceb99f5958, dbname=<optimized out>,
> username=<optimized out>) at postgres.c:4297
> --Type <RET> for more, q to quit, c to continue without paging--
> #17 0x000055ceb9363ce2 in BackendRun (port=0x55ceb99f1920,
> port=0x55ceb99f1920) at postmaster.c:4431
> #18 BackendStartup (port=0x55ceb99f1920) at postmaster.c:4122
> #19 ServerLoop () at postmaster.c:1704
> #20 0x000055ceb9364c10 in PostmasterMain (argc=1, argv=0x55ceb99c7270)
> at postmaster.c:1377
> #21 0x000055ceb90f3886 in main (argc=1, argv=0x55ceb99c7270) at main.c:228
>
> Thank you
> Marcel
>
> Am 04.11.19 um 15:24 schrieb Tom Lane:
>> Alvaro Herrera <[email protected]> writes:
>>> On 2019-Nov-04, Marcel Ruff wrote:
>>>> Now I have upgraded to postgresql 12 and get a crash about once per day,
>>>> ...
>>>> How can I track down the problem?
>>> The first you need to do is obtain a crash dump.  This page may help:
>>> https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD#Getting_a_trace_from_a_reproducibly_crashing_backend
>> Also, as long as you're building from source anyway, you might pull down
>> v12 branch tip from our git repo, or use a nightly snapshot tarball [1]
>> to see if the problem's already fixed.
>>
>> 			regards, tom lane
>>
>> [1] https://www.postgresql.org/ftp/snapshot/12/
> -- 
> NetwakeVision
> Alte Owinger Straße 100
> D-88662 Überlingen
> Phone: +49 7551 309372
> http://www.netwakevision.com
> http://www.royal-gps.com
-- 
NetwakeVision
Alte Owinger Straße 100
D-88662 Überlingen
Phone: +49 7551 309372
http://www.netwakevision.com
http://www.royal-gps.com

--------------BA51159822242DA14E190FE8
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p><font color="#000000">FYI</font></p>
    <p><font color="#000000">with the self compiled release 12.1 the
        crash has disappeared,</font></p>
    <p><font color="#000000">thank you for your world leading DB</font></p>
    <p><font color="#000000">Marcel</font><br>
    </p>
    <div class="moz-cite-prefix">06.11.19 um 07:48 Marcel Ruff:<br>
    </div>
    <blockquote type="cite"
      cite="mid:[email protected]">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <p><font color="#000000">Hi,</font></p>
      <p><font color="#000000">here is the crash dump, it is the current
          postgresql 12 release (not yet a current git clone) on Debian
          Linux with 40G RAM and AMD Opteron 23xx (Gen 3 Class Opteron)<br>
        </font></p>
      <p><font color="#000000">    ./configure --build=x86_64-linux-gnu
          --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-pam
          --with-openssl --with-libxml –with-libxslt –with-llvm
          --enable-debug<br>
              make -j 8<br>
              make install<br>
              cd contrib<br>
              make -j 8<br>
              make install</font></p>
      <p><font color="#000000">and the default postgresql.conf but with
          jit = false (it also happens with jit = on):<br>
        </font></p>
      <p><font color="#000000">--------------<br>
          [New LWP 21468]<br>
          [Thread debugging using libthread_db enabled]<br>
          Using host libthread_db library
          "/lib/x86_64-linux-gnu/libthread_db.so.1".<br>
          Core was generated by `postgres: watchee watchee 127.'.<br>
          Program terminated with signal SIGSEGV, Segmentation fault.<br>
          #0  0x000055ceb9518cc3 in GetMemoryChunkContext (pointer=0x0)
          at ../../../../src/include/utils/memutils.h:127<br>
          127             context = *(MemoryContext *) (((char *)
          pointer) - sizeof(void *));<br>
          (gdb) where<br>
          #0  0x000055ceb9518cc3 in GetMemoryChunkContext (pointer=0x0)
          at ../../../../src/include/utils/memutils.h:127<br>
          #1  pfree (pointer=0x0) at mcxt.c:1033<br>
          #2  0x000055ceb90fcfe5 in heap_freetuple (htup=&lt;optimized
          out&gt;) at heaptuple.c:1340<br>
          #3  0x000055ceb92931a1 in tts_buffer_heap_clear
          (slot=0x55ceba097a18) at execTuples.c:652<br>
          #4  0x000055ceb9293b05 in ExecClearTuple (slot=0x55ceba097a18)
          at ../../../src/include/executor/tuptable.h:428<br>
          #5  ExecForceStoreHeapTuple (tuple=0x55ceb9aad760,
          slot=0x55ceba097a18, shouldFree=&lt;optimized out&gt;) at
          execTuples.c:1446<br>
          #6  0x000055ceb926b851 in ExecBRUpdateTriggers
          (estate=estate@entry=0x55ceb9a4e090,
          epqstate=epqstate@entry=0x55ceb9a4ed28,
          relinfo=relinfo@entry=0x55ceb9a4e300, <br>
              tupleid=tupleid@entry=0x7fff3cbcb92a,
          fdw_trigtuple=fdw_trigtuple@entry=0x0,
          newslot=newslot@entry=0x55ceba097a18) at trigger.c:3109<br>
          #7  0x000055ceb92ac464 in ExecUpdate
          (mtstate=mtstate@entry=0x55ceb9a4ec30, tupleid=0x7fff3cbcb92a,
          oldtuple=0x0, slot=0x55ceba097a18, planSlot=0x55ceba0978b8, <br>
              epqstate=epqstate@entry=0x55ceb9a4ed28,
          estate=0x55ceb9a4e090, canSetTag=true) at
          nodeModifyTable.c:1072<br>
          #8  0x000055ceb92ad862 in ExecModifyTable
          (pstate=0x55ceb9a4ec30) at nodeModifyTable.c:2223<br>
          #9  0x000055ceb928860b in ExecProcNode (node=0x55ceb9a4ec30)
          at ../../../src/include/executor/executor.h:239<br>
          #10 ExecutePlan (execute_once=&lt;optimized out&gt;,
          dest=0x55ceb977fb60 &lt;donothingDR&gt;,
          direction=&lt;optimized out&gt;, numberTuples=0,
          sendTuples=&lt;optimized out&gt;, operation=CMD_UPDATE, <br>
              use_parallel_mode=&lt;optimized out&gt;,
          planstate=0x55ceb9a4ec30, estate=0x55ceb9a4e090) at
          execMain.c:1646<br>
          #11 standard_ExecutorRun (queryDesc=0x55ceba065378,
          direction=&lt;optimized out&gt;, count=0,
          execute_once=&lt;optimized out&gt;) at execMain.c:364<br>
          #12 0x000055ceb93d9140 in ProcessQuery (plan=&lt;optimized
          out&gt;, <br>
              sourceText=0x55ceba065140 "update Location set enmea=$1,
          guid=$2, latDeci=$3, lonDeci=$4, modifiedByLoginName=$5,
          poiObjectId=$6, timestamp=$7, TRACK_ID=$8 where
          LOCATION_ID=$9", params=0x55ceba065268, queryEnv=0x0,
          dest=0x55ceb977fb60 &lt;donothingDR&gt;,
          completionTag=0x7fff3cbcbdd0 "") at pquery.c:161<br>
          #13 0x000055ceb93d9370 in PortalRunMulti
          (portal=portal@entry=0x55ceb9a31750,
          isTopLevel=isTopLevel@entry=true,
          setHoldSnapshot=setHoldSnapshot@entry=false, <br>
              dest=0x55ceb977fb60 &lt;donothingDR&gt;,
          dest@entry=0x55ceb99cd950, altdest=0x55ceb977fb60
          &lt;donothingDR&gt;, altdest@entry=0x55ceb99cd950, <br>
              completionTag=completionTag@entry=0x7fff3cbcbdd0 "") at
          pquery.c:1283<br>
          #14 0x000055ceb93d9e51 in PortalRun
          (portal=portal@entry=0x55ceb9a31750, count=count@entry=1,
          isTopLevel=isTopLevel@entry=true,
          run_once=run_once@entry=false, <br>
              dest=dest@entry=0x55ceb99cd950,
          altdest=altdest@entry=0x55ceb99cd950,
          completionTag=0x7fff3cbcbdd0 "") at pquery.c:796<br>
          #15 0x000055ceb93d7557 in exec_execute_message (max_rows=1,
          portal_name=0x55ceb99cd540 "") at postgres.c:2090<br>
          #16 PostgresMain (argc=&lt;optimized out&gt;,
          argv=argv@entry=0x55ceb99f5958, dbname=&lt;optimized out&gt;,
          username=&lt;optimized out&gt;) at postgres.c:4297<br>
          --Type &lt;RET&gt; for more, q to quit, c to continue without
          paging--<br>
          #17 0x000055ceb9363ce2 in BackendRun (port=0x55ceb99f1920,
          port=0x55ceb99f1920) at postmaster.c:4431<br>
          #18 BackendStartup (port=0x55ceb99f1920) at postmaster.c:4122<br>
          #19 ServerLoop () at postmaster.c:1704<br>
          #20 0x000055ceb9364c10 in PostmasterMain (argc=1,
          argv=0x55ceb99c7270) at postmaster.c:1377<br>
          #21 0x000055ceb90f3886 in main (argc=1, argv=0x55ceb99c7270)
          at main.c:228<br>
        </font></p>
      <p><font color="#000000">Thank you<br>
          Marcel</font><br>
      </p>
      <div class="moz-cite-prefix">Am 04.11.19 um 15:24 schrieb Tom
        Lane:<br>
      </div>
      <blockquote type="cite" cite="mid:[email protected]">
        <pre class="moz-quote-pre" wrap="">Alvaro Herrera <a class="moz-txt-link-rfc2396E" href="mailto:[email protected]" moz-do-not-send="true">&lt;[email protected]&gt;</a> writes:
</pre>
        <blockquote type="cite">
          <pre class="moz-quote-pre" wrap="">On 2019-Nov-04, Marcel Ruff wrote:
</pre>
          <blockquote type="cite">
            <pre class="moz-quote-pre" wrap="">Now I have upgraded to postgresql 12 and get a crash about once per day,
...
How can I track down the problem?
</pre>
          </blockquote>
        </blockquote>
        <blockquote type="cite">
          <pre class="moz-quote-pre" wrap="">The first you need to do is obtain a crash dump.  This page may help:
</pre>
        </blockquote>
        <blockquote type="cite">
          <pre class="moz-quote-pre" wrap=""><a class="moz-txt-link-freetext" href="https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD#Getting_a_trace_from_a_reproducibly_crashing_backend" moz-do-not-send="true">https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD#Getting_a_trace_from_a_reproducibly_crashing_backend</a>
</pre>
        </blockquote>
        <pre class="moz-quote-pre" wrap="">Also, as long as you're building from source anyway, you might pull down
v12 branch tip from our git repo, or use a nightly snapshot tarball [1]
to see if the problem's already fixed.

			regards, tom lane

[1] <a class="moz-txt-link-freetext" href="https://www.postgresql.org/ftp/snapshot/12/" moz-do-not-send="true">https://www.postgresql.org/ftp/snapshot/12/</a>
</pre>
      </blockquote>
      <div class="moz-signature">-- <br>
        Netwake<font color="#e66f24">Vision</font><br>
        Alte Owinger Straße 100<br>
        D-88662 Überlingen<br>
        Phone: +49 7551 309372<br>
        <a class="moz-txt-link-freetext"
          href="http://www.netwakevision.com" moz-do-not-send="true">http://www.netwakevision.com</a><br>
        <a class="moz-txt-link-freetext" href="http://www.royal-gps.com"
          moz-do-not-send="true">http://www.royal-gps.com</a></div>
    </blockquote>
    <div class="moz-signature">-- <br>
      Netwake<font color="#e66f24">Vision</font><br>
      Alte Owinger Straße 100<br>
      D-88662 Überlingen<br>
      Phone: +49 7551 309372<br>
      <a class="moz-txt-link-freetext" href="http://www.netwakevision.com">http://www.netwakevision.com</a><br>
      <a class="moz-txt-link-freetext" href="http://www.royal-gps.com">http://www.royal-gps.com</a></div>
  </body>
</html>

--------------BA51159822242DA14E190FE8--