Re: BUG #16205: background worker "logical replication worker" (PID 25218) was terminated by signal 11: Segmentation

Pendekar Dikala Senja <[email protected]> Wed, 15 Jan 2020 04:02:38 +0000
Newsgroups	gmane.comp.db.postgresql.bugs
Message-ID	<PS2PR02MB352770FEDFB21B14FA60F60083370@PS2PR02MB3527.apcprd02.prod.outlook.com>
--_006_PS2PR02MB352770FEDFB21B14FA60F60083370PS2PR02MB3527apcp_
Content-Type: multipart/alternative;
	boundary="_000_PS2PR02MB352770FEDFB21B14FA60F60083370PS2PR02MB3527apcp_"

--_000_PS2PR02MB352770FEDFB21B14FA60F60083370PS2PR02MB3527apcp_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable


Here are the logs and our configuration.
Actually we are facing problem when replicating partition table from 11.6 t=
o 12.1, but if there is no partition table the replication works so well.
As FYI, our server spec is 16 core and 64GB. And we are using this ubuntu

LSB Version: core-9.20160110ubuntu0.2-amd64:core-9.20160110ubuntu0.2-noarch=
:security-9.20160110ubuntu0.2-amd64:security-9.20160110ubuntu0.2-noarch
Distributor ID: Ubuntu
Description: Ubuntu 16.04.6 LTS
Release: 16.04
Codename: xenial

Can you help us to find out the solution and configuration?
Thanks for your help

________________________________
From: Michael Paquier
Sent: Tuesday, January 14, 2020 6:58 PM
To: [email protected]; [email protected]
Subject: Re: BUG #16205: background worker "logical replication worker" (PI=
D 25218) was terminated by signal 11: Segmentation

On Wed, Jan 15, 2020 at 02:00:48AM +0000, PG Bug reporting form wrote:
> I got error when replicate with native logical method from PostgreSQL 11.=
6
> to PostgreSQL 12.1, I will connecting table partition to table partition
> from Version 11.6 to Version 12.1, but after all tables already replicate=
d
> around more than 2 hours, server can't access and recovery mode. please h=
elp
> to advice, below log.

Could you provide a backtrace?  Looking at the recent commit history,
we had one fix which could be relevant to you after 12.1 was released:
https://www.postgresql.org/message-id//[email protected]=
g

And here is the commit:
https://git.postgresql.org/gitweb/?p=3Dpostgresql.git;a=3Dcommit;h=3Da2aa22=
4e

The problem may be different though, so a backtrace would be good to
check anyway.
--
Michael

--_000_PS2PR02MB352770FEDFB21B14FA60F60083370PS2PR02MB3527apcp_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
1">
<style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo=
ttom:0;} </style>
</head>
<body dir=3D"ltr">
<div style=3D"font-family: &quot;Times New Roman&quot;, Times, serif; font-=
size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style=3D"font-family: &quot;Times New Roman&quot;, Times, serif; font-=
size: 11pt; color: rgb(0, 0, 0);">
Here are the logs and our configuration.</div>
<div style=3D"font-family: &quot;Times New Roman&quot;, Times, serif; font-=
size: 11pt; color: rgb(0, 0, 0);">
Actually we are facing problem when replicating partition table from 11.6 t=
o 12.1, but if there is no partition table the replication works so well.
<br>
</div>
<div style=3D"font-family: &quot;Times New Roman&quot;, Times, serif; font-=
size: 11pt; color: rgb(0, 0, 0);">
As FYI, our server spec is 16 core and 64GB. And we are using this ubuntu</=
div>
<div style=3D"font-family: &quot;Times New Roman&quot;, Times, serif; font-=
size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style=3D"font-family: &quot;Times New Roman&quot;, Times, serif; font-=
size: 11pt; color: rgb(0, 0, 0);">
<span>LSB Version: core-9.20160110ubuntu0.2-amd64:core-9.20160110ubuntu0.2-=
noarch:security-9.20160110ubuntu0.2-amd64:security-9.20160110ubuntu0.2-noar=
ch<br>
</span>
<div>Distributor ID: Ubuntu<br>
</div>
<div>Description: Ubuntu 16.04.6 LTS<br>
</div>
<div>Release: 16.04<br>
</div>
<div>Codename: xenial<br>
</div>
</div>
<div style=3D"font-family: &quot;Times New Roman&quot;, Times, serif; font-=
size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style=3D"font-family: &quot;Times New Roman&quot;, Times, serif; font-=
size: 11pt; color: rgb(0, 0, 0);">
Can you help us to find out the solution and configuration?</div>
<div style=3D"font-family: &quot;Times New Roman&quot;, Times, serif; font-=
size: 11pt; color: rgb(0, 0, 0);">
Thanks for your help<br>
</div>
<div style=3D"font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12p=
t; color:rgb(0,0,0);">
<br>
<hr tabindex=3D"-1" style=3D"display:inline-block; width:98%;">
<b>From:</b> Michael Paquier<br>
<b>Sent:</b> Tuesday, January 14, 2020 6:58 PM<br>
<b>To:</b> [email protected]; [email protected]<br>
<b>Subject:</b> Re: BUG #16205: background worker &quot;logical replication=
 worker&quot; (PID 25218) was terminated by signal 11: Segmentation
<div><br>
</div>
</div>
<div class=3D"BodyFragment"><font size=3D"2"><span style=3D"font-size:11pt;=
">
<div class=3D"PlainText">On Wed, Jan 15, 2020 at 02:00:48AM &#43;0000, PG B=
ug reporting form wrote:<br>
&gt; I got error when replicate with native logical method from PostgreSQL =
11.6<br>
&gt; to PostgreSQL 12.1, I will connecting table partition to table partiti=
on<br>
&gt; from Version 11.6 to Version 12.1, but after all tables already replic=
ated<br>
&gt; around more than 2 hours, server can't access and recovery mode. pleas=
e help<br>
&gt; to advice, below log.<br>
<br>
Could you provide a backtrace?&nbsp; Looking at the recent commit history,<=
br>
we had one fix which could be relevant to you after 12.1 was released:<br>
<a href=3D"https://www.postgresql.org/message-id//16129-a0c0f48e71741e5f@po=
stgresql.org" target=3D"_blank" rel=3D"noopener noreferrer" data-auth=3D"No=
tApplicable">https://www.postgresql.org/message-id//16129-a0c0f48e71741e5f@=
postgresql.org</a><br>
<br>
And here is the commit:<br>
<a href=3D"https://git.postgresql.org/gitweb/?p=3Dpostgresql.git;a=3Dcommit=
;h=3Da2aa224e" target=3D"_blank" rel=3D"noopener noreferrer" data-auth=3D"N=
otApplicable">https://git.postgresql.org/gitweb/?p=3Dpostgresql.git;a=3Dcom=
mit;h=3Da2aa224e</a><br>
<br>
The problem may be different though, so a backtrace would be good to<br>
check anyway.<br>
--<br>
Michael<br>
</div>
</span></font></div>
</body>
</html>

--_000_PS2PR02MB352770FEDFB21B14FA60F60083370PS2PR02MB3527apcp_--

--_006_PS2PR02MB352770FEDFB21B14FA60F60083370PS2PR02MB3527apcp_
Content-Type: text/plain; name="postgres-config.txt"
Content-Description: postgres-config.txt
Content-Disposition: attachment; filename="postgres-config.txt"; size=1028;
	creation-date="Wed, 15 Jan 2020 03:58:05 GMT";
	modification-date="Wed, 15 Jan 2020 03:58:25 GMT"
Content-Transfer-Encoding: base64

ZGF0YV9kaXJlY3RvcnkgPSAnL2RhdGEvcG9zdGdyZXMvZGF0YS8nCmxpc3Rlbl9hZGRyZXNzZXMg
PSAnKicKcG9ydCA9IDU0MzIKbWF4X2Nvbm5lY3Rpb25zID0gMTAyNApzc2wgPSBvZmYKc2hhcmVk
X2J1ZmZlcnMgPSAxNkdCCndvcmtfbWVtID0gOE1CCm1haW50ZW5hbmNlX3dvcmtfbWVtID0gMUdC
CnNoYXJlZF9wcmVsb2FkX2xpYnJhcmllcyA9ICdwZ19zdGF0X3N0YXRlbWVudHMscGdfcGFydG1h
bl9iZ3cnCnBnX3N0YXRfc3RhdGVtZW50cy50cmFjayA9IGFsbAp3YWxfbGV2ZWwgPSBsb2dpY2Fs
CmNoZWNrcG9pbnRfdGltZW91dCA9IDE1bWluCm1heF93YWxfc2l6ZSA9IDFHQgptaW5fd2FsX3Np
emUgPSA4ME1CCmNoZWNrcG9pbnRfY29tcGxldGlvbl90YXJnZXQgPSAwLjcKYXJjaGl2ZV9tb2Rl
ID0gb24KYXJjaGl2ZV9jb21tYW5kID0gJy9iaW4vdHJ1ZScKbWF4X3dhbF9zZW5kZXJzID0gNjQK
d2FsX2tlZXBfc2VnbWVudHMgPSAxMDAKbWF4X3JlcGxpY2F0aW9uX3Nsb3RzID0gMjAKaG90X3N0
YW5kYnkgPSBvbgpob3Rfc3RhbmRieV9mZWVkYmFjayA9IG9uCmxvZ19kZXN0aW5hdGlvbiA9ICdz
dGRlcnInCmxvZ2dpbmdfY29sbGVjdG9yID0gb24KbG9nX2RpcmVjdG9yeSA9ICdwZ19sb2cnCmxv
Z19maWxlX21vZGUgPSAwNjAwCmxvZ19maWxlbmFtZSA9ICdwb3N0Z3Jlc3FsLSVZLSVtLSVkXyVI
JU0lUy5sb2cnCmxvZ190cnVuY2F0ZV9vbl9yb3RhdGlvbiA9IG9uCmxvZ19yb3RhdGlvbl9hZ2Ug
PSAxZApsb2dfcm90YXRpb25fc2l6ZSA9IDUwTUIKbG9nX21pbl9kdXJhdGlvbl9zdGF0ZW1lbnQg
PSAxMDAKbG9nX2NoZWNrcG9pbnRzID0gb24KbG9nX2xpbmVfcHJlZml4ID0gJyV0IFslcF0gJXEl
dUAlZCAlaCAnCmxvZ19sb2NrX3dhaXRzID0gb24KbG9nX3N0YXRlbWVudCA9ICdub25lJwpsb2df
dGltZXpvbmUgPSAnQXNpYS9KYWthcnRhJwp0aW1lem9uZSA9ICdBc2lhL0pha2FydGEnCnBnX3Bh
cnRtYW5fYmd3LmludGVydmFsID0gMzYwMApwZ19wYXJ0bWFuX2Jndy5kYm5hbWUgPSAnZGJvcmRl
cic=

--_006_PS2PR02MB352770FEDFB21B14FA60F60083370PS2PR02MB3527apcp_
Content-Type: text/plain; name="postgresql-log.txt"
Content-Description: postgresql-log.txt
Content-Disposition: attachment; filename="postgresql-log.txt"; size=2342;
	creation-date="Wed, 15 Jan 2020 03:58:05 GMT";
	modification-date="Wed, 15 Jan 2020 03:58:25 GMT"
Content-Transfer-Encoding: base64

MjAyMC0wMS0xNSAwOTozNDozMSBXSUIgWzE0NTE1XSBMT0c6ICBkYXRhYmFzZSBzeXN0ZW0gaXMg
cmVhZHkgdG8gYWNjZXB0IGNvbm5lY3Rpb25zCjIwMjAtMDEtMTUgMDk6MzQ6MzEgV0lCIFsxNDky
OV0gTE9HOiAgcGdfcGFydG1hbiBtYXN0ZXIgYmFja2dyb3VuZCB3b3JrZXIgbWFzdGVyIHByb2Nl
c3MgaW5pdGlhbGl6ZWQgd2l0aCByb2xlIHBvc3RncmVzCjIwMjAtMDEtMTUgMDk6MzQ6MzEgV0lC
IFsxNDkzMV0gTE9HOiAgbG9naWNhbCByZXBsaWNhdGlvbiBhcHBseSB3b3JrZXIgZm9yIHN1YnNj
cmlwdGlvbiAidGtwb3JkZXJzdWJzIiBoYXMgc3RhcnRlZAoyMDIwLTAxLTE1IDA5OjM0OjMxIFdJ
QiBbMTQ5MzNdIExPRzogIGxvZ2ljYWwgcmVwbGljYXRpb24gdGFibGUgc3luY2hyb25pemF0aW9u
IHdvcmtlciBmb3Igc3Vic2NyaXB0aW9uICJ0a3BvcmRlcnN1YnMiLCB0YWJsZSAid3Nfb3JkZXJf
cDIwMTdfMTIiIGhhcyBzdGFydGVkCjIwMjAtMDEtMTUgMDk6MzQ6MzEgV0lCIFsxNDkzNF0gTE9H
OiAgbG9naWNhbCByZXBsaWNhdGlvbiB0YWJsZSBzeW5jaHJvbml6YXRpb24gd29ya2VyIGZvciBz
dWJzY3JpcHRpb24gInRrcG9yZGVyc3VicyIsIHRhYmxlICJ3c19vcmRlcl9wMjAxN18xMSIgaGFz
IHN0YXJ0ZWQKMjAyMC0wMS0xNSAwOTozNDozMSBXSUIgWzE0NTE1XSBMT0c6ICBiYWNrZ3JvdW5k
IHdvcmtlciAibG9naWNhbCByZXBsaWNhdGlvbiB3b3JrZXIiIChQSUQgMTQ5MzEpIHdhcyB0ZXJt
aW5hdGVkIGJ5IHNpZ25hbCAxMTogU2VnbWVudGF0aW9uIGZhdWx0CjIwMjAtMDEtMTUgMDk6MzQ6
MzEgV0lCIFsxNDUxNV0gTE9HOiAgdGVybWluYXRpbmcgYW55IG90aGVyIGFjdGl2ZSBzZXJ2ZXIg
cHJvY2Vzc2VzCjIwMjAtMDEtMTUgMDk6MzQ6MzEgV0lCIFsxNDkyN10gV0FSTklORzogIHRlcm1p
bmF0aW5nIGNvbm5lY3Rpb24gYmVjYXVzZSBvZiBjcmFzaCBvZiBhbm90aGVyIHNlcnZlciBwcm9j
ZXNzCjIwMjAtMDEtMTUgMDk6MzQ6MzEgV0lCIFsxNDkyN10gREVUQUlMOiAgVGhlIHBvc3RtYXN0
ZXIgaGFzIGNvbW1hbmRlZCB0aGlzIHNlcnZlciBwcm9jZXNzIHRvIHJvbGwgYmFjayB0aGUgY3Vy
cmVudCB0cmFuc2FjdGlvbiBhbmQgZXhpdCwgYmVjYXVzZSBhbm90aGVyIHNlcnZlciBwcm9jZXNz
IGV4aXRlZCBhYm5vcm1hbGx5IGFuZCBwb3NzaWJseSBjb3JydXB0ZWQgc2hhcmVkIG1lbW9yeS4K
MjAyMC0wMS0xNSAwOTozNDozMSBXSUIgWzE0OTI3XSBISU5UOiAgSW4gYSBtb21lbnQgeW91IHNo
b3VsZCBiZSBhYmxlIHRvIHJlY29ubmVjdCB0byB0aGUgZGF0YWJhc2UgYW5kIHJlcGVhdCB5b3Vy
IGNvbW1hbmQuCjIwMjAtMDEtMTUgMDk6MzQ6MzEgV0lCIFsxNDUxNV0gTE9HOiAgYWxsIHNlcnZl
ciBwcm9jZXNzZXMgdGVybWluYXRlZDsgcmVpbml0aWFsaXppbmcKMjAyMC0wMS0xNSAwOTozNDoz
MSBXSUIgWzE0NTE1XSBMT0c6ICByZWNlaXZlZCBmYXN0IHNodXRkb3duIHJlcXVlc3QKMjAyMC0w
MS0xNSAwOTozNDozMSBXSUIgWzE0OTQwXSBMT0c6ICBkYXRhYmFzZSBzeXN0ZW0gd2FzIGludGVy
cnVwdGVkOyBsYXN0IGtub3duIHVwIGF0IDIwMjAtMDEtMTUgMDk6MzQ6MzEgV0lCCjIwMjAtMDEt
MTUgMDk6MzQ6MzEgV0lCIFsxNDk0MV0gcmVwbGljYXRpb25fcGdAZGJvcmRlciAxNzIuMzEuMDAw
LjAwMCBGQVRBTDogIHRoZSBkYXRhYmFzZSBzeXN0ZW0gaXMgc2h1dHRpbmcgZG93bgoyMDIwLTAx
LTE1IDA5OjM0OjMxIFdJQiBbMTQ5NDJdIHJlcGxpY2F0aW9uX3BnQGRib3JkZXIgMTcyLjMxLjAw
MC4wMDAgRkFUQUw6ICB0aGUgZGF0YWJhc2Ugc3lzdGVtIGlzIHNodXR0aW5nIGRvd24KMjAyMC0w
MS0xNSAwOTozNDozMSBXSUIgWzE0OTQzXSByZXBsaWNhdGlvbl9wZ0BkYm9yZGVyIDE3Mi4zMS4w
MDAuMDAwIEZBVEFMOiAgdGhlIGRhdGFiYXNlIHN5c3RlbSBpcyBzaHV0dGluZyBkb3duCjIwMjAt
MDEtMTUgMDk6MzQ6MzIgV0lCIFsxNDk0MF0gTE9HOiAgcmVjb3ZlcmVkIHJlcGxpY2F0aW9uIHN0
YXRlIG9mIG5vZGUgMSB0byAyRUEvQjcwQjhEQzgKMjAyMC0wMS0xNSAwOTozNDozMiBXSUIgWzE0
OTQwXSBMT0c6ICBkYXRhYmFzZSBzeXN0ZW0gd2FzIG5vdCBwcm9wZXJseSBzaHV0IGRvd247IGF1
dG9tYXRpYyByZWNvdmVyeSBpbiBwcm9ncmVzcwoyMDIwLTAxLTE1IDA5OjM0OjMyIFdJQiBbMTQ5
NDBdIExPRzogIHJlZG8gc3RhcnRzIGF0IDEzMi9FMDJFOTdFOAoyMDIwLTAxLTE1IDA5OjM0OjMy
IFdJQiBbMTQ1MTVdIExPRzogIGFibm9ybWFsIGRhdGFiYXNlIHN5c3RlbSBzaHV0ZG93bgoyMDIw
LTAxLTE1IDA5OjM0OjMyIFdJQiBbMTQ1MTVdIExPRzogIGRhdGFiYXNlIHN5c3RlbSBpcyBzaHV0
IGRvd24=

--_006_PS2PR02MB352770FEDFB21B14FA60F60083370PS2PR02MB3527apcp_
Content-Type: text/plain; name="syslog.txt"
Content-Description: syslog.txt
Content-Disposition: attachment; filename="syslog.txt"; size=879;
	creation-date="Wed, 15 Jan 2020 03:58:06 GMT";
	modification-date="Wed, 15 Jan 2020 03:58:25 GMT"
Content-Transfer-Encoding: base64

SmFuIDE1IDEwOjM0OjE1IGxvY2FsaG9zdCBrZXJuZWw6IFsgMTg0Ni42MTA4OTVdIHBvc3RncmVz
WzE0NTI2XTogc2VnZmF1bHQgYXQgNTYwODVmNzk2MTI4IGlwIDAwMDA3ZjI4NjBlMzQxNjggc3Ag
MDAwMDdmZmU1OWUxNGNhOCBlcnJvciA0IGluIGxpYmMtMi4yMy5zb1s3ZjI4NjBjZTYwMDArMWMw
MDAwXQpKYW4gMTUgMTA6MzQ6MTUgbG9jYWxob3N0IGtlcm5lbDogWyAxODQ3LjE3MTI3Ml0gcG9z
dGdyZXNbMTQ1MzhdOiBzZWdmYXVsdCBhdCA1NjA4NWY3OTZjNDggaXAgMDAwMDdmMjg2MGUzNDE2
OCBzcCAwMDAwN2ZmZTU5ZTE0Y2E4IGVycm9yIDQgaW4gbGliYy0yLjIzLnNvWzdmMjg2MGNlNjAw
MCsxYzAwMDBdCkphbiAxNSAxMDozNDoxNiBsb2NhbGhvc3Qga2VybmVsOiBbIDE4NDcuNzI1MDc5
XSBwb3N0Z3Jlc1sxNDU1NF06IHNlZ2ZhdWx0IGF0IDU2MDg1Zjc5NmM0OCBpcCAwMDAwN2YyODYw
ZTM0MTY4IHNwIDAwMDA3ZmZlNTllMTRjYTggZXJyb3IgNCBpbiBsaWJjLTIuMjMuc29bN2YyODYw
Y2U2MDAwKzFjMDAwMF0KSmFuIDE1IDEwOjM0OjE2IGxvY2FsaG9zdCBrZXJuZWw6IFsgMTg0OC4y
ODE3MDZdIHBvc3RncmVzWzE0NTY5XTogc2VnZmF1bHQgYXQgNTYwODVmNzljZGM4IGlwIDAwMDA3
ZjI4NjBlMzQxNjggc3AgMDAwMDdmZmU1OWUxNGNhOCBlcnJvciA0IGluIGxpYmMtMi4yMy5zb1s3
ZjI4NjBjZTYwMDArMWMwMDAwXQpKYW4gMTUgMTA6MzQ6MTcgbG9jYWxob3N0IGtlcm5lbDogWyAx
ODQ4Ljg4OTQ2NV0gcG9zdGdyZXNbMTQ1ODZdOiBzZWdmYXVsdCBhdCA1NjA4NWY3OWNkYzggaXAg
MDAwMDdmMjg2MGUzNDE2OCBzcCAwMDAwN2ZmZTU5ZTE0Y2E4IGVycm9yIDQgaW4gbGliYy0yLjIz
LnNvWzdmMjg2MGNlNjAwMCsxYzAwMDBd

--_006_PS2PR02MB352770FEDFB21B14FA60F60083370PS2PR02MB3527apcp_--