Re: [Proposal] Global temporary tables
"曾文旌(义从)" <[email protected]> Tue, 14 Jan 2020 22:16:12 +0800
| Newsgroups | gmane.comp.db.postgresql.devel.general |
|---|---|
| Message-ID | <[email protected]> |
--Apple-Mail=_B5C08BE4-0B2B-4389-AF37-0E62F2268581 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > 2020=E5=B9=B41=E6=9C=8814=E6=97=A5 =E4=B8=8B=E5=8D=889:20=EF=BC=8CPavel = Stehule <[email protected]> =E5=86=99=E9=81=93=EF=BC=9A >=20 >=20 >=20 > =C3=BAt 14. 1. 2020 v 14:09 odes=C3=ADlatel =E6=9B=BE=E6=96=87=E6=97=8C(= =E4=B9=89=E4=BB=8E) <[email protected] = <mailto:[email protected]>> napsal: > Thank you for review my patch. >=20 >=20 >> 2020=E5=B9=B41=E6=9C=8812=E6=97=A5 =E4=B8=8A=E5=8D=884:27=EF=BC=8CPavel= Stehule <[email protected] <mailto:[email protected]>> = =E5=86=99=E9=81=93=EF=BC=9A >>=20 >> Hi >>=20 >> so 11. 1. 2020 v 15:00 odes=C3=ADlatel =E6=9B=BE=E6=96=87=E6=97=8C(=E4=B9= =89=E4=BB=8E) <[email protected] = <mailto:[email protected]>> napsal: >> Hi all >>=20 >> This is the latest patch >>=20 >> The updates are as follows: >> 1. Support global temp Inherit table global temp partition table >> 2. Support serial column in GTT >> 3. Provide views pg_gtt_relstats pg_gtt_stats for GTT=E2=80=99s = statistics >> 4. Provide view pg_gtt_attached_pids to manage GTT >> 5. Provide function pg_list_gtt_relfrozenxids() to manage GTT >> 6. Alter GTT or rename GTT is allowed under some conditions >>=20 >>=20 >> Please give me feedback. >>=20 >> I tested the functionality >>=20 >> 1. i think so "ON COMMIT PRESERVE ROWS" should be default mode (like = local temp tables). > makes sense, I will fix it. >=20 >>=20 >> I tested some simple scripts=20 >>=20 >> test01.sql >>=20 >> CREATE TEMP TABLE foo(a int, b int); >> INSERT INTO foo SELECT random()*100, random()*1000 FROM = generate_series(1,1000); >> ANALYZE foo; >> SELECT sum(a), sum(b) FROM foo; >> DROP TABLE foo; -- simulate disconnect >>=20 >>=20 >> after 100 sec, the table pg_attribute has 3.2MB >> and 64 tps, 6446 transaction >>=20 >> test02.sql >>=20 >> INSERT INTO foo SELECT random()*100, random()*1000 FROM = generate_series(1,1000); >> ANALYZE foo; >> SELECT sum(a), sum(b) FROM foo; >> DELETE FROM foo; -- simulate disconnect >>=20 >>=20 >> after 100 sec, 1688 tps, 168830 transactions >>=20 >> So performance is absolutely different as we expected. >>=20 >> =46rom my perspective, this functionality is great. > Yes, frequent ddl causes catalog bloat, GTT avoids this problem. >=20 >>=20 >> Todo: >>=20 >> pg_table_size function doesn't work > Do you mean that function pg_table_size() need get the storage space = used by the one GTT in the entire db(include all session) . >=20 > It's question how much GTT tables should be similar to classic tables. = But the reporting in psql should to work \dt+, \l+, \di+ Get it, I will fix it. >=20 >=20 >=20 >>=20 >> Regards >>=20 >> Pavel >>=20 >>=20 >> Wenjing >>=20 >>=20 >>=20 >>=20 >>=20 >>> 2020=E5=B9=B41=E6=9C=886=E6=97=A5 =E4=B8=8A=E5=8D=884:06=EF=BC=8CTomas= Vondra <[email protected] = <mailto:[email protected]>> =E5=86=99=E9=81=93=EF=BC=9A >>>=20 >>> Hi, >>>=20 >>> I think we need to do something with having two patches aiming to = add >>> global temporary tables: >>>=20 >>> [1] https://commitfest.postgresql.org/26/2349/ = <https://commitfest.postgresql.org/26/2349/> >>>=20 >>> [2] https://commitfest.postgresql.org/26/2233/ = <https://commitfest.postgresql.org/26/2233/> >>>=20 >>> As a reviewer I have no idea which of the threads to look at - = certainly >>> not without reading both threads, which I doubt anyone will really = do. >>> The reviews and discussions are somewhat intermixed between those = two >>> threads, which makes it even more confusing. >>>=20 >>> I think we should agree on a minimal patch combining the = necessary/good >>> bits from the various patches, and terminate one of the threads = (i.e. >>> mark it as rejected or RWF). And we need to do that now, otherwise >>> there's about 0% chance of getting this into v13. >>>=20 >>> In general, I agree with the sentiment Rober expressed in [1] - the >>> patch needs to be as small as possible, not adding "nice to have" >>> features (like support for parallel queries - I very much doubt just >>> using shared instead of local buffers is enough to make it work.) >>>=20 >>> regards >>>=20 >>> --=20 >>> Tomas Vondra http://www.2ndQuadrant.com = <http://www.2ndquadrant.com/> >>> PostgreSQL Development, 24x7 Support, Remote DBA, Training & = Services >>=20 >=20 --Apple-Mail=_B5C08BE4-0B2B-4389-AF37-0E62F2268581 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br = class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">2020=E5=B9=B41=E6=9C=8814=E6=97=A5 =E4=B8=8B=E5=8D=889:20=EF=BC= =8CPavel Stehule <<a href=3D"mailto:[email protected]" = class=3D"">[email protected]</a>> =E5=86=99=E9=81=93=EF=BC=9A</di= v><br class=3D"Apple-interchange-newline"><div class=3D""><div dir=3D"ltr"= class=3D""><div dir=3D"ltr" class=3D""><br class=3D""></div><br = class=3D""><div class=3D"gmail_quote"><div dir=3D"ltr" = class=3D"gmail_attr">=C3=BAt 14. 1. 2020 v 14:09 odes=C3=ADlatel = =E6=9B=BE=E6=96=87=E6=97=8C(=E4=B9=89=E4=BB=8E) <<a = href=3D"mailto:[email protected]" = class=3D"">[email protected]</a>> napsal:<br = class=3D""></div><blockquote class=3D"gmail_quote" style=3D"margin:0px = 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"><div style=3D"overflow-wrap: = break-word;" class=3D"">Thank you for review my patch.<div class=3D""><br = class=3D""><div class=3D""><br class=3D""><blockquote type=3D"cite" = class=3D""><div class=3D"">2020=E5=B9=B41=E6=9C=8812=E6=97=A5 = =E4=B8=8A=E5=8D=884:27=EF=BC=8CPavel Stehule <<a = href=3D"mailto:[email protected]" target=3D"_blank" = class=3D"">[email protected]</a>> =E5=86=99=E9=81=93=EF=BC=9A</di= v><br class=3D""><div class=3D""><div dir=3D"ltr" class=3D""><div = class=3D"">Hi<br class=3D""></div><br class=3D""><div = class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">so 11. 1. = 2020 v 15:00 odes=C3=ADlatel =E6=9B=BE=E6=96=87=E6=97=8C(=E4=B9=89=E4= =BB=8E) <<a href=3D"mailto:[email protected]" = target=3D"_blank" class=3D"">[email protected]</a>> = napsal:<br class=3D""></div><blockquote class=3D"gmail_quote" = style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"><div class=3D"">Hi all<div = class=3D""><br class=3D""></div><div class=3D"">This is the latest = patch</div><div class=3D""><br class=3D""></div><div class=3D"">The = updates are as follows:</div><div class=3D"">1. Support global temp = Inherit table <span class=3D"">global temp partition = table</span></div><div class=3D""><span class=3D"">2. = Support </span><font class=3D""><span class=3D"">serial column in = GTT</span></font></div><div class=3D""><span = class=3D"">3. </span><font class=3D""><span class=3D"">Provide = views pg_gtt_relstats pg_gtt_stats for GTT=E2=80=99s = statistics</span></font></div><div class=3D""><span = class=3D"">4. </span><span = class=3D"">Provide</span> view <span = class=3D"">pg_gtt_attached_pids to </span><span = class=3D"">manage GTT</span></div><div class=3D""><span = class=3D"">5. </span><span class=3D"">Provide = function </span><font class=3D"">pg_list_gtt_relfrozenxids() to = manage GTT</font></div><div class=3D""><font class=3D"">6. <span= class=3D"">Alter GTT or rename GTT is allowed under some = conditions</span></font></div><div class=3D""><br class=3D""></div><div = class=3D""><br class=3D""></div><div class=3D"">Please give me = feedback.</div></div></blockquote><div class=3D""><br = class=3D""></div><div class=3D"">I tested the functionality</div><div = class=3D""><br class=3D""></div><div class=3D"">1. i think so "ON COMMIT = PRESERVE ROWS" should be default mode (like local temp = tables).</div></div></div></div></blockquote>makes sense, I will fix = it.</div><div class=3D""><br class=3D""><blockquote type=3D"cite" = class=3D""><div class=3D""><div dir=3D"ltr" class=3D""><div = class=3D"gmail_quote"><div class=3D""><br class=3D""></div><div = class=3D"">I tested some simple scripts <br class=3D""></div><div = class=3D""><br class=3D""></div><div class=3D"">test01.sql</div><div = class=3D""><br class=3D""></div><div class=3D"">CREATE TEMP TABLE foo(a = int, b int);<br class=3D"">INSERT INTO foo SELECT random()*100, = random()*1000 FROM generate_series(1,1000);<br class=3D"">ANALYZE = foo;<br class=3D"">SELECT sum(a), sum(b) FROM foo;<br class=3D"">DROP = TABLE foo; -- simulate disconnect<br class=3D""></div><div class=3D""><br = class=3D""></div><div class=3D""><br class=3D""></div><div = class=3D"">after 100 sec, the table pg_attribute has 3.2MB</div><div = class=3D"">and 64 tps, 6446 transaction<br class=3D""></div><div = class=3D""><br class=3D""></div><div class=3D"">test02.sql</div><div = class=3D""><br class=3D""></div><div class=3D"">INSERT INTO foo SELECT = random()*100, random()*1000 FROM generate_series(1,1000);<br = class=3D"">ANALYZE foo;<br class=3D"">SELECT sum(a), sum(b) FROM foo;<br = class=3D"">DELETE FROM foo; -- simulate disconnect<br = class=3D""></div><div class=3D""><br class=3D""></div><div class=3D""><br = class=3D""></div><div class=3D"">after 100 sec, 1688 tps, 168830 = transactions</div><div class=3D""><br class=3D""></div><div class=3D"">So = performance is absolutely different as we expected.</div><div = class=3D""><br class=3D""></div><div class=3D"">=46rom my perspective, = this functionality is great.</div></div></div></div></blockquote>Yes, = frequent ddl causes <span style=3D"" class=3D"">catalog = bloat, </span><font class=3D""><span class=3D"">GTT avoids this = problem.</span></font></div><div class=3D""><font class=3D""><span = class=3D""><br class=3D""></span></font><blockquote type=3D"cite" = class=3D""><div class=3D""><div dir=3D"ltr" class=3D""><div = class=3D"gmail_quote"><div class=3D""><br class=3D""></div><div = class=3D"">Todo:</div><div class=3D""><br class=3D""></div><div = class=3D"">pg_table_size function doesn't = work</div></div></div></div></blockquote>Do you mean that = function <span style=3D"" class=3D"">pg_table_size() need = get</span> the storage space used by the one GTT in the entire = db(include all session) .</div></div></div></blockquote><div = class=3D""><br class=3D""></div><div class=3D"">It's question how much = GTT tables should be similar to classic tables. But the reporting in = psql should to work \dt+, \l+, \di+<br = class=3D""></div></div></div></div></blockquote><span = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=3D"">Get = it, I will fix it.</span><br class=3D""><blockquote type=3D"cite" = class=3D""><div class=3D""><div dir=3D"ltr" class=3D""><div = class=3D"gmail_quote"><div class=3D""><br class=3D""></div><div = class=3D""> <br class=3D""></div><blockquote class=3D"gmail_quote" = style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"><div style=3D"overflow-wrap: = break-word;" class=3D""><div class=3D""><div class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = dir=3D"ltr" class=3D""><div class=3D"gmail_quote"><div class=3D""><br = class=3D""></div><div class=3D"">Regards</div><div class=3D""><br = class=3D""></div><div class=3D"">Pavel<br class=3D""></div><div = class=3D""><br class=3D""></div><blockquote class=3D"gmail_quote" = style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"><div class=3D""><div class=3D""><br = class=3D""></div><div class=3D"">Wenjing</div><div class=3D""><br = class=3D""></div><div class=3D""><br class=3D""></div><div class=3D""><br = class=3D""></div><div class=3D""></div></div><div class=3D""><div = class=3D""><br class=3D""><div class=3D""><br class=3D""><blockquote = type=3D"cite" class=3D""><div class=3D"">2020=E5=B9=B41=E6=9C=886=E6=97=A5= =E4=B8=8A=E5=8D=884:06=EF=BC=8CTomas Vondra <<a = href=3D"mailto:[email protected]" target=3D"_blank" = class=3D"">[email protected]</a>> =E5=86=99=E9=81=93=EF=BC=9A= </div><br class=3D""><div class=3D""><div class=3D"">Hi,<br class=3D""><br= class=3D"">I think we need to do something with having two patches = aiming to add<br class=3D"">global temporary tables:<br class=3D""><br = class=3D"">[1] <a href=3D"https://commitfest.postgresql.org/26/2349/" = target=3D"_blank" = class=3D"">https://commitfest.postgresql.org/26/2349/</a><br = class=3D""><br class=3D"">[2] <a = href=3D"https://commitfest.postgresql.org/26/2233/" target=3D"_blank" = class=3D"">https://commitfest.postgresql.org/26/2233/</a><br = class=3D""><br class=3D"">As a reviewer I have no idea which of the = threads to look at - certainly<br class=3D"">not without reading both = threads, which I doubt anyone will really do.<br class=3D"">The reviews = and discussions are somewhat intermixed between those two<br = class=3D"">threads, which makes it even more confusing.<br class=3D""><br = class=3D"">I think we should agree on a minimal patch combining the = necessary/good<br class=3D"">bits from the various patches, and = terminate one of the threads (i.e.<br class=3D"">mark it as rejected or = RWF). And we need to do that now, otherwise<br class=3D"">there's about = 0% chance of getting this into v13.<br class=3D""><br class=3D"">In = general, I agree with the sentiment Rober expressed in [1] - the<br = class=3D"">patch needs to be as small as possible, not adding "nice to = have"<br class=3D"">features (like support for parallel queries - I very = much doubt just<br class=3D"">using shared instead of local buffers is = enough to make it work.)<br class=3D""><br class=3D"">regards<br = class=3D""><br class=3D"">-- <br class=3D"">Tomas Vondra = &n= bsp; <a href=3D"http://www.2ndquadrant.com/" = target=3D"_blank" class=3D"">http://www.2ndQuadrant.com</a><br = class=3D"">PostgreSQL Development, 24x7 Support, Remote DBA, Training = & Services</div></div></blockquote></div><br = class=3D""></div></div></blockquote></div></div> </div></blockquote></div><br = class=3D""></div></div></blockquote></div></div> </div></blockquote></div><br class=3D""></body></html>= --Apple-Mail=_B5C08BE4-0B2B-4389-AF37-0E62F2268581--