Re: Best ways around memory Error on large st_union for overlay processing

--000000000000e0db88064884ae8c
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

It's hard!
I was just thinking that a light snap using the ST_ReducePrecision might
also be good to avoid the generation of some super-tiny slivers, but
whether that's done during preprocessing of rings or using
ST_CoverageSimplify after polygons are re-formed is an unknown. I look
forward to your blog post!
P

On Thu, Jan 15, 2026 at 5:59=E2=80=AFPM Mike Treglia <[email protected]> w=
rote:

> Thanks, Paul! Appreciate these thoughts - and considerations.  I'll keep
> these ideas in mind as I proceed and see where I end up.  Definitely tryi=
ng
> to keep things simpler than not all things considered... we'll see!
> Best,
> Mike
>
> On Thu, Jan 15, 2026 at 1:16=E2=80=AFPM Paul Ramsey <pramsey@cleverelepha=
nt.ca>
> wrote:
>
>>
>>
>> On Jan 15, 2026, at 9:46=E2=80=AFAM, Mike Treglia <[email protected]> w=
rote:
>>
>> Is there an obvious or more optimal way to do that
>> st_union(st_exteriorring(geom)) step for large datasets?
>>
>>
>> Just spitballing=E2=80=A6
>>
>> Starting with an ST_Subdivide on all the rings, writing out the ring
>> segments and a unique key into a staging table.
>> Then do the st_union on a gridded basis. I think that should be safe?
>> The part I worry about is doing the polygon building, unless you do the
>> build with overlapping grid cells to select potential input ring segment=
s,
>> and then post-filter the polygon collection you get to remove any
>> overlapping/duplicated polygons. My concern is that a very large input
>> polygon relative to the grid size might fail to be built, if all its
>> component pieces do not happen to fall into a single grid cell.
>> At some point you end up building something of the scale and complexity
>> of the topology module, and maybe you would be able to get some good
>> results starting there instead.
>> P.
>>
>

--000000000000e0db88064884ae8c
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">It&#39;s hard!<div>I was just thinking that a light snap u=
sing the ST_ReducePrecision might also be good to avoid the generation of s=
ome super-tiny slivers, but whether that&#39;s done during preprocessing of=
 rings or using ST_CoverageSimplify after polygons are re-formed is an unkn=
own. I look forward to your blog post!</div><div>P</div></div><br><div clas=
s=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_att=
r">On Thu, Jan 15, 2026 at 5:59=E2=80=AFPM Mike Treglia &lt;<a href=3D"mail=
to:[email protected]">[email protected]</a>&gt; wrote:<br></div><blockquo=
te class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-widt=
h:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-le=
ft:1ex"><div dir=3D"ltr"><div>Thanks, Paul! Appreciate these thoughts - and=
 considerations.=C2=A0 I&#39;ll keep these ideas in mind as I proceed and s=
ee where I end up.=C2=A0 Definitely trying to keep things simpler than not =
all things considered... we&#39;ll see!</div><div>Best,</div><div>Mike</div=
></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr"=
>On Thu, Jan 15, 2026 at 1:16=E2=80=AFPM Paul Ramsey &lt;<a href=3D"mailto:=
[email protected]" target=3D"_blank">[email protected]</a>&=
gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-colo=
r:rgb(204,204,204);padding-left:1ex"><div><br id=3D"m_8669330319712955592m_=
-2293784447302591354lineBreakAtBeginningOfMessage"><div><br><blockquote typ=
e=3D"cite"><div>On Jan 15, 2026, at 9:46=E2=80=AFAM, Mike Treglia &lt;<a hr=
ef=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>&g=
t; wrote:</div><br><div><div style=3D"font-family:HelveticaNeue;font-size:1=
3px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spaci=
ng:normal;text-align:start;text-indent:0px;text-transform:none;white-space:=
normal;word-spacing:0px;text-decoration:none">Is there an obvious or more o=
ptimal way to do that st_union(st_exteriorring(geom)) step for large datase=
ts?=C2=A0</div><br></div></blockquote></div><br><div>Just spitballing=E2=80=
=A6</div><div><br></div><div>Starting with an ST_Subdivide on all the rings=
, writing out the ring segments and a unique key into a staging table.</div=
><div>Then do the st_union on a gridded basis. I think that should be safe?=
</div><div>The part I worry about is doing the polygon building, unless you=
 do the build with overlapping grid cells to select potential input ring se=
gments, and then post-filter the polygon collection you get to remove any o=
verlapping/duplicated polygons. My concern is that a very large input polyg=
on relative to the grid size might fail to be built, if all its component p=
ieces do not happen to fall into a single grid cell.</div><div>At some poin=
t you end up building something of the scale and complexity of the topology=
 module, and maybe you would be able to get some good results starting ther=
e instead.</div><div>P.</div></div></blockquote></div>
</blockquote></div>

--000000000000e0db88064884ae8c--