Re: Best ways around memory Error on large st_union for overlay processing

--0000000000009fe637064877b28c
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thanks, Paul! Appreciate these thoughts - and considerations.  I'll keep
these ideas in mind as I proceed and see where I end up.  Definitely trying
to keep things simpler than not all things considered... we'll see!
Best,
Mike

On Thu, Jan 15, 2026 at 1:16=E2=80=AFPM Paul Ramsey <pramsey@cleverelephant=
.ca>
wrote:

>
>
> On Jan 15, 2026, at 9:46=E2=80=AFAM, Mike Treglia <[email protected]> wr=
ote:
>
> Is there an obvious or more optimal way to do that
> st_union(st_exteriorring(geom)) step for large datasets?
>
>
> Just spitballing=E2=80=A6
>
> Starting with an ST_Subdivide on all the rings, writing out the ring
> segments and a unique key into a staging table.
> Then do the st_union on a gridded basis. I think that should be safe?
> The part I worry about is doing the polygon building, unless you do the
> build with overlapping grid cells to select potential input ring segments=
,
> and then post-filter the polygon collection you get to remove any
> overlapping/duplicated polygons. My concern is that a very large input
> polygon relative to the grid size might fail to be built, if all its
> component pieces do not happen to fall into a single grid cell.
> At some point you end up building something of the scale and complexity o=
f
> the topology module, and maybe you would be able to get some good results
> starting there instead.
> P.
>

--0000000000009fe637064877b28c
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Thanks, Paul! Appreciate these thoughts - and conside=
rations.=C2=A0 I&#39;ll keep these ideas in mind as I proceed and see where=
 I end up.=C2=A0 Definitely trying to keep things simpler than not all thin=
gs considered... we&#39;ll see!</div><div>Best,</div><div>Mike</div></div><=
br><div class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=
=3D"gmail_attr">On Thu, Jan 15, 2026 at 1:16=E2=80=AFPM Paul Ramsey &lt;<a =
href=3D"mailto:[email protected]">[email protected]</a>&gt;=
 wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px =
0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><br=
 id=3D"m_-2293784447302591354lineBreakAtBeginningOfMessage"><div><br><block=
quote type=3D"cite"><div>On Jan 15, 2026, at 9:46=E2=80=AFAM, Mike Treglia =
&lt;<a href=3D"mailto:[email protected]" target=3D"_blank">mtreglia@gmail.=
com</a>&gt; wrote:</div><br><div><div style=3D"font-family:HelveticaNeue;fo=
nt-size:13px;font-style:normal;font-variant-caps:normal;font-weight:400;let=
ter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;whi=
te-space:normal;word-spacing:0px;text-decoration:none">Is there an obvious =
or more optimal way to do that st_union(st_exteriorring(geom)) step for lar=
ge datasets?=C2=A0</div><br></div></blockquote></div><br><div>Just spitball=
ing=E2=80=A6</div><div><br></div><div>Starting with an ST_Subdivide on all =
the rings, writing out the ring segments and a unique key into a staging ta=
ble.</div><div>Then do the st_union on a gridded basis. I think that should=
 be safe?</div><div>The part I worry about is doing the polygon building, u=
nless you do the build with overlapping grid cells to select potential inpu=
t ring segments, and then post-filter the polygon collection you get to rem=
ove any overlapping/duplicated polygons. My concern is that a very large in=
put polygon relative to the grid size might fail to be built, if all its co=
mponent pieces do not happen to fall into a single grid cell.</div><div>At =
some point you end up building something of the scale and complexity of the=
 topology module, and maybe you would be able to get some good results star=
ting there instead.</div><div>P.</div></div></blockquote></div>

--0000000000009fe637064877b28c--