Re: MapCache clusters

--===============3547324356301822546==
Content-Type: multipart/alternative; boundary="000000000000ac303b0597b550fc"

--000000000000ac303b0597b550fc
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I don't think multiple instances of MapCache will help for seeding, in
general most of the seeding time depends on the zoom level, with each
higher zoom level taking 4 x the time as the proceeding zoom level (e.g.
zoom 20 take 4 x the time to seed as zoom 19)  .  This can vary depending
on your source service as there may be certain map scales / zoom levels
that take longer per tile to render due to expression, labels, # of
features etc... If you were to seed two levels for each mapcache instance
you still would be waiting for the higher zoom levels to complete.   Using
multiple load balanced mapserver nodes across multiple VM may improve
things a bit however if all the VMs are hosted on the same physical server
this may be slower overall.  For seeding I have heard you can use 2x the
number of cores so you could set up a single mapcache server using -n 16
pointing at the load balanced source.

Regards



On Mon, 18 Nov 2019 at 10:47, Sebastiano Laini <
[email protected]> wrote:

> Hi All,
>
>
>
> I was wondering if is possible to dockerise my server and then use
> multiple cluster to seed the cache? I know I can use multiple processes
> with the =E2=80=93n and =E2=80=93p flag though I=E2=80=99m wondering ther=
e is a limit on how much
> you can really increase those number.
>
>
>
> E.g. I dockerise the server and I spawn a 8core VM with 16GB ram, I can
> use sure =E2=80=93p 8 or maybe 10 but still I think that will require lik=
e 60 days
> to seed the whole UK with our data, can I spawn another machine and speed
> up the process? For example another 8core VM.
>
>
>
> We have 4 caches with different style that are updated twice a year so if
> it takes me 60 days for each by the time I finish it=E2=80=99s already ti=
me to
> start again.
>
>
>
> I was wondering maybe I can run multiple VM and seed one or maybe 2 level=
s
> on each VM or if I spawn a new VM the original process will take advantag=
e
> of the new VM?
>
> In a VM I=E2=80=99m going to have MapServer and MapCache and I will use a
> subdomain under a load balancer just to seed while the domain will be use
> just for the seeded cache and in the same way will be under a load balanc=
er.
>
>
>
> Regards,
>
>
>
> Sebastiano Laini
>
> Web Developer
>
> Buchanan Computing
> _______________________________________________
> mapserver-users mailing list
> [email protected]
> https://lists.osgeo.org/mailman/listinfo/mapserver-users

--000000000000ac303b0597b550fc
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I don&#39;t think multiple instances of MapCache will help=
 for seeding, in general most of the seeding time depends on the zoom level=
, with each higher zoom level taking 4 x the time as the proceeding zoom le=
vel (e.g. zoom 20 take 4 x the time to seed as zoom 19)=C2=A0 .=C2=A0 This =
can vary depending on your source service as there may be certain map scale=
s / zoom levels that take longer per tile to render due to expression, labe=
ls, # of features etc... If you were to seed two levels for each mapcache i=
nstance you still would be waiting for the higher zoom levels to complete.=
=C2=A0 =C2=A0Using multiple load balanced mapserver nodes across multiple V=
M may improve things a bit however if all the VMs are hosted on the same ph=
ysical server this may be slower overall.=C2=A0 For seeding I have heard yo=
u can use 2x the number of cores so you could set up a single mapcache serv=
er using -n 16 pointing at the load balanced source.<div><br></div><div>Reg=
ards</div><div><br></div><div><br></div></div><br><div class=3D"gmail_quote=
"><div dir=3D"ltr" class=3D"gmail_attr">On Mon, 18 Nov 2019 at 10:47, Sebas=
tiano Laini &lt;<a href=3D"mailto:[email protected]"=
>[email protected]</a>&gt; wrote:<br></div><blockquo=
te class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex">





<div lang=3D"EN-GB">
<div class=3D"gmail-m_-5314020211825401210WordSection1">
<p class=3D"MsoNormal">Hi All,<u></u><u></u></p>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
<p class=3D"MsoNormal">I was wondering if is possible to dockerise my serve=
r and then use multiple cluster to seed the cache? I know I can use multipl=
e processes with the =E2=80=93n and =E2=80=93p flag though I=E2=80=99m wond=
ering there is a limit on how much you can really increase
 those number.<u></u><u></u></p>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
<p class=3D"MsoNormal">E.g. I dockerise the server and I spawn a 8core VM w=
ith 16GB ram, I can use sure =E2=80=93p 8 or maybe 10 but still I think tha=
t will require like 60 days to seed the whole UK with our data, can I spawn=
 another machine and speed up the process?
 For example another 8core VM.<u></u><u></u></p>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
<p class=3D"MsoNormal">We have 4 caches with different style that are updat=
ed twice a year so if it takes me 60 days for each by the time I finish it=
=E2=80=99s already time to start again.<u></u><u></u></p>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
<p class=3D"MsoNormal">I was wondering maybe I can run multiple VM and seed=
 one or maybe 2 levels on each VM or if I spawn a new VM the original proce=
ss will take advantage of the new VM?<u></u><u></u></p>
<p class=3D"MsoNormal">In a VM I=E2=80=99m going to have MapServer and MapC=
ache and I will use a subdomain under a load balancer just to seed while th=
e domain will be use just for the seeded cache and in the same way will be =
under a load balancer.<u></u><u></u></p>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
<p class=3D"MsoNormal">Regards,<u></u><u></u></p>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
<p class=3D"MsoNormal"><span style=3D"color:rgb(31,73,125)">Sebastiano Lain=
i<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"color:rgb(31,73,125)">Web Developer<u=
></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"color:rgb(31,73,125)">Buchanan Comput=
ing<u></u><u></u></span></p>
</div>
</div>

_______________________________________________<br>
mapserver-users mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">mapser=
[email protected]</a><br>
<a href=3D"https://lists.osgeo.org/mailman/listinfo/mapserver-users" rel=3D=
"noreferrer" target=3D"_blank">https://lists.osgeo.org/mailman/listinfo/map=
server-users</a></blockquote></div>

--000000000000ac303b0597b550fc--

--===============3547324356301822546==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline

X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbWFwc2VydmVy
LXVzZXJzIG1haWxpbmcgbGlzdAptYXBzZXJ2ZXItdXNlcnNAbGlzdHMub3NnZW8ub3JnCmh0dHBz
Oi8vbGlzdHMub3NnZW8ub3JnL21haWxtYW4vbGlzdGluZm8vbWFwc2VydmVyLXVzZXJz

--===============3547324356301822546==--