Re: libpng and multithreading

--===============3596666831101261516==
Content-Type: multipart/alternative; boundary="00000000000071512405ee022610"

--00000000000071512405ee022610
Content-Type: text/plain; charset="UTF-8"

On Mon, Nov 21, 2022 at 6:51 AM Manish Upadhyay <[email protected]>
wrote:

> I was looking to improve performance of libpng by adding threading support
> in compression workflow.
>

You need to do some profiling to see where the time is really going in your
specific case.  Different cases produce radically different results; often
an optimization really only works on one set of test cases.

The  compression is done using zlib - it's not part of libpng - and it is
very heavily optimized.  It also has a parallel implementation called pigz
and it should be relatively easy to fork libpng to use that.  I don't know
what the likely speedups are; it depends on the specific test case...

The main place where libpng itself does something inherently parallelable
is in filter code.  This has two aspects; vector parallel operation which
is, of course, not MT, and MT.  So far as I can remember there is no vector
write code but someone might have done something at some point.  This is
the point where existing parallel code exists on the read side.

For MT it's fairly obvious; there are multiple rows in a typical input
image and they can be filtered independently.  Indeed the current code
tests up to five different filter methods and I think it's easy to make
those tests run in parallel.

It is also possible to replace the heuristics by complete compression using
zlib; this involves running multiple zlib instances using the zlib feature
to clone the compression state.  Actually doing the latter without MT is
likely to have a disastrous effect on performance, but with MT the overhead
is the zlib clone and it sort-of sounds possible.

That said the single fastest way of making a PNG write blindingly fast is
to drop all the filters and select level 3 compression.  You could time
these approaches across a range of images and, in fact, the (unfortunately
named) program 'pngcp' in contrib/tools/pngcp.c has a non-default
compilation option -DPNG_PNGCP_TIMING_SUPPORTED which will enable accurate
compression timing given a POSIX 1993/09 compliant system (i.e. almost
anywhere).

--00000000000071512405ee022610
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Mon, Nov 21, 2022 at 6:51 AM Manish Up=
adhyay &lt;<a href=3D"mailto:[email protected]">[email protected]=
om</a>&gt; wrote:<br></div><div class=3D"gmail_quote"><blockquote class=3D"=
gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20=
4,204,204);padding-left:1ex"><div dir=3D"auto">I was looking to improve per=
formance of libpng by adding threading support in compression workflow.</di=
v></blockquote><div><br></div><div>You need to do some profiling to see whe=
re the time is really going in your specific case.=C2=A0 Different cases pr=
oduce radically different results; often an optimization really only works =
on one set of test cases.</div><div><br></div><div>The=C2=A0 compression is=
 done using zlib - it&#39;s not part of libpng - and it is very heavily opt=
imized.=C2=A0 It also has a parallel implementation called pigz and it shou=
ld be relatively easy to fork libpng to use that.=C2=A0 I don&#39;t know wh=
at the likely speedups are; it depends on the specific test case...</div><d=
iv><br></div><div>The main place where libpng itself does something inheren=
tly parallelable is in filter code.=C2=A0 This has two aspects; vector para=
llel operation which is, of course, not MT, and MT.=C2=A0 So far as I can r=
emember there is no vector write code but someone might have done something=
 at some point.=C2=A0 This is the point where existing parallel code exists=
 on the read side.</div><div><br></div><div>For MT it&#39;s fairly obvious;=
 there are multiple rows in a typical input image and they can be filtered =
independently.=C2=A0 Indeed the current code tests up to five different fil=
ter methods and I think it&#39;s easy to make those tests run in parallel.<=
/div><div><br></div><div>It is also possible to replace the heuristics by c=
omplete compression using zlib; this involves running multiple zlib instanc=
es using the zlib feature to clone the compression state.=C2=A0 Actually do=
ing the latter without MT is likely to have a disastrous effect on performa=
nce, but with MT the overhead is the zlib clone and it sort-of sounds possi=
ble.</div><div><br></div><div>That said the single fastest way of making a =
PNG write blindingly fast is to drop all the filters and select level 3 com=
pression.=C2=A0 You could time these approaches across a range of images an=
d, in fact, the (unfortunately named) program &#39;pngcp&#39; in contrib/to=
ols/pngcp.c has a non-default compilation option -DPNG_PNGCP_TIMING_SUPPORT=
ED which will enable accurate compression timing given a POSIX 1993/09 comp=
liant system (i.e. almost anywhere).</div></div></div>

--00000000000071512405ee022610--


--===============3596666831101261516==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--===============3596666831101261516==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
png-mng-implement mailing list
png-mng-implement-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/png-mng-implement

--===============3596666831101261516==--