Re: Speeding up Python

--===============6550896955655986128==
Content-Type: multipart/alternative; boundary="00000000000023997c05857d7759"

--00000000000023997c05857d7759
Content-Type: text/plain; charset="UTF-8"

The native imath python arrays are pretty fast, I think your version is
triggering an element by element python copy of data into the numpy array.
Here's the timings with the native (fast) array copy and the python only
iteration on the array types.  Here's the script:

import imath
import time

count = 1000000

ai = imath.V3dArray(imath.V3d(0.25, 0.3, 0.75), count)
ao = imath.V3dArray(count)
start = time.time()
ao[:] = ai
end = time.time()

print 'Copying %d elements: %0.06f seconds' % (count, end-start)

start = time.time()

for i in range(count):
    ao[i] = ai[i]

end = time.time()

print 'Python iteration %d elements: %0.06f seconds' % (count, end-start)

And the output:

> python2 foo.py
Copying 1000000 elements: 0.003245 seconds
Python iteration 1000000 elements: 0.994242 seconds

Does that match what you're seeing?  That's also without enabling the
multithreading that's built into the array operations.

-nick

On Mon, Apr 1, 2019 at 12:46 PM Tyler Fox <[email protected]> wrote:

> Getting data in and out of imath in python is *SLOW*. And yes, Python is a
> slow language, but that doesn't mean we can't kick it up a notch.
> Could I get some feedback on my proposals to speed things up?
>
> https://github.com/openexr/openexr/pull/373
> First is a PR with a *very* simple change that just exposes the memory
> address of the imath object.
> With a few lines of python code, you can read the c-data and re-interpret
> it in whatever way you want.
>
> https://github.com/openexr/openexr/issues/367
> This is an issue where I described a more complex (and maybe more
> pythonic?) way of exposing the underlying data.
>
> Either of these proposals could remove the need for imathnumpy... Or
> imathnumpy could be replaced with a pure-python module that handles the
> more error prone memory mapping.
>
> Thoughts?
> ~T.Fox
> _______________________________________________
> Openexr-devel mailing list
> [email protected]
> https://lists.nongnu.org/mailman/listinfo/openexr-devel
>

--00000000000023997c05857d7759
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div><b=
r></div><div>The native imath python arrays are pretty fast, I think your v=
ersion is triggering an element by element python copy of data into the num=
py array.=C2=A0 Here&#39;s the timings with the native (fast) array copy an=
d the python only iteration on the array types.=C2=A0 Here&#39;s the script=
:</div><div><br></div><div><div>i<font face=3D"monospace, monospace">mport =
imath</font></div><div><font face=3D"monospace, monospace">import time</fon=
t></div><div><font face=3D"monospace, monospace"><br></font></div><div><fon=
t face=3D"monospace, monospace">count =3D 1000000</font></div><div><font fa=
ce=3D"monospace, monospace"><br></font></div><div><font face=3D"monospace, =
monospace">ai =3D imath.V3dArray(imath.V3d(0.25, 0.3, 0.75), count)</font><=
/div><div><font face=3D"monospace, monospace">ao =3D imath.V3dArray(count)<=
/font></div><div><font face=3D"monospace, monospace">start =3D time.time()<=
/font></div><div><font face=3D"monospace, monospace">ao[:] =3D ai</font></d=
iv><div><font face=3D"monospace, monospace">end =3D time.time()</font></div=
><div><font face=3D"monospace, monospace"><br></font></div><div><font face=
=3D"monospace, monospace">print &#39;Copying %d elements: %0.06f seconds&#3=
9; % (count, end-start)</font></div><div><font face=3D"monospace, monospace=
"><br></font></div><div><font face=3D"monospace, monospace">start =3D time.=
time()</font></div><div><font face=3D"monospace, monospace"><br></font></di=
v><div><font face=3D"monospace, monospace">for i in range(count):</font></d=
iv><div><font face=3D"monospace, monospace">=C2=A0 =C2=A0 ao[i] =3D ai[i]</=
font></div><div><font face=3D"monospace, monospace"><br></font></div><div><=
font face=3D"monospace, monospace">end =3D time.time()</font></div><div><fo=
nt face=3D"monospace, monospace"><br></font></div><div><font face=3D"monosp=
ace, monospace">print &#39;Python iteration %d elements: %0.06f seconds&#39=
; % (count, end-start)</font></div></div><div><font face=3D"monospace, mono=
space"><br></font></div>And the output:<div><font face=3D"monospace, monosp=
ace"><br></font></div><div><div><div><font face=3D"monospace, monospace">&g=
t; python2 foo.py</font></div><div><font face=3D"monospace, monospace">Copy=
ing 1000000 elements: 0.003245 seconds</font></div><div><font face=3D"monos=
pace, monospace">Python iteration 1000000 elements: 0.994242 seconds</font>=
</div></div></div><div><br></div><div>Does that match what you&#39;re seein=
g?=C2=A0 That&#39;s also without enabling the multithreading that&#39;s bui=
lt into the array operations.</div><div><br></div><div>-nick</div></div></d=
iv></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gma=
il_attr">On Mon, Apr 1, 2019 at 12:46 PM Tyler Fox &lt;<a href=3D"mailto:ty=
[email protected]">[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"g=
mail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204=
,204,204);padding-left:1ex"><div dir=3D"ltr"><div>Getting data in and out o=
f imath in python is *SLOW*. And yes, Python is a slow language, but that d=
oesn&#39;t mean we can&#39;t kick it up a notch.</div><div>Could I get some=
 feedback on my proposals to speed things up?</div><div><br></div><a href=
=3D"https://github.com/openexr/openexr/pull/373" target=3D"_blank">https://=
github.com/openexr/openexr/pull/373</a><div>First is a PR with a *very* sim=
ple change that just exposes the memory address of the imath object.</div><=
div>With a few lines of python code, you can read the c-data and re-interpr=
et it in whatever way you want.</div><div><br></div><div><div><div><a href=
=3D"https://github.com/openexr/openexr/issues/367" target=3D"_blank">https:=
//github.com/openexr/openexr/issues/367</a>=C2=A0<br></div></div></div><div=
>This is an issue where I described a more complex (and maybe more pythonic=
?) way of exposing the underlying data.</div><div><br></div><div>Either of =
these proposals could remove the need for imathnumpy... Or imathnumpy could=
 be replaced with a pure-python module that handles the more error prone me=
mory mapping.</div><div><br></div><div>Thoughts?</div><div>~T.Fox</div></di=
v>
_______________________________________________<br>
Openexr-devel mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Openexr-devel=
@nongnu.org</a><br>
<a href=3D"https://lists.nongnu.org/mailman/listinfo/openexr-devel" rel=3D"=
noreferrer" target=3D"_blank">https://lists.nongnu.org/mailman/listinfo/ope=
nexr-devel</a><br>
</blockquote></div>

--00000000000023997c05857d7759--


--===============6550896955655986128==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Openexr-devel mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/openexr-devel

--===============6550896955655986128==--