Re: Encode UTF-8 optimizations

On 08/31/2016 03:43 PM, [email protected] wrote:
> On Monday 29 August 2016 17:00:00 Karl Williamson wrote:
>> If you'd be willing to test this out, especially the performance
>> parts that would be great!
> [snip]
>> There are 2 experimental performance commits.  If you want to see if
>> they actually improve performance by doing a before/after compare
>> that would be nice.
>
> So here are my results:
>
> strict =3D bless({strict_utf8 =3D> 1}, "Encode::utf8")->encode_xs/decod=
e_xs
> lax    =3D bless({strict_utf8 =3D> 0}, "Encode::utf8")->encode_xs/decod=
e_xs
> int    =3D utf8::encode/decode
>
> all    =3D join "", map { chr } 0 .. 0x10FFFF
> short  =3D "=C5=BElu=C5=A5ou=C4=8Dk=C3=BD k=C5=AF=C5=88 p=C4=9Bl =C4=8F=
=C3=A1belsk=C3=A9 =C3=B3dy " x 45
> long   =3D $short x 1000
> ishort =3D "\xA0" x 1000
> ilong  =3D "\xA0" x 1000000
>
> your   =3D 9c03449800417dd02cc1af613951a1002490a52a
> orig   =3D f16e7fa35c1302aa056db5d8d022b7861c1dd2e8
> my     =3D orig without c8247c27c13d1cf152398e453793a91916d2185d
> your1  =3D your without b65e9a52d8b428146ee554d724b9274f8e77286c
> your2  =3D your without 9ccc3ecd1119ccdb64e91b1f03376916aa8cc6f7
>
>
> decode
>                           all            ilong          ishort         =
long           short
>     my: - int           285.94/s     14988.61/s   4694109.54/s       70=
4.15/s    599678.93/s
>   orig: - int           292.41/s     15121.98/s   4782883.50/s       49=
4.33/s    553182.28/s
>  your1: - int           271.21/s     14232.25/s   4706722.93/s       59=
9.68/s    554941.90/s
>  your2: - int           280.85/s     14090.33/s   4210573.40/s       59=
3.93/s    558487.86/s
>   your: - int           283.23/s     15121.98/s   4500252.51/s       69=
1.95/s    678859.55/s
>
>                           all            ilong          ishort         =
long           short
>     my: - lax            83.28/s       202.22/s    142049.67/s       18=
1.82/s    163352.41/s
>   orig: - lax            53.49/s       201.58/s    152422.11/s       14=
7.13/s    133974.37/s
>  your1: - lax           255.13/s        53.75/s     47590.82/s       56=
0.34/s    431447.77/s
>  your2: - lax           281.71/s        48.41/s     43260.19/s       63=
4.16/s    445365.29/s
>   your: - lax           286.96/s        46.35/s     42848.40/s       63=
2.20/s    442546.52/s
>
>                           all            ilong          ishort         =
long           short
>     my: - strict         90.48/s       200.00/s    143081.15/s       19=
7.53/s    175800.00/s
>   orig: - strict         49.21/s       202.22/s    149447.34/s       14=
2.81/s    128290.63/s
>  your1: - strict        154.94/s        48.16/s     44237.93/s       19=
1.36/s    169228.16/s
>  your2: - strict        158.75/s        40.06/s     37244.06/s       19=
5.95/s    173588.68/s
>   your: - strict        158.26/s        38.54/s     36898.14/s       19=
5.95/s    172504.61/s
>
>
> encode
>                           all            ilong          ishort         =
long           short
>     my: - int       5197722.67/s   5227338.26/s   5210583.97/s   516352=
0.62/s   5227338.26/s
>   orig: - int       5449888.54/s   5381336.48/s   5370254.05/s   544988=
8.54/s   5301624.60/s
>  your1: - int       5244200.62/s   5293830.28/s   5277183.02/s   536148=
3.07/s   5260640.13/s
>  your2: - int       5435994.67/s   5432587.30/s   5398312.30/s   548760=
2.22/s   5606457.74/s
>   your: - int       5261172.17/s   5327441.90/s   5310582.91/s   531058=
2.91/s   5361483.07/s
>
>                           all            ilong          ishort         =
long           short
>     my: - lax          2442.24/s     15084.08/s   2882995.00/s      799=
3.15/s   2716293.65/s
>   orig: - lax          2438.39/s     15121.98/s   2933419.33/s      796=
5.22/s   2665521.81/s
>  your1: - lax          2229.94/s     14908.60/s   2117316.51/s      742=
8.89/s   2011133.75/s
>  your2: - lax          2400.92/s     15121.98/s   3046739.87/s      806=
5.41/s   2742961.18/s
>   your: - lax          2368.00/s     15168.94/s   2862328.67/s      809=
0.85/s   2685694.50/s
>
>                           all            ilong          ishort         =
long           short
>     my: - strict         92.16/s       204.81/s    157772.05/s       20=
0.00/s    190344.59/s
>   orig: - strict         49.04/s       202.22/s    160767.72/s       14=
2.81/s    133548.90/s
>  your1: - strict        147.75/s        46.91/s     46095.57/s       19=
4.36/s    176949.84/s
>  your2: - strict        159.25/s        40.19/s     38034.59/s       19=
6.20/s    185166.45/s
>   your: - strict        158.26/s        38.54/s     37012.73/s       19=
6.20/s    186357.23/s
>
>
> So looks like that experimental commits did not speed up encoder or dec=
oder.
>
> What is relevant from these tests is that your patches slow down encodi=
ng
> and decoding of illegal sequences like "\xA0" x 1000000 about 4-5 times=
.
>

Thanks for your efforts.  Also relevant is that this speeds up=20
validation under decode by over a factor of 5.  Given that most inputs=20
will be mostly valid, this outweighs the slowdown, and so I have pushed=20
the non-experimental non-Encode-changes portions to blead.

We may change Encode in blead too, since it already differs from cpan.=20
I'll have to get Sawyer's opinion on that.  But the next step is for me=20
to fix Devel::PPPort to handle the things that Encode needs, issue a=20
pull request there, and after that is resolved issue an Encode PR.