Re: Adding tau-a to scipy.stats.kendalltau variants and then changing Somers' D calculation to using tau-a instead of crosstab for better significant runtime improvements
Lucas Colley <[email protected]> Fri, 19 Jan 2024 13:36:19 +0000
| Newsgroups | gmane.comp.python.scientific.devel |
|---|---|
| Message-ID | <[email protected]> |
--===============0209549887977772431== Content-Type: multipart/alternative; boundary=Apple-Mail-F0155C63-3A8B-47F3-B8EE-CA23AD30CB20 Content-Transfer-Encoding: 7bit --Apple-Mail-F0155C63-3A8B-47F3-B8EE-CA23AD30CB20 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Paul, Feel free to email me (or even better message me or the #newcomers channel o= n our community slack (https://join.slack.com/t/scipy-community/shared_invit= e/zt-1a76bomjr-fuS1ZTnmP7b32kIhLb6QMg)) if you have any questions about subm= itting a PR.=20 That performance improvement sounds promising! Cheers, Lucas > On 19 Jan 2024, at 12:41, P. v.H. <[email protected]> wrote: > =EF=BB=BFHello, >=20 > this is my first time trying to contribute, so please be not too harsh. >=20 > When I recently used the scipy.stats.somersd function on larger data I exp= erienced quite some runtime problems. I found a way to calculate Somers' D i= n an equivalent manner by using D(Y|X) =3D tau_a(X, Y)/tau_a(X, X), for whic= h I added the support for variant "a" to the scipy.stats.kendalltau function= . The runtime improvement was significant for large datasets where this appr= oach achieved approx. 30 times faster runtimes. I believe the reason for thi= s runtime improvement is due to the crosstab calculation in the current setu= p, while kendalltau uses for the disconcordant measures a cypthon implementa= tion making it much faster. >=20 > Would be great to have someone I could ask if I have questions in the proc= ess of submitting my contribution and maybe to also review my code. >=20 > Thanks a lot and best regards coming from Vienna >=20 > Paul > _______________________________________________ > SciPy-Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/scipy-dev.python.org/ > Member address: [email protected] --Apple-Mail-F0155C63-3A8B-47F3-B8EE-CA23AD30CB20 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D= utf-8"></head><body dir=3D"auto"><div dir=3D"ltr"><meta http-equiv=3D"conten= t-type" content=3D"text/html; charset=3Dutf-8"><div dir=3D"ltr"></div><div d= ir=3D"ltr">Hi Paul,</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">Feel fr= ee to email me (or even better message me or the #newcomers channel on our c= ommunity slack (<a href=3D"https://join.slack.com/t/scipy-community/shared_i= nvite/zt-1a76bomjr-fuS1ZTnmP7b32kIhLb6QMg">https://join.slack.com/t/scipy-co= mmunity/shared_invite/zt-1a76bomjr-fuS1ZTnmP7b32kIhLb6QMg</a>)) if you have a= ny questions about submitting a PR. </div><div dir=3D"ltr"><br></div><d= iv dir=3D"ltr">That performance improvement sounds promising!</div><div dir=3D= "ltr"><br></div><div dir=3D"ltr">Cheers,</div><div dir=3D"ltr">Lucas</div><d= iv dir=3D"ltr"><br><blockquote type=3D"cite">On 19 Jan 2024, at 12:41, P. v.= H. <[email protected]> wrote:<br><br></blockquote></div><bl= ockquote type=3D"cite"><div dir=3D"ltr">=EF=BB=BF<span>Hello, </span><br><sp= an></span><br><span>this is my first time trying to contribute, so please be= not too harsh. </span><br><span></span><br><span>When I recently used the s= cipy.stats.somersd function on larger data I experienced quite some runtime p= roblems. I found a way to calculate Somers' D in an equivalent manner by usi= ng D(Y|X) =3D tau_a(X, Y)/tau_a(X, X), for which I added the support for var= iant "a" to the scipy.stats.kendalltau function. The runtime improvement was= significant for large datasets where this approach achieved approx. 30 time= s faster runtimes. I believe the reason for this runtime improvement is due t= o the crosstab calculation in the current setup, while kendalltau uses for t= he disconcordant measures a cypthon implementation making it much faster. </= span><br><span></span><br><span>Would be great to have someone I could ask i= f I have questions in the process of submitting my contribution and maybe to= also review my code. </span><br><span></span><br><span>Thanks a lot and bes= t regards coming from Vienna </span><br><span></span><br><span>Paul</span><b= r><span>_______________________________________________</span><br><span>SciP= y-Dev mailing list -- [email protected]</span><br><span>To unsubscribe se= nd an email to [email protected]</span><br><span>https://mail.pytho= n.org/mailman3/lists/scipy-dev.python.org/</span><br><span>Member address: l= [email protected]</span><br></div></blockquote></div></body></html>= --Apple-Mail-F0155C63-3A8B-47F3-B8EE-CA23AD30CB20-- --===============0209549887977772431== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ SciPy-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/scipy-dev.python.org/ Member address: [email protected] --===============0209549887977772431==--