Re: Should we revert float16 loops (and what are the precision requirements)?
Ralf Gommers via NumPy-Discussion <[email protected]> Wed, 11 Mar 2026 12:42:32 +0100
| Newsgroups | gmane.comp.python.numeric.general |
|---|---|
| Message-ID | <CABL7CQj0xmEiEZbi4ivo75kkRdYB1fWJP8Mz3gaEfH--U+DHmg@mail.gmail.com> |
--===============6917465063792582636== Content-Type: multipart/alternative; boundary="0000000000004db438064cbe22f5" --0000000000004db438064cbe22f5 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Mar 11, 2026 at 12:13=E2=80=AFPM Sebastian Berg <sebastian@sipsolut= ions.net> wrote: > On Wed, 2026-03-11 at 11:59 +0100, Ralf Gommers via NumPy-Discussion > wrote: > > On Wed, Mar 11, 2026 at 10:58=E2=80=AFAM matti picus via NumPy-Discussi= on < > > [email protected]> wrote: > > > > > On Tue, Mar 10, 2026 at 1:28=E2=80=AFPM Sebastian Berg > > > <[email protected]> wrote: > > > > > > > > Hi all, > > > > > > > > In the NumPy 2.4 cycle, there were some native float16 > > > > implementations > > > > merged with rather low precision leading to the following issue: > > > > https://github.com/numpy/numpy/issues/30821 > > > > > > > > That is, previously, it used float loops so ~0.5 ULP error, now > > > > is is > > > > 2+ULP for many algorithms, on _some_ hardware: > > > > https://github.com/numpy/numpy/pull/23351 > > > > > > > > There is always an argument around that users of float16 probably > > > > don't > > > > care about many ULP, but I guess they also have very few bits of > > > > precision to begin with? > > > > I don't have a huge opinion on it, but we are more and more in > > > > the > > > > position where it is unclear if sacrificing a bit of precision is > > > > the > > > > right thing or not... > > > > > > > > Similar questions actually arise for float32 math, is it OK to > > > > trade- > > > > off precision for performance (or to what degree, everything > > > > trades a > > > > bit)? > > > > We have had discussions around this before but it is still a > > > > difficult > > > > trade-off to make and there is no choice that makes everyone > > > > happy. [1] > > > > > > > > - Sebastian > > > > > > > > [1] We can work towards something like `np.opts(precision=3D"low")` > > > > or > > > > so, but that doesn't change the question of defaults much... > > > > > > I do like the idea of having a precise/fast toggle. Until we can > > > develop one, I think we should prefer precise. So we should revert > > > and > > > document somewhere that float16 (and the soon-to-be-incoming > > > bfloat16) > > > are, in NumPy, container types, and that all the math for them is > > > done > > > as float16. > > > > > > > You meant `float32` here. And yes, I agree. Having a few code paths > > use > > > No, I meant float16, I was replying to "done as float16". Operations on arrays with dtype float16 are largely done as "upcast to float32, perform the operation, downcast again". That's what "container types" means (I've been calling them "storage types", not sure what's the most standard name here). So I agreed with everything you said, and was just pointing out what looks like a confusing typo. I don't think we have a bad variability for > float32 right now and while there is a different discussion to be had > about float32, I think those paths would at least be consistent across > architectures (as it would be custom implementations). > Yes agreed, float32 operations are done in float32, with many a few exceptions. And that's fine and expected. > But it sounds like you agree with "revert" here, which would is my > tendency, even if I don't have a clear picture where to draw the line, > since hardware/platform differences always exist to some degree. > Indeed, I agree. I think the design rules should be simple: - Operations on both float32 and float64 should be done at their native precision without casts. - Operations for float16, and a future bfloat16 if we add that, should be done in float32 when floating-point errors occur. Lossless operations can of course done without casts. - Output dtypes should always be unchanged, so if there's an internal upcast, there should be a matching downcast. There may be some exceptions for operations that are very sensitive to numerical errors accumulating; those should be documented and treated as exceptions to the general rules. Cheers, Ralf > - Sebastian > > > > platform/CPU-dependent instructions like AVX512-xxx ones, and as a > > result > > having a small subset of the NumPy API have different accuracy/speed > > trade-offs seems not all that useful to almost all users. And makes > > it > > harder to build up a mental model of what NumPy is actually doing. > > > > Cheers, > > Ralf > > _______________________________________________ > > NumPy-Discussion mailing list -- [email protected] > > To unsubscribe send an email to [email protected] > > https://mail.python.org/mailman3//lists/numpy-discussion.python.org > > Member address: [email protected] > _______________________________________________ > NumPy-Discussion mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3//lists/numpy-discussion.python.org > Member address: [email protected] > --0000000000004db438064cbe22f5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote g= mail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Mar 11,= 2026 at 12:13=E2=80=AFPM Sebastian Berg <<a href=3D"mailto:sebastian@si= psolutions.net">[email protected]</a>> wrote:<br></div><blockqu= ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px= solid rgb(204,204,204);padding-left:1ex">On Wed, 2026-03-11 at 11:59 +0100= , Ralf Gommers via NumPy-Discussion<br> wrote:<br> > On Wed, Mar 11, 2026 at 10:58=E2=80=AFAM matti picus via NumPy-Discuss= ion <<br> > <a href=3D"mailto:[email protected]" target=3D"_blank">numpy= [email protected]</a>> wrote:<br> > <br> > > On Tue, Mar 10, 2026 at 1:28=E2=80=AFPM Sebastian Berg<br> > > <<a href=3D"mailto:[email protected]" target=3D"_blan= k">[email protected]</a>> wrote:<br> > > > <br> > > > Hi all,<br> > > > <br> > > > In the NumPy 2.4 cycle, there were some native float16<br> > > > implementations<br> > > > merged with rather low precision leading to the following is= sue:<br> > > > <a href=3D"https://github.com/numpy/numpy/issues/30821" rel= =3D"noreferrer" target=3D"_blank">https://github.com/numpy/numpy/issues/308= 21</a><br> > > > <br> > > > That is, previously, it used float loops so ~0.5 ULP error, = now<br> > > > is is<br> > > > 2+ULP for many algorithms, on _some_ hardware:<br> > > > <a href=3D"https://github.com/numpy/numpy/pull/23351" rel=3D= "noreferrer" target=3D"_blank">https://github.com/numpy/numpy/pull/23351</a= ><br> > > > <br> > > > There is always an argument around that users of float16 pro= bably<br> > > > don't<br> > > > care about many ULP, but I guess they also have very few bit= s of<br> > > > precision to begin with?<br> > > > I don't have a huge opinion on it, but we are more and m= ore in<br> > > > the<br> > > > position where it is unclear if sacrificing a bit of precisi= on is<br> > > > the<br> > > > right thing or not...<br> > > > <br> > > > Similar questions actually arise for float32 math, is it OK = to<br> > > > trade-<br> > > > off precision for performance (or to what degree, everything= <br> > > > trades a<br> > > > bit)?<br> > > > We have had discussions around this before but it is still a= <br> > > > difficult<br> > > > trade-off to make and there is no choice that makes everyone= <br> > > > happy. [1]<br> > > > <br> > > > - Sebastian<br> > > > <br> > > > [1] We can work towards something like `np.opts(precision=3D= "low")`<br> > > > or<br> > > > so, but that doesn't change the question of defaults muc= h...<br> > > <br> > > I do like the idea of having a precise/fast toggle. Until we can<= br> > > develop one, I think we should prefer precise. So we should rever= t<br> > > and<br> > > document somewhere that float16 (and the soon-to-be-incoming<br> > > bfloat16)<br> > > are, in NumPy, container types, and that all the math for them is= <br> > > done<br> > > as float16.<br> > > <br> > <br> > You meant `float32` here. And yes, I agree. Having a few code paths<br= > > use<br> <br> <br> No, I meant float16,=C2=A0</blockquote><div><br></div><div>I was replying t= o "done as float16". Operations on arrays with dtype=20 float16 are largely done as "upcast to float32, perform the operation,= =20 downcast again". That's what "container types" means (I&= #39;ve been calling them "storage types", not sure what's the= most standard name here). So I agreed with everything you said, and was ju= st pointing out what looks like a confusing typo.</div><div><br></div><bloc= kquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:= 1px solid rgb(204,204,204);padding-left:1ex">I don't think we have a ba= d variability for<br> float32 right now and while there is a different discussion to be had<br> about float32, I think those paths would at least be consistent across<br> architectures (as it would be custom implementations).<br></blockquote><div= ><br></div><div>Yes agreed, float32 operations are done in float32, with ma= ny a few exceptions. And that's fine and expected.</div><div>=C2=A0<br>= </div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b= order-left:1px solid rgb(204,204,204);padding-left:1ex"> But it sounds like you agree with "revert" here, which would is m= y<br> tendency, even if I don't have a clear picture where to draw the line,<= br> since hardware/platform differences always exist to some degree.<br></block= quote><div><br></div><div>Indeed, I agree.</div><div><br></div><div>I think= the design rules should be simple:</div><div><br></div><div>- Operations o= n both float32 and float64 should be done at their native precision without= casts.</div><div>- Operations for float16, and a future bfloat16 if we add= that, should be done in float32 when floating-point errors occur. Lossless= operations can of course done without casts.</div><div>- Output dtypes sho= uld always be unchanged, so if there's an internal upcast, there should= be a matching downcast.</div><div><br></div><div>There may be some excepti= ons for operations that are very sensitive to numerical errors accumulating= ; those should be documented and treated as exceptions to the general rules= .</div><div><br></div><div><br></div><div>Cheers,<br></div><div>Ralf</div><= div><br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D"mar= gin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1= ex"> <br> - Sebastian<br> <br> <br> > platform/CPU-dependent instructions like AVX512-xxx ones, and as a<br> > result<br> > having a small subset of the NumPy API have different accuracy/speed<b= r> > trade-offs seems not all that useful to almost all users. And makes<br= > > it<br> > harder to build up a mental model of what NumPy is actually doing.<br> > <br> > Cheers,<br> > Ralf<br> > _______________________________________________<br> > NumPy-Discussion mailing list -- <a href=3D"mailto:numpy-discussion@py= thon.org" target=3D"_blank">[email protected]</a><br> > To unsubscribe send an email to <a href=3D"mailto:numpy-discussion-lea= [email protected]" target=3D"_blank">[email protected]</a><br> > <a href=3D"https://mail.python.org/mailman3//lists/numpy-discussion.py= thon.org" rel=3D"noreferrer" target=3D"_blank">https://mail.python.org/mail= man3//lists/numpy-discussion.python.org</a><br> > Member address: <a href=3D"mailto:[email protected]" target= =3D"_blank">[email protected]</a><br> _______________________________________________<br> NumPy-Discussion mailing list -- <a href=3D"mailto:numpy-discussion@python.= org" target=3D"_blank">[email protected]</a><br> To unsubscribe send an email to <a href=3D"mailto:numpy-discussion-leave@py= thon.org" target=3D"_blank">[email protected]</a><br> <a href=3D"https://mail.python.org/mailman3//lists/numpy-discussion.python.= org" rel=3D"noreferrer" target=3D"_blank">https://mail.python.org/mailman3/= /lists/numpy-discussion.python.org</a><br> Member address: <a href=3D"mailto:[email protected]" target=3D"_blank"= >[email protected]</a><br> </blockquote></div></div> --0000000000004db438064cbe22f5-- --===============6917465063792582636== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ NumPy-Discussion mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/numpy-discussion.python.org Member address: [email protected] --===============6917465063792582636==--