Re: +-Inf and NaN

Re CDC7600 Lisps:

I also broke my pick on deep-pipelined Lisp; I worked on trying to speed up
Lisp on the Intel 80860.  Using today's Gnu C and LLVM Clang compiler would
help enormously, as they understand and deal with pipelining quite well.

BTW, today's GPU's are the logical descendants of deeply pipelined machines
like the 7600; GPU's achieve their FLOP counts through pipelines that might
be 10's or 100's of steps deep.

Deeply pipelined programs typically require lots of *inlining* to eliminate
subroutine overheads and indirections; sometimes this inlining requires 'whole
program optimization', which is now available in the LLVM/Clang system.

But there is no question that Lisp-like programs *love* HW caches -- particularly
HW caches that allow so-called 'write-allocate' caching.  A write-allocate cache
allows short-lived Lisp objects to live & die w/o ever being written into normal
RAM.  (I'm not sure of today's situation, but Intel used to be steadfastly against
write-allocate caches; I hope that this brain damage has finally been eliminated.)

One of the items on my bucket list of programs is to write a Lisp interpreter for
SHA256 hash-consed S-expressions, where the *entire RAM* is simply a cache for
this ginormous 256-bit address space of CONS cells.  Note that the entire interpreter
should fit in the HW caches, so that the whole system should be able to run reasonably
fast, as cache accesses are hundreds of times faster than RAM accesses.

Re short-precision floats:

Yes, graphics -- and now AI -- love short-precision floats (graphics: 16 bits;
AI: 8 bits).  The wonderful thing about really short precision -- e.g., 8-bit
precision -- is that one can do the conversion with HW (or table lookup).  For
example, so-called "mu-law" or "A-law" "companding" is actually a crude form of
*floating point*.  A more elegant form of "mu-law" or "A-law" companders is
so-called 'asinh' coding -- used by astronomers for relative brightness.
'Asinh' coding achieves the same thing as Kahan's *gradual underflow* far
more elegantly than IEEE floats.

"The inverse hyperbolic sine reproduces the perceptive capacity of the human eye, allowing to perceive significantly different brightness levels simultaneously. The asinh function is close to the logarithmic mode but has a better behavior around zero."

(Asinh(x) also handles *signed* numbers, which pure logarithms cannot.)

-----Original Message-----
From: Richard Fateman <[email protected]>
Sent: Feb 21, 2024 2:14 PM
To: Stavros Macrakis <[email protected]>
Cc: <[email protected]>, Robert Dodier <[email protected]>, <[email protected]>
Subject: Re: [Maxima-discuss] +-Inf and NaN

One rationale for NaNs is based on the assumption that you might have pipelined/ vector/ etccomputers where an "interrupt" in impossible -- it  may happen at a time and place that no longer exists.
So the NaN is carried along and as Stavros says, it colors all future results as NaNs.
Since the NaN-ness of the data is encoded in the exponent, the fraction part can be a payload of
some sort, explaining what happened.  If that is possible.   e.g. the program counter at the time...
 
On another topic, it was my understanding that stuff like tensorflow cpu float precision was way way
lower than IEEE.  Like 10 bit mantissa.  I don't know about the various GPU instruction sets, but
it seems like way overkill for graphics to have double (or even full single) precision.  But who
uses GPUs for graphics.  I don't know about bitcoin mining algorithms at all.
Should lisp use GPUs?  If you have stuff to compute that works on GPUs, fine.
 
In 1976 or so I worked one summer at Lawrence Livermore National Labs, bringing up
lisp (intention: Macsyma) on a CDC 7600.  Didn't nearly do it, but ran some timing on it,
and compared it to a PDP-10  [which was available as a front end to the CDC, and of no interest to
people doing "real" computing..]  The PDP-10 was about as fast for Lisp-ish stuff as CDC,
which had 60 bit words, multiple arithmetic units, pipelining, and other features that didn't help.
So finding a computer that is fast for <something> doesnt necessarily mean it is fast
on Lisp.  Consider in-memory pointer-following, subroutine calls, packing/unpacking addresses,
testing types at runtime, etc.
 


On Wed, Feb 21, 2024 at 1:20 PM Stavros Macrakis <[email protected] (mailto:[email protected])> wrote:
On Wed, Feb 21, 2024 at 1:46 PM Raymond Toy <[email protected] (mailto:[email protected])> wrote:

...
I think CLHS isn't really super clear on IEEE floats.


 
CL was defined before IEEE floats existed.
...I agree with what Robert says.  Compilation safety here should change the computed results for valid inputs.


 
I think you mean should not change, right?
 
>  If NaN was truly 'not a number', the numerical functions would trigger
>  an error on input only when compiled with safety, and might be arranged

 
Don't agree. If you're doing a calculation with more than one result, it is perfectly reasonable for some of the results to be NaNs and other results to be valid. Aborting the calculation is not the right thing to do.
 
Ray Toy:
NaN is an important part of preventing numeric calculations from giving incorrect results.

 


Not quite sure what you mean by this because NaN is, in a sense, an incorrect result because we couldn't represent what was asked, like 0d0/0d0 or inf-inf.


 
 
It isn't "incorrect", it's invalid. And arithmetic with it should preserve invalidity. The incorrect result happens if 0.0*NaN gives 0.0 rather than NaN, so you think you have a valid result when you don't.
 
           -s

_______________________________________________
Maxima-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/maxima-discuss