NavList:

Hi Tony

Frank's history of floating point representation is well written and accurate.  As some one who has wtitten software since the late 1960's, last professionally in 2015 for a small company known as GE, I agree whileheartedly with what he wrote. 

Floating point can be a PITA. 

The problem of floating point representation is hardly confined, however, to electronics.  Its a fundamental issue of computation, to wit, how to accurately express a real number in the real world.

Here is an example: My 1849 Bowditch expresses cos(9) as 9.99462 which can be found by taking round(10+log10(cos(9)),5).  It is rounded at the 5th decimal place.  Without rounding, I get 9.99461982706507, for an error in Bowditch of 0.00000007293493.  Yikes?

If I attempt acos(cos(9)), using Bowditch's representation, I must use 

Acos(cos(10^(9.99462-10))) which yields 8.99993924767998 for an error of 0.0000406075232.  Yikes?

You asked if I was expressing irritation.  No sir, the problem of representation is a very old one, not novel to electronics (although it is most evident there).  The solution is to either get a better calculator to resolve the issue electronically or to revise your equation to avoid these issues, just as you have done.  One other way is to ignore trivial differences (7.29*10^-8 indeed).

The issue you bring forward really demonstrates why calculators (and computers) should not be blindly trusted to provide what you seek.  

Brad

On Tue, Jun 4, 2019, 11:18 AM Tony Oz <NoReply_TonyOz@fer3.com> wrote:

Dear Brad, may be I mis-read your reply, but I feel some degree of irritation on your side.

I'm so interested in this Cosine calculator issue because my TI-83+ was constantly throwing an exception for several days in March while doing plain arc-cosine of no-fancy Law of Cosines H_C computation at LAN. Investigation has shown that the input value to the arccos() was equal to 1,0000000000001. I had to guard that function with an "If X<=1 ..." check.

Warm regards,
Tony

60°N 30°E