NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
Re: Calculators, cosines, and floating point computation
From: Brad Morris
Date: 2019 Jun 4, 13:09 -0400
From: Brad Morris
Date: 2019 Jun 4, 13:09 -0400
Hi Tony
Frank's history of floating point representation is well written and accurate. As some one who has wtitten software since the late 1960's, last professionally in 2015 for a small company known as GE, I agree whileheartedly with what he wrote.
Floating point can be a PITA.
The problem of floating point representation is hardly confined, however, to electronics. Its a fundamental issue of computation, to wit, how to accurately express a real number in the real world.
Here is an example: My 1849 Bowditch expresses cos(9) as 9.99462 which can be found by taking round(10+log10(cos(9)),5). It is rounded at the 5th decimal place. Without rounding, I get 9.99461982706507, for an error in Bowditch of 0.00000007293493. Yikes?
If I attempt acos(cos(9)), using Bowditch's representation, I must use
Acos(cos(10^(9.99462-10))) which yields 8.99993924767998 for an error of 0.0000406075232. Yikes?
You asked if I was expressing irritation. No sir, the problem of representation is a very old one, not novel to electronics (although it is most evident there). The solution is to either get a better calculator to resolve the issue electronically or to revise your equation to avoid these issues, just as you have done. One other way is to ignore trivial differences (7.29*10^-8 indeed).
The issue you bring forward really demonstrates why calculators (and computers) should not be blindly trusted to provide what you seek.
Brad
On Tue, Jun 4, 2019, 11:18 AM Tony Oz <NoReply_TonyOz@fer3.com> wrote:
Dear Brad, may be I mis-read your reply, but I feel some degree of irritation on your side.
I'm so interested in this Cosine calculator issue because my TI-83+ was constantly throwing an exception for several days in March while doing plain arc-cosine of no-fancy Law of Cosines HC computation at LAN. Investigation has shown that the input value to the arccos() was equal to 1,0000000000001. I had to guard that function with an "If X<=1 ..." check.
Warm regards,
Tony60°N 30°E
