NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
Re: Floating-Point 4-place Nat-Haversine Table
From: Roger W. Sinnott
Date: 2018 Dec 21, 23:46 +0000
From: Roger W. Sinnott
Date: 2018 Dec 21, 23:46 +0000
Tony,
In an earlier message you also asked if the haversine table could have fewer than 4 decimal places for small angles, as well as for those approaching 180. That's what got me curious to investigate this whole problem. But, unfortunately, I don't think fewer places helps much. Here are the maximum possible errors when a 3-place floating-point table is used to find arc-haversines:
Deg Error(')
1.0 0.02
2.0 0.10
3.0 0.07
4.0 0.49
5.0 0.39
6.0 0.33
7.0 0.28
8.0 0.25
9.0 0.22
10.0 0.20
11.0 0.18
12.0 1.65
13.0 1.53
14.0 1.42
15.0 1.33
16.0 1.25
17.0 1.18
18.0 1.11
etc....
1.0 0.02
2.0 0.10
3.0 0.07
4.0 0.49
5.0 0.39
6.0 0.33
7.0 0.28
8.0 0.25
9.0 0.22
10.0 0.20
11.0 0.18
12.0 1.65
13.0 1.53
14.0 1.42
15.0 1.33
16.0 1.25
17.0 1.18
18.0 1.11
etc....
So a 3-place floating-point table from 1 to 10 deg has maximum errors that are about the same as those in a 4-place table between 37 and 143 deg. But in the region from 11 to 36 deg the accuracy of the 3-place table becomes poor.
Roger
