NavList:

   

Reply

Previously I've mentioned my Monte Carlo simulation of celestial
navigation sight reduction with the HO 211 ("Ageton method") table.
Recently the software has been improved to more closely replicate a
human using a printed table, including linear interpolation between
entries. It can also test HO 211 variants such as the Bayless table.
More statistics are collected now too.

Random observing sites and celestial bodies are generated with
approximately uniform density on the Earth and celestial sphere.
Latitude and altitude can be restricted to specified ranges. In the
following tests latitudes were 0 to 70°, and altitudes 5° to 80°. In
each case the statistics were accumulated from one million random test
problems.

The basic simulation chooses the closest table entry in every operation
and does not interpolate. Another option is to interpolate as described
in one of my previous messages:

> "The interpolation of B(R) from A(R) will alone reduce the maximum error
> to about two miles for K = 90°, 0.8 mile for K = 80°, and thus bring the
> error within practical limits for most navigation problems."
>
> Samuel Herrick, THE ACCURACY OF AGETON'S METHOD IN CELESTIAL NAVIGATION,
> Publications of the Astronomical Society of the Pacific, Vol. 56, No.
> 331, p.149 (August 1944).
>
> http://adsabs.harvard.edu/abs/1944PASP...56..149H


Now for my results. With no interpolation, HO 211 root mean square
altitude error was 1.23', 14.6% exceeded half a minute, and worst was
31.3'. Azimuth RMS error was 2.76', 0.184% exceeded 0.5°, worst was
2.0°. (Note the mix of degrees and minutes in the azimuth statistics.)

When every computation of B(R) from A(R) was interpolated, as suggested
in the quote from Herrick, altitude RMS error improved greatly from
1.23' to .55', and errors more than half a minute decreased from 14.6%
to 8.8%. But, contrary to Herrick's "two miles" claim, the worst
altitude error was 25.0'.

In his version of Ageton's table, Bayless recommends interpolating B(R)
from A(R) if angle K (one of the intermediate results of the
computation) is within 8° of 90°. That's much easier, since only 9% of
sights are in that range. If we narrow the interpolation window further,
to K = 90° plus or minus 5°, only 5% of sights fall in that range, yet
accuracy is still much better than the non-interpolated solution.

Still more work is saved if you interpolate when t (meridian angle), not
K, is near 90°. That's because t is one of the inputs of the sight
reduction, whereas K must be calculated, then recalculated if you decide
to interpolate B(R). Whether you use K or t, the error statistics are
almost identical.

In other words, interpolating only the transformation of A(R) to B(R),
when t is within 5° of 90, improves altitude accuracy significantly with
only a small workload increase. Only 4.4% of the sights required
interpolation. RMS altitude error was 0.59', 13.5% exceeded 0.5', worst
altitude was 24.4' off. Azimuth RMS error was 2.65', .171% exceeded
0.5°, worst error was 2.0°.

Whether or not you interpolate, the error distribution is not Gaussian.
If it were, about 1/3 of the results would be more than one sigma (= the
RMS value) from the true values. But if you interpolate per the
preceding paragraph, only 9% of the altitudes and 3% of the azimuths
exceed one sigma from the truth. With no interpolation, those figures
become 3% and 4%. Compared to a Gaussian distribution, many fewer points
are outside the 1-sigma zone, but they have a wider scatter.

Speaking of the Bayless table, it's tabulated every minute instead of
half minute. Another difference is that the precision of the A function
increases from whole numbers to tenths at 85° instead of 84°. With no
interpolation, altitude RMS error = 2.02', 43.0% exceed 0.5° error,
worst error = 43.4'. Azimuth RMS error = 4.56', 0.503% exceeded 0.5'
error, worst was 2.4°.

If you interpolate B(R) from A(R) when t is within 5° of 90, altitude
RMS error = 0.82', 41.8% exceed 0.5' error, worst error = 21.4'

As expected, Bayless is generally less accurate than Ageton. However, in
cases when B(R) is interpolated from A(R), the worst altitude error is
consistently a little better with Bayless. The reason for this paradox
is a mystery.

Both tables increase the precision of function A from whole numbers to
tenths when the angle is near 90°. Often I've wondered if that's worthwhile.

In the case of standard Ageton, with no interpolation, RMS altitude
error increases from 1.18' to 1.25' if all tabular values are whole
numbers. The other error statistics have similar small increases.

If B(R) is interpolated from A(R) when t is within 5° of 90, RMS
altitude error increases from .59' to .68'. The other error statistics
degrade too, though not so much. In my opinion, extracting A and B
values to tenths of a unit from the standard Ageton table is not worth
the extra work.

Results are summarized below. The criterion for interpolation of B(R)
from A(R) is shown on the right. For example, "K<8" means the program
interpolated when K was within 8° of 90°. In some cases I also give the
percentage of sights that meet the criterion.

In a later message I'll explore some theoretical HO 211 variants, such
as a 0.2' tabulation interval. Also, I plan to test the Sadler technique
described in the Bayless book.


alt     alt   alt    az      az    az
RMS     >.5'  max    RMS     >.5°  max

1.23'  14.6%  31.3   2.76'  .184%  2.0°  Ageton no interp
1.26'  14.9%  31.5   2.87'  .184%  2.2°  no interp, whole numbers
  .55'   8.8%  25.0'  2.25'  .111%  2.1°  interp all
  .57'  12.8%  25.6'  2.65   .170%  1.9°  interp K<8 8.5%
  .59'  13.6%  25.4'  2.65   .168%  2.1°  interp K<5 5.3%
  .58'  12.8%  24.6'  2.68'  .174%  2.1°  interp t<8 7.3%
  .59'  13.5%  24.4'  2.65'  .171%  2.0°  interp t<5 4.4%
  .68'  14.0%  24.9'  2.82'  .195%  2.3°  interp t<5, whole numbers

2.02'  43.0%  43.4'  4.56'  .503%  2.4°  Bayless no interp
  .71'  34.2%  21.1'  3.56'  .311%  2.4°  interp all
  .77'  40.9%  20.7'  4.34'  .463%  2.5°  interp B(R) k<8
  .80'  41.8%  22.0'  4.33'  .460%  2.6°  interp B(R) k<5
  .78'  40.9%  21.1'  4.36'  .460%  2.2°  interp B(R) t<8
  .82'  41.8%  21.4'  4.32'  .459%  2.5°  interp B(R) t<5

   

Reply