Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Name or NavList Code:
    HO 211 (Ageton) sight reduction accuracy
    From: Paul Hirose
    Date: 2016 Jun 11, 21:42 -0700

    Previously I've mentioned my Monte Carlo simulation of celestial
    navigation sight reduction with the HO 211 ("Ageton method") table.
    Recently the software has been improved to more closely replicate a
    human using a printed table, including linear interpolation between
    entries. It can also test HO 211 variants such as the Bayless table.
    More statistics are collected now too.
    Random observing sites and celestial bodies are generated with
    approximately uniform density on the Earth and celestial sphere.
    Latitude and altitude can be restricted to specified ranges. In the
    following tests latitudes were 0 to 70°, and altitudes 5° to 80°. In
    each case the statistics were accumulated from one million random test
    The basic simulation chooses the closest table entry in every operation
    and does not interpolate. Another option is to interpolate as described
    in one of my previous messages:
    > "The interpolation of B(R) from A(R) will alone reduce the maximum error
    > to about two miles for K = 90°, 0.8 mile for K = 80°, and thus bring the
    > error within practical limits for most navigation problems."
    > Publications of the Astronomical Society of the Pacific, Vol. 56, No.
    > 331, p.149 (August 1944).
    > http://adsabs.harvard.edu/abs/1944PASP...56..149H
    Now for my results. With no interpolation, HO 211 root mean square
    altitude error was 1.23', 14.6% exceeded half a minute, and worst was
    31.3'. Azimuth RMS error was 2.76', 0.184% exceeded 0.5°, worst was
    2.0°. (Note the mix of degrees and minutes in the azimuth statistics.)
    When every computation of B(R) from A(R) was interpolated, as suggested
    in the quote from Herrick, altitude RMS error improved greatly from
    1.23' to .55', and errors more than half a minute decreased from 14.6%
    to 8.8%. But, contrary to Herrick's "two miles" claim, the worst
    altitude error was 25.0'.
    In his version of Ageton's table, Bayless recommends interpolating B(R)
    from A(R) if angle K (one of the intermediate results of the
    computation) is within 8° of 90°. That's much easier, since only 9% of
    sights are in that range. If we narrow the interpolation window further,
    to K = 90° plus or minus 5°, only 5% of sights fall in that range, yet
    accuracy is still much better than the non-interpolated solution.
    Still more work is saved if you interpolate when t (meridian angle), not
    K, is near 90°. That's because t is one of the inputs of the sight
    reduction, whereas K must be calculated, then recalculated if you decide
    to interpolate B(R). Whether you use K or t, the error statistics are
    almost identical.
    In other words, interpolating only the transformation of A(R) to B(R),
    when t is within 5° of 90, improves altitude accuracy significantly with
    only a small workload increase. Only 4.4% of the sights required
    interpolation. RMS altitude error was 0.59', 13.5% exceeded 0.5', worst
    altitude was 24.4' off. Azimuth RMS error was 2.65', .171% exceeded
    0.5°, worst error was 2.0°.
    Whether or not you interpolate, the error distribution is not Gaussian.
    If it were, about 1/3 of the results would be more than one sigma (= the
    RMS value) from the true values. But if you interpolate per the
    preceding paragraph, only 9% of the altitudes and 3% of the azimuths
    exceed one sigma from the truth. With no interpolation, those figures
    become 3% and 4%. Compared to a Gaussian distribution, many fewer points
    are outside the 1-sigma zone, but they have a wider scatter.
    Speaking of the Bayless table, it's tabulated every minute instead of
    half minute. Another difference is that the precision of the A function
    increases from whole numbers to tenths at 85° instead of 84°. With no
    interpolation, altitude RMS error = 2.02', 43.0% exceed 0.5° error,
    worst error = 43.4'. Azimuth RMS error = 4.56', 0.503% exceeded 0.5'
    error, worst was 2.4°.
    If you interpolate B(R) from A(R) when t is within 5° of 90, altitude
    RMS error = 0.82', 41.8% exceed 0.5' error, worst error = 21.4'
    As expected, Bayless is generally less accurate than Ageton. However, in
    cases when B(R) is interpolated from A(R), the worst altitude error is
    consistently a little better with Bayless. The reason for this paradox
    is a mystery.
    Both tables increase the precision of function A from whole numbers to
    tenths when the angle is near 90°. Often I've wondered if that's worthwhile.
    In the case of standard Ageton, with no interpolation, RMS altitude
    error increases from 1.18' to 1.25' if all tabular values are whole
    numbers. The other error statistics have similar small increases.
    If B(R) is interpolated from A(R) when t is within 5° of 90, RMS
    altitude error increases from .59' to .68'. The other error statistics
    degrade too, though not so much. In my opinion, extracting A and B
    values to tenths of a unit from the standard Ageton table is not worth
    the extra work.
    Results are summarized below. The criterion for interpolation of B(R)
    from A(R) is shown on the right. For example, "K<8" means the program
    interpolated when K was within 8° of 90°. In some cases I also give the
    percentage of sights that meet the criterion.
    In a later message I'll explore some theoretical HO 211 variants, such
    as a 0.2' tabulation interval. Also, I plan to test the Sadler technique
    described in the Bayless book.
    alt     alt   alt    az      az    az
    RMS     >.5'  max    RMS     >.5°  max
    1.23'  14.6%  31.3   2.76'  .184%  2.0°  Ageton no interp
    1.26'  14.9%  31.5   2.87'  .184%  2.2°  no interp, whole numbers
      .55'   8.8%  25.0'  2.25'  .111%  2.1°  interp all
      .57'  12.8%  25.6'  2.65   .170%  1.9°  interp K<8 8.5%
      .59'  13.6%  25.4'  2.65   .168%  2.1°  interp K<5 5.3%
      .58'  12.8%  24.6'  2.68'  .174%  2.1°  interp t<8 7.3%
      .59'  13.5%  24.4'  2.65'  .171%  2.0°  interp t<5 4.4%
      .68'  14.0%  24.9'  2.82'  .195%  2.3°  interp t<5, whole numbers
    2.02'  43.0%  43.4'  4.56'  .503%  2.4°  Bayless no interp
      .71'  34.2%  21.1'  3.56'  .311%  2.4°  interp all
      .77'  40.9%  20.7'  4.34'  .463%  2.5°  interp B(R) k<8
      .80'  41.8%  22.0'  4.33'  .460%  2.6°  interp B(R) k<5
      .78'  40.9%  21.1'  4.36'  .460%  2.2°  interp B(R) t<8
      .82'  41.8%  21.4'  4.32'  .459%  2.5°  interp B(R) t<5

    Browse Files

    Drop Files


    What is NavList?

    Get a NavList ID Code

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    A NavList ID Code guarantees your identity in NavList posts and allows faster posting of messages.

    Retrieve a NavList ID Code

    Enter the email address associated with your NavList messages. Your NavList code will be emailed to you immediately.

    Email Settings

    NavList ID Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site