NavList:

Comparison of Some Land-based Celestial Navigation Methods

Part I: Initial Observations and Comments on Equipment Used

Greg Herdt

Abstract

In this work, the accuracy of lines of positions (LOPs) for different instruments are compared as a means of gauging the relative accuracy of different approaches to land-based celestial navigation.   More that 3700 shots (not including index error measurements) were taken in the course of this study.  The comparison in this work includes two artificial horizon devices (Davis and Freiberger) used in conjunction with two nautical sextants (Tamaya Spica with 7x35 scope and a Cassens and Plath with a 6x30 scope), three WWII era aircraft sextants (Link A-12 and Bausch and Lomb AN5854-1 and A8-A), and two bubble attachments (Celestaire practice bubble and Cassens and Plath Professional Horizon).   The data presented show that bubble sextants or attachments are generally accurate to 1.7-2.0nm nautical miles.  Accuracies of ca. 0.5-0.7nm were attained with an inexpensive artificial horizon (or a pie pan) and a nautical sextant.

Introduction

Last year after a hiatus of almost ten years, I resumed my practice of backyard celestial navigation.  Powered by a generous bonus from my employer, a supportive wife, and low celestial navigation equipment costs on e-bay, I was fortunate to secure a wide variety of different sextants and artificial horizon devices for the purpose of continuing my practice of land-based celestial navigation and investigating the pros and cons of different instruments and approaches.  The intention of this work is to also establish approximate limits on the accuracy with which artificial horizon or bubble devices can be used to determine latitude and longitude on land.  My purpose in writing this paper is to share some preliminary findings from that work in the hope of stimulating discussion within the celestial navigation community about best practices and some of the limitations of backyard celestial navigation.  Results reported here will necessarily reflect my limited experience with each of the instruments and as a navigator, as well as some limitations on how the tests were conducted.  Experts who have spent years with the same instrument or who are using a tripod or other mount will undoubtedly achieve better results than I am reporting here.  Your mileage may vary.  

Experimental Approach

The majority of the data reported here was taken from my home in Rio Rancho, NM.  One comparison data set was taken 12 years ago in Albany, NY. GPS measurements of the longitude and latitude of these locations were double checked against Google Earth and topo maps of the area.  Time was measured using a Digital Clock application on my Android phone, which was double checked against a radio atomic clock accuracy to the second.  For the older measurements, a radio atomic clock was used as the chronometer.   In order to correct refraction for the high (ca. 5200 ft) altitude of Rio Rancho, atmospheric pressure was estimated using the U.S. Standard Atmosphere Model (1) and a table of refraction corrections were calculated for 30°F, 50°F, 70°F and 90°F using formulas provided in the Nautical Almanac (2).  Day-to-day variations in atmospheric pressure were not measured or correct in this study.   

Shots were taken while seated on a stool, with elbows braced against the body to provide a stable platform for each set of measurements.  No special mounting devices or tripods were used for this work.   Artificial horizon (A.H.) devices were placed on a small table on the most level surface available, the Freiberger A.H. being carefully leveled upon each setup using the two high quality bubble levels provided for that purpose.  The typical time between shots, including recording results, was about a minute.  The averaging devices provided with the aircraft sextants were not used for this study. 

Index corrections for shots with the Tamaya Spica and Cassens and Plath sextants were calculated from five shots with the reflected image above the sun in the AH and five shots with the reflected image of the sun below the sun in the AH using the standard method (3).   Index corrections for the aircraft sextants, C&P Professional Horizon, and Celestaire Practice Bubble were determined by periodically comparing averages of ten to 15 LOPs consisting of eight shots each against known position.  The shots were chosen to span a range of altitudes (typically 20-60) to account for any such variation in the bubble instruments (none was noted upon analysis).

Each line of position (LOP) was established from eight sequential shots of a given body.  Additional shots were taken if one or more shots appeared to be “bad” during the shooting process.   Eight measurements per LOP were chosen as a good compromise between gathering enough data to get a good average and not taking too much time to shoot each body.   Each set of shots was analyzed using a Microsoft Excel® spreadsheet to plot time v.s. altitude with a linear least squares fit of the data (a quadratic was used near meridian pass), single “flier” data points were removed as necessary, and then shot averages of time and altitude were calculated.  A test that I conducted early in this study showed that this “filtered average” approach yields superior results to using a shot median for bubble sextant observations.  Fits typically had an R² value of 0.98 or better.  Frank Reed’s wonderful USNO Clone site (4) was used as the source for astronomical data used in analysis to save labor.  

Data Summary

Some 463 lines of position, each based on an eight shot average plus measurements to establish index corrections were used in generating the comparisons for this work.  As data was collected and analyzed, the distributions in error from known position for LOPs were evaluated for each method.  Some typical results are shown below.  Although many of the distributions are roughly gaussian or normal, most of the data sets exhibited significant deviations from normal when evaluated with a normal probability plot (not shown here, but see sample distributions, below).  The distributions are not strictly Gaussian due to fluctuations or variations in individual shots, especially with bubble devices.  To address this concern, I decided to use a metric based on half the interquartile (third minus first) range rather than standard deviation (designated Δp herein) as a proxy for accuracy (or spread in the data distribution) in this work.  This approach addresses the challenge of avoiding giving undue weight to fluctuations or flier data that were noted especially in the distributions for bubble devices. By definition, 50% of all LOPs taken with a given approach will fall within ±Δp of the actual position. 

Observations for the different approaches are summarized in the table below.  The column labeled “n” represents the number of LOPs sampled and Δp is the mean error in measured LOPs (in nm) calculated from the observations.  Historical data marked with an asterisk were taken using six shot averages for each LOP instead of the eight used in more recent shots.   The historical data were normalized to eight shots based on the fact that Δp should trend as 1/√n.

Conclusions and Future Work

The key finding of this work has been that position can be established to within less than one nautical mile using a nautical sextant and an artificial horizon (Freiberger or Davis) and the bubble horizon devices typically provide ca. 2-3nm accuracy in position.  These results are consistent with findings reported in earlier literature.  With respect to marine sextants, Schufeldt (5) reported that “with 6x and 3x star telescopes, it has heretofore been possible under similar conditions to obtain an accuracy of about 0.4 miles.”  This is consistent with best sights with a marine sextant and artificial horizon that we report here.   With regard to aircraft sextants, the discussion of the Kollsman periscopic sextant in  H.O. 216 (6) claims that “altitudes can be measured with an instrument error of not more than 2 minutes of arc.”  Our best values align with this number, but remember that our Δp metric represents 50% of all sights.   A table in TM 1-206 gives an expected accuracy of ca. 5 minutes of arc for an average of 10 shots “under normal flying conditions” for observations taken in the air (7).   I take the latter to be an upper bound on what we might expect taking shots in more benign land-based conditions, and the majority of my shots do indeed fall within this limit.  The current measurements are therefore generally consistent with historical expectations.

 The slight difference in accuracy between the Davis AH and the Freiberger AH measured in this study is not compelling.  More data is needed on the Freiberger AH, especially for star shots, where I would expect it will be advantageous for dimmer stars.  Star shots taken with the Davis horizon were relatively consistent with the accuracy of sun shots.  These were limited to brighter stars and planets, with Procyan (mag. 0.4) being the dimmest star I have had success shooting consistently.  This is largely a function of light pollution at my location and probably the fact that I am using water instead of molasses or oil in my AH.  The differences noted between the Spica with a 7x35 scope and the Cassens are Plath with a 6x30 scope appear to be real.  The Tamaya data set was taken first, and I suspect that these shots improved my technique prior to shooting the Cassens and Plath data set.    

The majority of bubble sextant results are clustered in the range of Δp of 1.7 to 2.0nm.  Given the variation in sample size between the observations, we probably cannot draw strong conclusions about the merits of individual instruments, but a couple of points stand out.  The aircraft sextants generally perform on par with the more expensive Cassens and Plath Professional Horizon (CPAH), making it difficult to justify the purchase of the latter.  The fact that comparable results were obtained for star shots in 2010 and 2022 with different sextants and different sample sizes gives us some confidence in these measurements.  Not surprisingly, the low cost Celestaire Practice Bubble, which - after all - was never intended to be used for accurate navigation delivered the worst performance of the approaches evaluated in this study.  

The data presented here is far from complete and larger sample sizes are needed in many cases to draw strong conclusions.  In particular, the lack of star shots with the A8-A aircraft sextants and the Celestaire Practice Bubble will need to be addressed in future work.   The accuracy of shots is likely to be improved with a more stable mount, and the author has recently acquired a tripod for this purpose.  Future studies will evaluate the extent to which mounting or supporting a bubble sextant will improve accuracy.  Use of these instruments is an acquired skill and one would also expect accuracy to improve with experience.  I plan to also evaluate how accuracy improves with use in a future study.   In the current study, results were undoubtedly be skewed by the fact that I do not have extensive experience with most of instruments used.  Comparison of the drift in index error between instruments would also be useful in future work.  For example, I noted less stability in the Cassens and Plath Professional Horizon than my WWII aircraft sextants, but I have not quantified this yet.  How many shots are really needed to establish a good index correction and how often should they be taken?  Is there an inherent advantage in using Polaris or bodies at meridian pass compared to using shots spanning a wider range of altitudes as was done in this study?  Other interesting topics for future work might include the effect of bubble size for those instruments with adjustable bubbles and comparison of averages from the top and bottom of the bubble with shots taken with the body centered in the bubble.   I am sure that there are many more topics than this that merit exploration.

References

1.      Useful standard atmosphere data is located here: https://www.engineeringtoolbox.com/standard-atmosphere-d_604.html 

2.      2021 Nautical Almanac, Commercial Edition, Paradise Cay Publications, p. 280.

3.      This standard practice is described here: https://www.nauticalalmanac.it/en/navigation-astronomy/marine-sextant-nautical-errors-corrections.html

4.      Frank Reed’s USNO Clone Site is located here:  https://clockwk.com/apps/USNOclone

5.      Schufeldt, H.H., Report on Celestial Observations Taken at Key West, Florida, June, 1961, p. 24

6.      H.O. Pub. No 216, Air Navigation, U.S. Navy Hydrographic Office, 1955, p. 511.  This statement was made as part of a discussion of the Kollsman perioscopic sextant.

7.      TM 1-206, Celestial Air Navigation, War Department, 1941, p. 119.

Acknowledgements

John Luykx and Ken Gebhardt Sr. were both very helpful when I started learning about celestial navigation in the late 90’s.  Their advice enabled me to get off to a good start with a variety of instruments at that time.   More recently, Robert Swartz has been great source of quality aircraft sextants and information that has enabled me to embark on the current program of comparison.  Finally, the Nav List community has been an ongoing source of inspiration and useful information to me.  I am indebted to the community for keeping traditional navigation practices alive and constantly teaching me new things.  Any errors in this paper are, of course, my own.

Appendix: Description of the Equipment Used

The Davis AH is well known to most land based celestial navigators.  It is a plastic basin with a set of plastic sun shades and a set of glass covers to protect against the wind.  In this work, the basin was filled with tap water and the plastic sun shades were used.  The Freiberg AH is a more sophisticated device consisting of a frame-mounted mirror with three leveling screws and two high quality bubble levels.  The bubble levels are set perpendicular on the mirror and the leveling screws are then adjusted to level the horizon.  The Freiberger horizon is quite a bit more expensive than the Davis AH and it requires more setup time, but it may provide an advantage in measurement of brighter stars.   In the present study, these artificial horizons were used in conjunction with a Tamaya Spica sextant with a 7x35 scope or a Cassens and Plath sextant with a 6x30 scope.

Three used aircraft sextants, a Bausch and Lomb AN5854-1, a Bausch and Lomb A8-A, and a Link A-12, were compared to a new Cassens and Plath Professional Horizon (CPAH) mounted on an Astra Professional sextant.  A Celestaire Practice Bubble mounted on a used Russian Sno-M sextant was added to the mix, as these are readily available to backyard navigators.  Sun shots using a Tamaya Spica sextant with a 7x35 scope and a Davis artificial horizon filled with water were used as a de facto standard for comparison.

The AN5848-1 aircraft sextant is easy to use, and is the author’s preferred sextant for making star shots.  It has a dimmable lighting system powered by two D-cell batteries, and a rotating drum scale marked every 2’, which can be interpolated to 1’ of arc.  Sights are taken through an eyepiece with 2.5x magnification (not adjustable) which enables sighting of dimmer stars.  There are two optical paths on this instrument, a direct path (familiar to readers of John Letcher’s Self-Contained Celestial Navigation with H.O. 208) for use with a natural horizon and a path through the bubble chamber that is used on land or in the air.  The bubble size is adjustable on this sextant and a variety of filters are available in the optical path.  The sextant has an averaging device that was not used in the present work and also comes with a hanging device for mounting the instrument to provide a more stable observational platform.   The latter was not used in this study.

The Link A-12 is well known to many members of the backyard navigation community.  It provides an indirect optical path for sun shots and a direct optical path using a projection of the bubble onto the viewing glass for star shots.  This is a very lightweight and simple sextant with an averaging device (not used for the present work) and Xylene filled bubble chambers with a bubble size that is fixed upon filling.  A larger bubble was used for sun shots, with a second, smaller bubble being available for star shots.  My A-12 has a dimmable AA battery powered lighting system for the bubble and a second C-cell powered system for lighting the vernier scale.  The vernier scale is incremented to 2’, but can readily be interpolated to 1’ of arc. The instrument also is threaded for mounting on a tripod, for those desiring to make higher accuracy shots.  The lack of magnification may pose limits on shots of dimmer stars for those with older eyes (like the author) or locations with higher levels of light pollution, but these are great, easy to use instruments. 

The A8-A has very similar direct- and indirect optical paths to those used in the Link A-12.  This instrument has a heavier and higher quality construction compared to the A-12, including a built-in handgrip for the right hand.  The A8-A has an easy-to-read drum scale incremented every 1’ of arc that is easily interpolated to 0.5’.  It has a dimmable lighting system powered by two D-cell batteries and a pencil averager that was not used in the present work.  The bubble size is adjustable and two vertical hairlines on the bubble chamber glass make alignment of bodies with this sextant a bit easier than the AN5854-1 or A-12.  

The Cassens and Plath Professional Bubble Horizon (CPAH) is limited to use with either the Celestaire Astra Professional or Cassens and Plath sextants due to a specific electrical connection to the two AA batteries in the sextant handle.  This unit is mounted in place of a scope on the sextant.  The bubble size is not adjustable, but the unit does have a 3x scope that makes sighting of dimmer stars easier and a dimmable lighting system.  There is a separate sunshade attached to the unit that can be used in tandem with the shades on the sextant.  The bubble is centered in a reticle ring and the body is centered in a cross-hair in the scope during observation.  It can be a challenge to center the bubble and the body and the same time, as this instrument is quite sensitive.   Although these units are well designed, they are as expensive as some sextants and probably not the best option for most backyard navigators.

The Celestaire practice bubble is a very inexpensive plastic unit that can be bolted in place of the scope on most standard sextants.  The bubble from an inexpensive level tube is reflected into the optical path and lined up with a hairline and the body of interest, making it possible to observe lower- or upper limbs of bodies as well as their center.  The unit is lighted with light admitted through an opening above the spirit level, meaning that star shots are limited to twilight.  Despite the limitations of these units, their low cost makes them readily accessible and they are probably pretty common in the backyard navigation community.  In this study, I mounted the Celestaire bubble on a light-weight (and really excellent) Russian Sno-M sextant that was purchased online at a relatively low cost.

File:

File: