# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Using a slide rule for celnav**

**From:**Gary LaPook

**Date:**2014 Jan 20, 17:44 -0800

To answer my own question, it is because we are limited by the accuracy of the input data, 0.1' precision stated for the tabulated data in the Nautical Almanac but the accuracy of that data taken out of the NA is less, up to a 0.25' error for the Sun and 0.30' for the Moon. Fully 10% of the data will have errors exceeding 0.1'. Then there is rounding of data in the other tables of the NA, the error in determining the horizon, and in determining the index error and in determining your height of eye and the error in the observation itself.

Put them all together and that 0.1' level of precision from your calculation doesn't mean much in the real world.

I posted this back in 2007:

-----------------------------------------------------

Put them all together and that 0.1' level of precision from your calculation doesn't mean much in the real world.

I posted this back in 2007:

-----------------------------------------------------

We are only talking about a position determined by celestial observations involving only random error and that any possible systematic error has been eliminated. The "fix" is the spot marked on the chart derived from analysis if the celestial observations. The "position of the observer" is the location where the observer was located when he took the sights and which can also be plotted on the chart. "Triangle" and "cocked hat" are names for the figure formed by plotting three lines of position on the chart that do not pass through the exact same point. Let's take, to start with, the simpler situation involving only two lines of position and let's have them cross at a 90º angle. In this situation we take their intersection as the fix since there is no triangle to be inside or outside of. However, no one actually believes that the position of the observer is actually located at this intersection given the normal random errors of observation and computation. In fact, we can draw circles around the fix at various radii to depict the various probabilities that the position of the observer is located within the various circles. This can also described as various levels of confidence. If the two lines do not cross at a right angle then you can draw ellipses for the same purpose. This is covered in great detail in appendix Q of the the 1977 edition of Bowditch. The size of the circles and ellipses is determined by the uncertainty in the sights expressed in terms of "standard deviation" or "sigma." I have spent a great deal of time lately analyzing the standard deviation of sights taken with bubble sextants and is appears to be about two minutes of arc or a two nautical mile sigma.. This also comports with the requirements of Federal Aviation Regulation part 63 (14 CFR 63, et. seq.) which sets the accuracy requirement for celestial fixes obtained in flight. Sights taken with a marine sextant should have a much smaller sigma but will be larger than just the accuracy of the instrument taking into account an indistinct horizon, waves on the horizon, observer bouncing up and down in heavy seas changing the dip, etc. For the sake of this discussion let's assume the sigma of sights taken with the marine sextant have a sigma of 1.5 nautical mile which was the value determined by the study of thousands

of observations taken by professional mariners. Based on the analysis of this situation the position of the observer will be somewhere within the 1.5 mile circle (one sigma) centered on the plotted fix 39.3% of the time and within a circle of 1.177 sigma (1.77 NM) 50% of the time. This circle is also known as CEP or circular probable error. Continuing drawing circles, 66% within 2.2 NM, 75% within 2.5 NM, 90% within 3.2 NM, 99% within 4.5 nm and 99.9% of the time within 5.6 NM. At the other end of the distribution there is only a 10% chance that the position of the observer will be within .72 NM of the fix. ( See table Q6c on page 1221 of vol. 1 of Bowditch, 1977 ed.) So, what does this tell us. There is about a 30% chance that the position of the observer will be more than .72 NM but less than 1.5 NM and about a 61% chance that the position of the observer will be more than 1.5 NM from the plotted fix. So what do we do with this knowledge? We use the plotted fix at the intersection of the two LOPs for our navigational purposes such as measuring our progress and planning the next leg. We also use this fix to deal with the proximity of danger keeping always in mind that the vessel may actually be almost 5 NM from the fix in any direction. Why do we use the intersection as the fix, because there is no better one available since this spot marks the center of possible positions of the observer. No other spot would be as useful for planning purposes or avoiding danger. Also, what methodology would you use in determining another spot to mark the fix?

gl

**From:**Gary LaPook <garylapook@pacbell.net>

**To:**garylapook@pacbell.net

**Sent:**Monday, January 20, 2014 5:08 PM

**Subject:**[NavList] Re: Using a slide rule for celnav

`A thorough study of thousands of sextant observations taken by professional navigators showed a standard deviation of about 1.5 minutes and if you are using H.O. 229 it states that the accuracy is usually within 0.2' with some weird cases producing errors up to 3.9' in the Hc so don't get too upset about the possible inaccuracy introduced by using the Bygrave formulas on a ten inch slide rule. It's nice to work to a precision of 0.1' in our tables and calculators but we are really going beyond our data if we think that we are going to get a significantly more accurate fix by doing so. This precision is swamped out by the measurement uncertainty, the uncertainty in the Nautical Almanac and in the computations themselves, why only work to a precision of 0.1' why not 0.0001'?`

gl

gl

**From:**Gary LaPook <garylapook---.net>

**To:**garylapook---.net

**Sent:**Monday, January 20, 2014 4:33 PM

**Subject:**[NavList] Re: Using a slide rule for celnav

`
`

You're right, he Bygrave does about 1 or 2 minutes of accuracy, about like HO 249. If you want greater precision than that then you must stick to the methods that you currently are using. I hope there is no EMP in your neighborhood.

gl

gl

**From:**Greg Licfi <cfi{at}licfi.com>

**To:**garylapook---.net

**Sent:**Monday, January 20, 2014 9:33 AM

**Subject:**[NavList] Re: Using a slide rule for celnav

`
`

Hi Gary,

Perhaps I'm just misinformed; and in this case I would love to be - so please take no offense if I am.

I think I read somewhere that a Bygrave sliderule can only resolve 1 arc minute(?).

This might be better suited to a aircraft with a bubble sextant (+/- 2 arc minutes after averaging ?) then a ship.

I currently do my calculations with: H.O.229, a smart phone app, ICE, and a scientific calculator.

all of which seem to agree within +/- 0.1 arc minute with the USNO. If it (the Bygrave sliderule) can resolve

better than that I would be a convert for sure. Also; how much resolution do you need on the sliderule?

Back in my college days 3 or 4 decimal places was considered good with a 'K&E Decalon(?)5'

BTW: I'm a big fan of ICE - I run it on my laptop & smart phone with something called 'dosbox' My smartphone

app uses USNO software and data, ICE was a USNO product, as is H.O.229. so you would expect them to all

yield the same results (no?)

~Greg

This new ship here is fitted according to the reported increase of knowledge among mankind. Namely,

she is cumbered end to end, with bells and trumpets and clock and wires, it has been told to me, can call

voices out of the air of the waters to con the ship while her crew sleep. But sleep Thou lightly. It has not

yet been told to me that the Sea has ceased to be the Sea.

—Rudyard Kipling

On 01/20/2014 04:28 AM, Gary LaPook wrote:

Perhaps I'm just misinformed; and in this case I would love to be - so please take no offense if I am.

I think I read somewhere that a Bygrave sliderule can only resolve 1 arc minute(?).

This might be better suited to a aircraft with a bubble sextant (+/- 2 arc minutes after averaging ?) then a ship.

I currently do my calculations with: H.O.229, a smart phone app, ICE, and a scientific calculator.

all of which seem to agree within +/- 0.1 arc minute with the USNO. If it (the Bygrave sliderule) can resolve

better than that I would be a convert for sure. Also; how much resolution do you need on the sliderule?

Back in my college days 3 or 4 decimal places was considered good with a 'K&E Decalon(?)5'

BTW: I'm a big fan of ICE - I run it on my laptop & smart phone with something called 'dosbox' My smartphone

app uses USNO software and data, ICE was a USNO product, as is H.O.229. so you would expect them to all

yield the same results (no?)

~Greg

This new ship here is fitted according to the reported increase of knowledge among mankind. Namely,

she is cumbered end to end, with bells and trumpets and clock and wires, it has been told to me, can call

voices out of the air of the waters to con the ship while her crew sleep. But sleep Thou lightly. It has not

yet been told to me that the Sea has ceased to be the Sea.

—Rudyard Kipling

On 01/20/2014 04:28 AM, Gary LaPook wrote:

But if you are thinking of doing celestial computations on a normal sliderule you should consider using the Bygrave formulas instead of the normal cosine formula because they give greater accuracy. See:

http://fer3.com/arc/m2.aspx/Bygrave-formula-accuracy-10-inch-slide-rule-Hirose-jul-2009-g8985

http://fer3.com/arc/m2.aspx/Bygrave-formula-accuracy-10-inch-slide-rule-LaPook-jul-2009-g9019

Or, you can make your very own Bygrave sliderule which provides even greater accuracy, complete plans are available here:

https://sites.google.com/site/fredienoonan/other-flight-navigation-information/modern-bygrave-slide-rule

gl

From:David Cortes <dcortes{at}rwlw.com>

To:garylapook---.net

Sent:Sunday, January 19, 2014 10:48 PM

Subject:[NavList] Using a slide rule for celnav

To Navlist: I learned how to use a slide rule back in high school, and it's been 45-plus years. Can some of you old-timers tell whether it's possible to multiply sin by sin or cos by cos, etc. n one continuous operation, without putting the rule down to write down the number of the first calculated sin or cosin, etc.? David -----Original Message----- From: NavList@fer3.com [mailto:NavList@fer3.com] On Behalf Of Frank Reed Sent: Monday, January 20, 2014 12:13 AM To: dcortes{at}rwlw.com Subject: [NavList] Re: What is a "Class A" sextant? Hi Brad, My understanding of the Kew "Class A" rating was that it was an overall rating. It was the certification required for sextants given to Royal Navy cadets. It combined several factors, and the instrument had to meet various standards on several tests. You may remember a NavList discussion a few years back about tables of "star distances" published in about 1905 for use with Lord Ellenborough's method of testing sextant arc error at sea (*). In the introduction, the authors say that a "Class A" certification implies among "other things" that the centering error (or "arc error" as we would call it today) amounted to less than 1' of arc maximum. Classes B and C would presumably permit progressively greater arc error, and this same source says that the sextant would be "rejected" (in other words, worse than class C) if the arc error was greater than 3'. *that discussion was in March 2010, and here's my first message on thee subject, specifically addressed to you personally, in fact. :) -FER ---------------------------------------------------------------- NavList message boards and member settings: www.fer3.com/NavList Members may optionally receive posts by email. To cancel email delivery, send a message to NoMail[at]fer3.com ---------------------------------------------------------------- Attached File: http://fer3.com/arc/imgx/class-A-sextant.jpg : http://fer3.com/arc/m2.aspx?i=126560

: http://fer3.com/arc/m2.aspx?i=126573

: http://fer3.com/arc/m2.aspx?i=126587

: http://fer3.com/arc/m2.aspx?i=126591