NavList:

Back in March this year the discussion of Bill Lionheart’s video (Topic:  Symmedian talk) ended with some speculative comments about minimizing the error in the observed position.  Bill’s final words “Interesting question is if experienced navigators have found the optimum by experience” set me thinking about what could be calculated. 

Rather than relying on hazy 60 year old memories of seminars such as Bill’s I have written a program to calculate the distribution of position errors for various observing practices using Monte Carlo simulations.  The results are ‘dimensionless’ in being relative to the magnitude of the input observer errors;  which were normally distributed.

I have been unable to find any data on the internet for typical observer errors for celestial measurements.  The article ‘Analysis of Random Errors in Horizontal Sextant Angles’ from the Naval Postgraduate School, Monterey (http://archive.org/stream/analysisofrandom00mill/analysisofrandom00mill_djvu.txt) quotes standard deviations from 2.7’ for observations from ‘unstable platforms’ to 0.6’ from stable ones.

Gary LaPook’s note (1 March) about Flight Navigator celestial qualifications provides an interesting comparison for these results.

Do any NavList readers have information on vertical sextant observer errors or have made an assessment of their own observational error distribution?

The following results come from runs of 100,000 repetitions of random (normally distributed) observation errors with randomised relative bearing differences.

3-body fixes

How often is the true position inside the cocked hat?
I was careless in asserting that with three position lines randomly placed on either side of the true position the probability would be one in eight – if you look at the options it is obviously two in eight.  The simulations confirmed this 25% probability.

Which triangle ‘centre’ to use?
I compared the distance of the true position from the centroid (centre of gravity) with the distance from the symmedian point.  It was interesting to see, in the light of the discussion earlier, that using the symmedian point resulted in a 13% reduction in the mean position error compared with using the centroid. 
But how would a practical navigator identify the symmedian point?  In equilateral triangles they are coincident.  It was interesting to observe the position of the symmedian point in markedly eccentric triangles:  if there was one longest side it moved towards the opposite (largest) angle, if there was one shortest side it moved towards the centre of that side (demonstrating its ‘least squares’ character?).  This is illustrated in Bill’s lecture notes (Frank Reed’s post of 25 Feb).
The lesson, I suppose, is always to choose well spaced bodies and thus avoid eccentric cocked hats – and use the centroid.

3-body fixes
Distance of true position from triangle centre.

Using centroid
Mean position error = 1.27 x observer’s std dev (osd)
Standard deviation of position error = 0.76 x osd
90% lay within 2.27 x osd
95% lay within 2.69 x osd
99% lay within 3.79 x osd

Using symmedian point
Mean position error = 1.14 x osd
Std dev of error = 0.66 x osd
90% lay within 2.00 x osd
95% lay within 2.34 x osd
99% lay within 3.21 x osd


2-body fixes

Single observation of each body
Mean position error = 1.80 x osd
Std dev of position error = 1.41 x osd
90% lay within 3.42 x osd
95% lay within 4.50 x osd
99% lay within 7.31 x osd


Using the average of 5 observations of each body
Mean position error = 0.80 x osd
Std dev of position error = 0.63 x osd
90% lay within 1.51 x osd
95% lay within 2.0 x osd
99% lay within 3.23 x osd


So, using the Monterey “unstable platform” observer’s std devn of 2.7’ a 2-body fix using a single observation for each should have a (2.7 x 7.31) 20 NM radius circle drawn around it to define a 99% probability of position (12 NM for 95%, 9 NM for 90%).

Taking the average of, say, 5 observations of each body reduces the input observer’s error by a factor of 2.24 (square root of 5) and results in a circle of (2.7 x 3.23) 9 NM radius for 99% probability (5 NM for 95%, 4 NM for 90%).

These compare with a 3-body fix using a single observation of each body:  a (2.7 x 3.79) 10 NM  radius circle for 99% probability (7 NM for 95%, 6 NM for 90%).

If 5 observations of each body were made and averaged for a 3-body fix the circles come down to (2.7 x 1.67) 4.5 NM radius for 99% probability (3.3 NM for 95%, 2.8 NM for 90%).

Comparing these results with Gary LaPook’s note on Flight Navigator testing:  if we interpret “all within 7 minutes of arc” as a probability of 99% then the implied std devn for readings is 7 / 2.58 (double sided 99% probability) = 2.7’ – exactly the same as the Monterey “unstable platform” figure – and “all” (say 99%) of fixes would plot within 10 NM as above.

One final observation from the runs:  you frequently come across ‘how to do CelNav’ articles which state something along the lines of “if you have a small cocked hat you can be confident in your fix”.  The simulations show this definitely not to be true.  Position errors for the smallest 5% of triangles were very little (less than 10%) different from the average of all sizes of triangle.  However the largest 5% had higher (up to twice) average position errors.

Geoff Butt