NavList:

Bill,

Something that Greg said suggests a really good example of known, varying variances in sextant observations: two observers using different quality instruments. This is not a purely theoretical case. Suppose you and I are taking sights during a brief observing period (clouds break right at the end of nautical twilight). You have a beautiful Plath sextant from the 1950s (lucky bastard) while I have a Davis Mk 15 plastic sextant. Each of us grabs a single sight on each of two stars. Your sights, based on prior experience with the instrument, have an expected standard deviation of 0.5' (after all corrections are made) while my instrument has an expected standard deviation of 2.0'. These are not unreasonable numbers.

Given that your sights have four times the expected accuracy compared to mine, how can we combine all of these sights in one fix? As I mentioned in an earlier post, one simple approach is to treat the sights with higher confidence as multiples. With four times greater accuracy, you would calculate a least squares fix using four "copies" of your Plath sights and single "copies" of my Davis sights. The advantage of this approach is that many celestial navigators already use a bit of software to work up a fix from multiple sights. Entering the same sight twice or even three or four times to give it greater weight is not difficult, makes sense to the navigator intuitively, and it's mathematically consistent, too (I believe).

This sort of methodology works poorly with outlier sights, and that's the biggest flaw with most of these techniques. Suppose we plot out lines of position from our sights, your pair from the Plath sextant and my pair from the Davis, and we discover that my pair plus one of your sights make a relatively small cocked hat while your second sight, despite the fine sextant and an observation that "felt good", yields a line of position that falls a long way from the crossing point of the other three. Then what do you do? Maybe that second sight was an outlier, a dud well outside the normal standard deviation that we expect. Should we ignore that sight? Or perhaps when we draw error ellipses for the separate pairs of sights, they overlap after all and the apparent discrepancy is imaginary...

I have occasionally written about the distribution of errors in celestial navigation, and I maintain that it's better to think of the errors as drawn from a distribution that has "kurtosis" (an ugly word meaning simply that there are outliers outside the expectations of a "normal" normal distribution). There's a simple way to model this that yields behavior similar to what we see in the real world: with some high probabality P (e.g. P=0.8 or 0.9) draw the errors from a normal distbution with standard deviation sd1, and with probablity 1-P, draw the errors from a normal distribution with standard deviation sd2 where sd2 is typically two or three times larger than sd1. These latter errors are much more common than a simple normal distribution would allow, and they're important. The "tails" in a pure normal distribution fade out much too rapidly. In real-world manual observations, we should expect outliers --errors pulled from that second distribution-- that can grossly affect the mean out of proprtion to their significance.

By far the simplest method for handling outliers (mathematically consistent and practically effective) is to use the median sight in a set. It's fast, requires relatively little calculation, and it ignores outliers neatly. Within the past couple of years, someone posted a link to an article suggesting this procedure. I can't find it right now, but I will keep looking --to give credit where credit is due. It's a good method! Note that using the median only applies to certain situations, especially when we have multiple sights on the same body, but in those cases it works quite well.

Frank Reed