NavList:

   

Reply

Suppose the standard deviation of your observational error in each sight is 
known approximately from experience. It might be 1.0 minutes of arc... 
perhaps 2.0 minutes of arc. Let's call it s0 generally. Suppose also that you 
make a dozen round-offs in a navigational calculation dropping the "tenths" 
at each step. How much does it matter?

Round-off error first: each individual rounding is equivalent to taking a 
random number from a uniformly distributed random variable with a range from 
-0.5 to +0.5. This distribution has a standard deviation of 0.29 (actually 
1/sqrt(12)). When we do multiple roundings they combine like a "random walk" 
increasing the expected error at a rate equal to sqrt(N) where N is the 
number of steps in the random walk or the number of roundings in this case. 
In the case of twelve roundings, this gives the convenient result that the 
standard deviation of the error is 1.0 minutes of arc. Note that twelve 
roundings combined together takes us from a uniformly distributed random 
variable to a distribution that is nearly a bell curve or gaussian 
distribution and therefore it is nearly indistinguishable from any other 
source of error in its distribution (the tails of the round-off error 
distribution are cut off at the maximum possible error but this is not 
important in practice).

Net observational error in celestial sights is approximately gaussian (a 
normal distribution, bell curve, etc.), though I think most people find that 
the tails of the distribution are "heavier" or "fatter" than a pure gaussian. 
The only significance to this aspect here is that large errors are much more 
likely to come from the observation process than round-off in most cases.

When working a celestial sight, the final result does not depend on these 
errors separately (except in practice sights, see below). All you get is a 
final number around which you place some confidence limits. In terms of 
plotting, you get an LOP and you visualize a band of uncertainty along it 
with some standard deviation width. How wide are those confidence limits? To 
combine two independent sources of error and find the net standard deviation 
of the result, we take the square root of the sum of the squares of the 
separate standard deviations. This is the CRITICAL step in the error analysis 
that would help you decide whether you feel it's legitimate to round-off 
under your observational circumstances. 

The numerical results of rounding compared to observational error are 
surprising to many people. Rounding off the tenths does not matter much if 
the observational error is a bit larger than one minute of arc. If the 
observational error, s0, is 1.0, then (again assuming twelve roundings in the 
total calculation) the net error is 1.4 minutes of arc. Is that acceptable? 
That's up to the user -- I would say it's borderline. If the observational 
error is 2.0 minutes of arc, then the net error is 2.2 minutes of arc. At 
that level, I think most navigators would agree that the small increase in 
the net standard deviation is acceptable and, in fact, effectively 
indistinguishable. Your "confidence in confidence limits" would not 
distinguish 2.0 from 2.2. Whether you round off or not, you would hardly 
notice the difference. And finally, suppose that your observational error is 
0.5 minutes of arc. In that case, the net error is 1.1 minutes of arc so 
rounding off would dominate the net error at that level of accuracy. Under 
the assumption that there are twelve round-offs of tenths in the clearing 
process, the difference between rounding and not rounding is probably 
negligible (it's less than 10%) if the observational error is 2.2 minutes of 
arc or higher, and it is not too significant (less than 33%) if the 
observational error is 1.14 minutes of arc or higher. Observational errors of 
this size would frequently be found in small boat conditions so dropping 
tenths would not be unreasonable in those circumstances.

John Karl previously mentioned one good reason for not droppint tenths, and 
that is when the whole purpose of the sights you're taking is to assess your 
observational error on its own. If you're taking practice sights from a known 
location (and most of us do this), there would no reason to work the 
calculations with reduced precision. Of course, if the goal is really to 
assess observational error, I would skip hand calculations and work them with 
a calculating device which would eliminate the issue of round-off entirely.

For the lunarians, divide everything by ten. We usually work to tenths and 
drop the hundredths of a minute of arc in lunars. Historically, they usually 
worked to seconds of arc, which is in between. If your standard deviation in 
lunar distance observations is 0.25 minutes of arc (which I find to be the 
case regularly) and you're working to tenths (dropping hundredths) with a 
dozen steps in the calculation, then the net standard deviation in the result 
is s=sqrt(0.25^2+0.1^2) or 0.27 minutes of arc, less than a ten percent 
difference and presumably indistinguishable from the observational error 
alone.

Finally, consider what happens when you average four sights, either four 
altitudes or four lunar observations. Assuming systematic error has been 
eliminated, the error of observation is generally cut in half. If that's the 
case, and you average the observations before doing the clearing calculation, 
then you might need to work with greater precision in the calculations. 
Alternatively, you would work up each sight separately, working the 
calculation with the precision appropriate for individual sights, and average 
the results at the end.

-FER




--~--~---------~--~----~------------~-------~--~----~
Navigation List archive: www.fer3.com/arc
To post, email NavList@fer3.com
To , email NavList-@fer3.com
-~----------~----~----~----~------~----~------~--~---

   

Reply