NavList:

   

Reply

Several years ago a similar question was discussed between myself and an
acquaintance.  He peaked my interest and I decided to attempt modeling sextant
sight errors in a somewhat formal way.  I focused on only these errors that
involve the actual taking of a sextant altitude, the simulation ignores time
errors, plotting errors (horizontal dilution of precision), or other errors
such as table errors, blunders by the navigator, or unusual atmospheric
conditions.

The following is the short paper that resulted from this attempt to model
sextant sight errors.  I don't claim that the method I've used is perfect (or
even correct!), it is offered it as food for thought.  The simulation program
is written in ANSI C and is available to anyone who wishes to play with it.

To view the results in a browser, you will probably have to turn off "wrap
long lines".  Also be sure to look at the list of descriptive statistics
symbols at the very bottom of the report in order to understand the meaning of
these symbols in the results table.

Unfortunately I'm must attend a funeral tomorrow, so if I don't respond
quickly to any questions that arise, please be patient.

Regards,

Jim Manzari

---- Excerpted from previous email ----

It is important that you understand what the following simulation does not
consider, such as errors introduced by inaccurate time or errors introduced
when transferring the reduced sights to a plotting sheet.  This simulation
addresses only those errors arising in the use of the sextant.

The conclusion is interesting: Given careful use of the sextant, accurate
time, good observing weather, and no blunders on the part of the navigator,
you should be able to fix your position to within plus or minus 2 nautical
miles of the actual position.

Misalignment of the sextant with the true vertical is THE major error to guard
against!

METHOD:
-------

In order to estimate the scale of any possible errors I wrote a short program
to simulate observations of 1000 randomly generated objects uniformly
distributed between 15-60 degrees altitude (elevation angle).

Next I generated a series of random values (of various statistical
distributions) to simulate errors that might effect the accuracy of a position
line.  These simulated errors are:

1) Misalignment of the sextant's vertical axis with the local vertical,
labeled Ve (vertical error) in the attached listing.

        hs = hs - ( hs * Cos( phi ) )

where hs = altitude of the object as measured by sextant, adjusted for error
introduced by misalignment of sextant vertical axis with respect to the true
local vertical, this angle called "phi".

This is, by far, the largest single error and completely dominates all other
errors, barring a blunder by the navigator!!

This error varies directly by the height (elevation angle) of the object and
the cosine of the misalignment angle.  Therefore, low altitude stars will
produce a smaller error for the same misalignment angle.  Of course, a small
misalignment error and a low object will produce the best results.

In the simulation this error varied from 0 to -34 arc-minutes, with an average
error of +1.8 arc-minutes.  This would have the effect of placing the position
line away from the actual position by 1.8 nautical miles in the direction of
the object.

See Dutton's for details regarding this type of error.

2) The sextant's index error, caused by sextant mis-calibration or changes in
the sextant's mechanical structure due to temperature instabilities (Ie).
Generally, a very small error, but may be a major contributor to overall error
in some cheap plastic sextants.  I don't know enough about plastic sextants to
model this error realistically.

In the simulation this error varied from -2.1 to +2.4 arc-minutes with an
average of 0 arc-minutes.  I assumed this error to be a normal/Gaussian
distribution with a mean of 0 and a variance of 0.5 arc-minutes.  This assumes
no fixed bias in the sextant, which in fact may exist, but can be checked
prior to each observation and removed from the solution.

3) The inability of the navigator to estimate the distance to the horizon or
rather the exact line of the horizon.  This may be caused by either wrongly
estimated height of eye (dip) or changes in the height of eye due to wave
action (dD).  This may be a rather large error for all, but experienced
navigators, and varies directly with the wave heights at the time of
observation.  I have assumed average wave height of 10 feet with variance 1/5
of this height for this simulation.

        dip = -60.0 * 0.0293 * sqrt( height-of-eye / 3.2808 )

where dip = correction for height of eye above LWL, in arc-minutes.
Height-of-eye given in feet and converted to meters by 3.2... factor.  The
-60.0 factor produces the result with the correct sign and in arc-minutes.

4) The error in the refraction factor (f) caused by a difference between
actual temperature and pressure at the time of the observation from the
standard temperature of 10C and standard pressure of 1010 millibars.  The
error will effect the correction applied to refraction correction (r) to
determine correct refraction correction (ro).

For the simulation I have assumed a more reasonable temperature of 25C (77F)
with a variance of 2 degrees around this mean value.  For pressure I have
assumed 1020 millibars with a variance of 25 millibars.  These values, I
believe are reasonable for someone cruising between 25N and 25S latitudes.

In any case, these differences from the standard temperature and pressure
produced only about 0.1 arc-minutes error.  They can be disregarded for all
practical purposes, except in extreme or unusual atmospheric conditions, when
any sextant observations should be suspect anyway.

        Ro = -60.0 * ( 0.0167 / tan( Hs + 7.31 / ( Hs + 4.4 ) ) )

where Ro = refraction correction in arc-minutes.  Hs = sextant height
previously corrected for dip and index errors.

        R = f * Ro

where R = refraction corrected for temperature and pressure differences from
standard temperature and pressure.

        f = 0.28 * ( pressure + dP ) / ( temperature * dT + 273.0 )

where f = dimensionless factor used to adjust Ro.  dP = error in estimated
actual pressure at time of observation.  dT = error in estimated temperature.

CONCLUSION:
-----------

Armed with all these corrections and their errors, I then calculated the
corrected sextant altitude.  This was then compared with the altitude with all
the same corrections with no errors introduced.


        ho = hs + Ve + Ie + dip + dD + ro     w/error

        Ho = hs + Dip + Ie + R                w/o error

        dH = Ho - ho

where dH = estimated error in sextant height, in arc-minutes.  In this
simulation the maximum error was -32.7 arc-minutes.  98% of this error was
contributed by an extreme error in vertical alignment of the sextant.

The mean error is -1.81 arc-minutes plus or minus 3.37 arc-minutes.

TABLE OF RESULTS (see key to symbols and column label descriptions):
--------------------------------------------------------------------

     hs    Ve    Ie   dip    dD     r       f    ro       ho   Dip     R       F    Ro       Ho      dH
--------------------------------------------------------------------------------------------------------
15.1010v  0.0v -2.1v -3.1v -4.2v -3.6v 0.9441v -3.5v 14.9649v -3.1v -3.6v 0.9584v -3.5v 14.9918v -32.75v
59.8882^ 33.7^  2.4^ -3.1^  5.1^ -0.6^ 0.9773^ -0.6^ 60.0118^ -3.1^ -0.6^ 0.9584^ -0.6^ 59.8278^   3.98^
37.7077~  1.8~ -0.0~ -3.1~  0.1~ -1.5~ 0.9583~ -1.4~ 37.6627~ -3.1~ -1.5~ 0.9584~ -1.4~ 37.6324~  -1.81~
37.6810M  0.7M  0.0M -3.1M  0.0M -1.3M 0.9582M -1.2M 37.6088M -3.1M -1.3M 0.9584M -1.2M 37.6093M  -1.19M
59.5603!  0.0! -1.7! -3.1! -3.4! -3.6! 0.9488! -3.4! 33.9232! -3.1! -3.6! 0.9584! -3.5! 59.4998!  -2.62!
12.9235D  3.0D  0.7D  0.0D  1.4D  0.8D 0.0051D  0.7D 12.9464D  0.0D  0.8D 0.0000D  0.7D 12.9353D   3.37D

**** COLUMN LABEL DESCRIPTIONS ****

hs  = Sextant altitude of simulated object.
Ve  = Sextant vertical alignment error, arc-minutes.
Ie  = Sextant index error, arc-minutes.
dip = Dip correction, arc-minutes.
dD  = Dip correction error, arc-minutes.
r   = Refraction correction, arc-minutes.
f   = Refraction factor, arc-minutes.
ro  = Total refraction correction, arc-minutes.
ho  = Height observed after corrections.
Dip = Dip correction w/o random errors, arc-minutes.
R   = Refraction correction, arc-minutes.
F   = Refraction factor w/o random errors, arc-minutes.
Ro  = Refraction correction, arc-minutes.
Ho  = Height observed w/o random errors, degrees.
dH  = Difference with or without random errors, arc-minutes.

**** KEY TO SYMBOLS ****

* = sum      ~ = mean     ^ = max      v = min        # = count
@ = range    ! = mode     M = median   V = variance   D = stddev
E = stderr   S = skew     K = kurtosis

   

Reply