NavList:

-----Original Message-----
From: Navigation Mailing List [mailto:NAVIGATION-L@LISTSERV.WEBKAHUNA.COM] On Behalf Of WSMurdoch@AOL.COM
Sent: Thursday, April 18, 2002 5:50 PM
To: NAVIGATION-L@LISTSERV.WEBKAHUNA.COM
Subject: Re: On LOPs

In a message dated 4/17/02 10:33:05 PM Eastern Daylight Time, Gadus@ISTAR.CA writes:


>I am still not convinced that an elliptical shape for the boundary is
>anything more than a convenient approximation when there are more than
>three LOPs but, that aside, the above equations make a lot of sense,
>even to this non-statistician.


I spoke to a statistician at work today who said the error contours were elliptical so long as the error was normally distributed in two dimensions and the two dimensions were at right angles to each other.  I am an engineer and do not pretend to understand.

>
>I don't understand. D, E and F all include the intercepts of the
>sights as terms in their calculation yet the size of the intercept is,
>in part, a result of the arbitrary choice of an assumed position. Unless
>this is supposed to be the standard deviation of that part of the error
>in the estimated position which arises from projecting an azimuth
>(calculated onto the the nearest full degree) and an LOP from the
>assumed position to the estimated position, I don't see why the
>calculation includes the azimuth at all. Can anyone explain that?
>

I don't know either how the formula for S is derived, but this is my guess for why it may be correct.

For each LOP, F is p1 squared and is something like the area of the rectangle with the first estimated position and the closest point to it on the LOP as diagonal corners.  From that is first subtracted the northerly distance from the first estimated position to the closest point on the LOP times the easterly distance from the first estimated position to the improved estimated position.  Like F this has the units of degrees squared.  Then a similar term is subtracted.  It is the easterly distance from the first estimated position to the closest point on the LOP times the northerly distance from the first estimated position to the improved estimated position foreshortened by the cosine of the latitude.  It too has units of degrees squared.  The remaining area is related to the error and may be independent of the first estimated position.  I think azimuth finds its way into the figures because the perpendicular to the LOP defines the clos! est point on the LOP to first estimated position.


>
>But shouldn't "BF" be the assumed latitude and "LF" the assumed
>longitude?
>

You are right.  It should have been (Long, Lat) rather than (Lat, Long).  Compact Data does it like (x, y).  I did not catch it.  With this first found error corrected....

Bill Murdoch

**************************

These are the now once corrected formulas. (Any more mistakes in copying and changing this to survive being typed and sent over the internet are mine.)

From B.D. Yallop and C.Y. Hohenkerk, Compact Data for Navigation and Astronomy for the Years 1991-1995.

If p1 and Z1 are the intercept and azimuth of the first observation, if p2 and Z2 are the intercept and azimuth of the second observation, and so on; form the summations

  A = cos^2 Z1 +cos^2 Z2+...
  B = cos Z1 sin Z1 + cos Z2 sin Z2 +...
  C = sin^2 Z1 + sin^2 Z2 + ...
  D = p1 cos Z1 + p2 cos Z2 + ...
  E = p1 sin Z1 + p2 sin Z2 + ...
  F = p1^2 + p2^2 + ...

As a check A + C should equal n, the number of sights.

  G = A C - B^2

The improved estimated position is (Long, Lat), (LI, BI) made by correcting the first estimated position (LF, BF)

LI = LF + dL
BI = BF + dB

where

dL = (A E - B D) / (G cos BF)

and

dB = (C D - B E) / G

The distance between the first estimated position and the improved estimated position is

  d = 60 sqrt(dL^2 cos^2 BF + dB^2)

If d exceeds about 20 nautical miles, set LF = LI and BF = BI then repeat the calculation until d, the distance between the previous estimate and the improved estimate, is less than about 20 nautical miles.

If three or more position lines are obtained, an estimate of the error may be calculated.  The standard deviation of the estimated position sigma in nautical miles is given by

  sigma = 60 sqrt(S / (n-2))

where

  S = F - D dB - E dL cos BF

  sigma sub L = sigma sqrt(A / G)
  sigma sub B = sigma sqrt(C / G)

In general as the number of observations increases the error in the estimated position decreases.  Statistical theory shows that the estimated position has a probability P of lying within a confidence ellipse which is specified by the lengths of its axes a and b and the azimuth theta of the a-axis, where

  tan 2 theta = 2 B / (A - C)
  a = sigma k / sqrt(n / 2 + B / sin 2 theta)
  b = sigma k / sqrt(n / 2 - B / sin 2 theta)

where

  k = sqrt(-2 ln(1-P)) is a scale factor

Values of the scale factor for selected values of P are given in the table below.

Probability, P    0.39  0.50  0.75  0.90  0.95
Scale factor, k   1.0   1.2   1.7   2.1   2.4

The usual confidence limit is 95%, that is P = 0.95.

The shape of the confidence ellipse depends only upon n and the distribution of the observations in azimuth; whilst the size of the ellipse, apart from the scale factor, depends upon the errors of observation.  The method assumes that the observations have equal weight.  The best results will be obtained when the observations are equally spaced in azimuth.  In such cases the effect of systematic errors on the final calculated position will be minimized.