NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
From: Robin Stuart
Date: 2021 Jun 23, 08:38 -0700
Bill Lionheart brought to attention an article entitled "The cocked hat: formal statements and proofs of the theorems" by Imre Bárány , William Steiger and Sivan Toled that had appeared online in the Journal of Navigation https://NavList.net/m2.aspx/Prob-14-being-cocked-hat-Lionheart-feb-2021-g49913 . A preprint version of the article can be found at https://arxiv.org/abs/2007.06838 and the paper has now appeared in print (The Journal of Navigation , Volume 74 , Issue 3 , May 2021, pp. 713 - 722 https://doi.org/10.1017/S0373463321000011).
The authors make what seems to be a surprising statement, "In this paper we prove that the likelihood is indeed 1/4 if we assume that the lines of position always intersect pairwise. We also show that the result does not hold under weaker (and more reasonable) assumptions,...". But don't LoPs always intersect pairwise?!
Digging deeper it becomes clear that the authors are not considering the types of cocked hat one gets in celestial navigation but rather those obtained by taking bearings on 3 landmarks or radio beacons. In celestial navigation the LoPs are lines of constant altitude and are true lines infinite in two directions. In this article the LoPs are rays of constant azimuth drawn from the landmark toward the observer that are only infinite in one direction. This opens up the possibility that some pairs of rays may not intersect but presumably limits the relevance of the paper's conclusions for celestial navigation.
Looking back at past papers on the cocked hat it does appear that many authors have in mind one constructed from lines of constant bearing. A seminal paper by R. G. Stansfield. Statistical theory of D.F. fixing, Journal of the Institution of Electrical Engineers Part IIIA: Radiocommunication, 94(15):762–770, 1947 which is concerned with getting a fix using radio direction finding and makes some reasonable approximations that make the probability distributions equivalent to what you have in celestial navigation. Other authors do not.
WILLIAMS' COCKED HAT PAPER
In reviewing this matter I decided to look again at J. E. D. Williams paper (The cocked hat. Journal of Navigation, 44(2):269–271, 1991 https://doi.org/10.1017/S0373463300009991 ) in which he makes a valiant but ultimately flawed attempt to compute probabilties for the exterior regions of the cocked hat. I wanted to try to put my finger on where the analysis went astray. If combinatorics work for points inside the cocked hat why shouldn't they work for the outside? Williams was an air navigator and served as president of the Royal Institute of Navigation.
(In my paper Probabilities in a Gaussian cocked hat. Journal of Navigation, 72(6), 1496–1512. https://doi.org/10.1017/S0373463319000110 I write the conditions for the true position to be inside the cocked hat in terms of the intercepts and azimuths and prove that the probability of being inside is indeed 1/4).
The standard handwaving argument that produces the correct conclusion is based on simple combinatorics. Roughly speaking the true position will be inside the cocked hat in all the intercepts are T, T, T or A, A, A for which the probability is
(1/2).(1/2).(1/2) + (1/2).(1/2).(1/2) =1/4
(Actually there's another case to consider also but this is roughly it). This conclusion is independent of the distribution of measurement errors and requires only that there is an equal probability that the true position falls on either side of the LoP. Williams attempted to extend this argument for regions outside the cocked hat and produces the first attached figure. Cook (Random cocked hats. Journal of Navigation, 46(1):132–135, 1993 https://doi.org/10.1017/S0373463300011383 ) was able to demonstrate that the probablities of the true position falling in the exterior regions depended on the error distributions and hence Williams' conclusion could not be correct. To quote Bárány et al "Williams answered with a witty (but scientifically wrong) rebuttal." The topic was then formally closed possibly since Williams had died in September 1992.
Williams lays out his argument by labelling the sides of each line "+" and "-" Side regions are characterized by signs ++- or +-+ and vertex regions by +--. He concludes that the probability of the true position failling in what we'll call a side region is 1/6 and a vertex region is 1/12. Look at the 2 cocked hats in the 2nd attached figure. The true position is marked with a dot. The two figures are identical expect for the magnitude of the intercept of the vertical LoP. All the azimuths, all of the directions (T or A) and all but one of the intercepts are identical. In the top figure the true position falls in a side region and in the bottom one it falls in a vertex region. The transition occurs when a particular value of the intercept is reached. This I think is a key complexity missing from Williams' argument.
There is another important point that Cook makes but that Williams did not take account of and in fact rejects in his rebuttal. The 1/4 probability is an average over all possible outcomes and does not apply to any one trial. It is relevant before any sights are made but is no longer applicable when the actual realized cocked hat that has been plotted. Once a particular round of sights have been taken the probability that the true position falls in any specific region either inside or outside of the cocked hat depends on the actual distribution of errors. To see this suppose that by dumb luck the sights produce a miniscule cocked hat. The chances that the true position falls inside will be much less than 1/4.
Robin Stuart