NavList:

Hello, Frank!

Thanks for your comments. The most valuable was the analogy with “metric stick”. It simplifies the process. I tried your proposed simple plane triangle method. I interpreted one leg of the right-angled triangle as sextant angle, and the other one as offset value (both using the same units). The difference between hypotenuse and the longer leg was the final error.

Then I calculated this triangle as spherical one (also with one right angle). For small angles the results were almost identical.

But the most interesting thing was that the results were practically almost identical with those I calculated using Chauvenet’s formula.

From your last post I understood that you think that his formula is incorrect. I started to study, how he derived it. I must confess it was not an easy task to get through all his entangled explanations and very poor designed drawings. But finally I understood them (at least I hope so). I used his geometrical model and calculated spherical triangles with the most comon spherical triangle formulas. I did it in Excel. The results were the same as those I calculated with his formula.

Chauvenet basically calculates a simple right angle spherical triangle equivalent to that I described in the beginning (one side is sextant angle, other is offset, the third: sextant angle+error). He solves it with the simple cosine rule.   The only difference in his aproach is: he introduces a little corrections witch depends on horizon mirror's inclinaliton angle to sighting direction. But this angle affects the results insignifacantly. So I don’t see where “he was not perfect”.       

The only thing I don’t understand is: why your numerical values differs from those I get using any of these methods. My values are about 3 times smaller. This confuses me. I have no other sorce of information where I could get some real values to compare. Your post from the past was the only sorce I could find.

You mentioned errors for 5’ offset:    

0,5°….error 1’

5°….error 0,1’

≥25°….error ≤2”

≥45°….error ≤0,5”

My results for the same input data are:

0,5°….error 0,37’ (with plane triangle method: 0,41’)

5°….error 0,037’ (with plane triangle method: 0,041’)

≥25°….error ≤0,007’ (0,42”) 

≥45°….error ≤0,003’ (0,18”)

Can you explain how did you derive your values? I understand very well that we are talking about insignificant small angles, but still I am curious. :)

Modris Fersters

PS: A Small remark about Chauvenet’s formula’s design. In the previous post I wanted to attach a page with formula from the Chauvenet’s book, but the pdf scan was not very good and the explanations were scattered all over several pages,so I decided to integrate his formula directly in my post. I kept the original form without any transformations as a primary source. I think it is obvious that today while performing calculations we transform his “old fashion” format to more modern one (for example, I did the calculations in Excel, where all inputs and the units were transformed in a way I feel more comfortable to work and taking into account the specific of the program).