NavList:

We got as far as a nice, rather short equation for distance in my last post:
  D = R·sqrt[α² + 2(H-h)/R] - R·α,
but this needs some dusting and scrubbing to turn it into a formula with practical units and to deal with refraction. I'll jump to the last line and give that now (also see image):
  D = sqrt[(R_f·α)² + R_f·1.13(H-h)] - R_f·α.
In this formula, the final result, D, is in nautical miles, alpha (α) is in minutes of arc, and H-h is in feet (if you prefer meters, change the constant 1.13 to 3.71. The factor Rf is a measure of the effective increase in the radius of the Earth due to refraction, and on average it is 1.2 or 6/5. There's no reason to assume average weather conditions, and I recommend running the calculation with Rf=1.1 and again with Rf=1.5 to put some reasonable error bars on the distance that you calculate. Or you can play it very safe and set Rf =1 which would apply if refraction did not exist. Important note: don't forget that alpha is the angle above the true horizon, not your sextant reading. Subtract dip from the sextant angle before entering this formula. Also, if you want to be extra careful, the dip should also be adjusted for variable refraction. 

If you open a recent edition of "Bowditch", you'll find Table 15 and an explanation of how it was computed (see image). There's some weirdness in the table. It is extended to unlikely angular altitudes. And the formula they provide is un-necessarily complicated [see attached page]. Notice that their formula uses tan(alpha). This is a common failing in many navigational analyses. For angles smaller than a few degrees, you don't trig. Just use the simple ratio definition of an angle: angle = size/distance. No tangents needed. Bizarrely they have included some tiny numbers in the denominators of their formula. I think we all know that x/0.00025 is the same as 4000·x. The math works out the same. But why would anyone prefer the former?! It's just odd. And, if we skip the whole tangent thing, the constants are much more manageable. We go from tan(α)/0.0002419 to 1.2·α with α in minutes of arc (try it: if α is 1°, you should get 72, either way). In addition, although this Bowditch explanation mentions that there may be errors if refraction is non-standard, it offers no further details and no option to try different values for refraction. And yet it's so easy (see above, and see image).

Here's something "fun" to ponder, both to understand how this relationship works and also to experiment with the math. Suppose I watch this vessel, that I have dubbed "SS Botany Bay" as it recedes into the distance from my annoying cockroach-infested paradise. Let's suppose it's getting dark by then, and maybe, just for visual detail, there is a white light at the very top of the mainmast. At some point, you will see that light flickering as it finally disappears on the horizon. How far away was the vessel at that moment, when the top of the mainmast was right on the sea horizon? We can find out by applying the equation here, with its square root and factors of Rf, for this special case when the sextant angle is zero. But we can also apply another equation that we already know: the formula for "range of visibility" equal to the sum of the distance to the horizon for an observer at height h and another observer at height H. Do they give the same result? And can you convince yourself that they must do so as a matter of mathematical necessity?

So what about all that "dusting and scrubbing" to get the units right and account for refraction that I mentioned at the top? I'll get to those remaining details in another post. 

Frank Reed
Clockwork Mapping / ReedNavigation.com
Conanicut Island USA

File:

File:

File: