NavList:

David Pike, you wrote:
"The situation will be close enough to a right-angled triangle to use Range = height/tan angle between top & bottom."

Of course, there's no actual right-angled triangle here. The triangle you're picturing is an imaginary thing layered on top of the real geometry. In fact, there are at least three distinct right triangles that you could so draw here, and none of them are "right". They're just different fictional geometries superimposed on the real problem. As you correctly note, the geometry is "close enough", but since that's the case, why not use the simplest geometry available? Why not draw something simpler that you can solve in your head (or solve trig-free on the back of the envelope) with exactly the same accuracy in the result?

How? It's 3438 again! Draw distance off and a bit of circular arc superimposed on the real geometry. The circular arc will now correspond to the height of your object instead of the straight vertical side of the former right triangle. And instead of requiring you to solve tan x = height/distance, you're solving x/3438 = height/distance, assuming that the angle is given in minutes of arc. The quantity 3438 is just the ratio of the total minutes of arc in a circle, 360·60, to the total number of radians in a circle, 2π. In both cases, we can convert these (with a drop of algebra) into equations for distance:
   distance = height/(tan x)
or
   distance = 3438·height/x.
For angles up to several degrees, these produce results that are indistinguishable for practical purposes, up to four significant figures for smaller angles (because I'm only memorizing the factor 3438 to that many digits). The huge advantage of the 3438 formula is that you can do it in your head or quickly on the "back of an envelope". And by the way, if you find it too mind-blowing to think of abandoning your trig geometry, then you can simply view x/3438 as an approximation for tan x when angles are small (*).

EXAMPLE: I see a ship in the distance which I am able to identify. I know from an online photo that the height of the vessel (height to some easily identified and fixed upper feature, like the top of an exhaust stack) is 110 feet (+/-2 feet). I grab my sextant and measure the angle from the waterline of the vessel to that top point, and I get an angle of 15.6' (+/- 0.3'). [how do we do that, by the way? What is the best procedure with a marine sextant for making this angular measurement to sufficient accuracy?] Now let's do the math both ways. By the tangent relationship, I get 24240 feet for the distance. By the circular arc relationship, I get 24242 feet. They agree perfectly to four significant digits, which is all we would ever need in any practical case. Indeed, what we should worry about here are the error bars on the inputs. The error bars I have suggested in this example for the height of the vessel are about +/-2%. The error bars I have suggested for the measured altitude are also about +/-2% (just a coincidence in my choices here but not an unusual case). Combined the error bars would shift the estimated distance by +/-3%. You can do that in your head: 1% of 24000 is 240 so 3% of 24000 is 720 feet, might as well call that 700 feet. So our final result, dropping spurious digits, is 24,200 feet +/-700 feet or 4.0 +/-0.1 nautical miles.

Frank Reed

* I said above that you can view x/3438 as an approximation for tan x, and there are well-known ways of justifying that. For example, the infinite Taylor series expansion for the tangent of an angle x, is given by:
   tan x = x + x³/3 + 2x⁵/15 + O(x⁷) 
That is a standard way to calculate "exact" values of tan x in the modern world [here O(...) means "order of" but it's just a shorthand for the continuation of the infinite series to higher powers of x]. Note that x here is the angle converted to a pure ratio or "radians", so the leading term, x, is just (m.o.a.)/3438 [where m.o.a. is the measured angle in minutes of arc]. Using this Taylor series expansion you can decide where x/3438 would be a "close enough" approximation to tan x by recognizing that x³/3 will be a very small number for angle smaller than some limit you choose. But I should emphasize that this "analysis" step is un-necessary. You don't have to think about a Taylor series and approximating the tangent function. Just think about the geometry. There is no natural triangle here, despite all the cases that were beaten into our heads when we first learned trigonometry in school. This "distance to a lighthouse" puzzle was the classic practical example that the instructors could wheel out to demonstrate the "practical value" of trig, but it's overkill for small angles. There is no long skinny triangle except in our mathematical imaginations. There is no "natural" trig problem here. Draw a long skinny sector of a circle instead. The difference is tiny in practical navigation cases and usually dramatically exceeded by the uncertainty in the inputs to the problem as in the example above. 3438 is your friend.