NavList:

Tony, you wrote:
"for several days in March while doing plain arc-cosine of no-fancy Law of Cosines HC computation at LAN. Investigation has shown that the input value to the arccos() was equal to 1,0000000000001. I had to guard that function with an If X<=1 ... check."

Aha. Yes! This is also an interesting issue, but it's almost entirely distinct from the issue of floating point representation and the problem of acos(cos(x)) when x is a small number. This case, where a calculated value come out slightly out-of-bounds for the inverse cosine, usually has a different origin, and there are different ways of managing it. If the value you calculate (just before the acos) come out greater than 1, it can be the result of rounding error, or it can result from small errors in the input parameters. So how do we manage these? First of all, rounding is your friend. You can get rid of "garbage digits" like that trailing one in your example just by rounding to an appropriate number of digits. Trial and error on "edge cases" is a reasonable method for deciding how much rounding to do. But sometimes, the answer is not rounding. Sometimes you should accept a calculated cosine value greater than one.

What?! Am I insane?!! The cosine by definition can never be greater than one. Accept it? What New-Age heresy is this??

Heh. It's actually true. Sometimes the best way to manage "noisy" input data is to let the math off the hook. Permit cases where a calculated cos value comes out greater than one.

When discussing lunars, I often tell folks to calculate "corner cosines" (for series methods) or the "cosine of the difference in azimuth" (for the simple law of cosines direct solurion). But you may have noticed that I do not recommend calculating the angle that corresponds to that cosine value. That is, I don't recommend going to that next step and "unwrapping" the angle inside by applying the acos() function. The reason here is that we really don't need the angle at all. In the direct solution of the lunars clearing problem, we calculate cosDZ, the cosine of the difference in azimuth based on the observed distance and the observed altitudes. We then correct the altitudes and use the calculated value of cosDZ as an input to get the correct distance as an output. At no point do we need to know the actual value of the angle DZ. It's awfully tempting to "unwrap" it by applying acos() but then all we're going to do with it is toss its cosine back in at the next step.

Suppose your calculation while clearing a lunar yields a value for cosDZ of 1.001. That's not just some tiny round-off error. It's a big difference. And if we attempt to get DZ from that value, we get a great big angry error from our device, and the whole analysis crashes to a halt (with rows of red blinking lights and smoke emerging from the top of the tape drives). But what we miss here is that the math doesn't care. The math is, in fact, insensitive to small errors like this in many cases. Although we probably reached this value of 1.001 due to a small error in an observed altitude, the final outcome will not be affected. So let the math do its thing, and don't over-calculate. It works! You can allow cosine values greater than one --in some cases. Crazy but true.

For a practical example, suppose I observe the distance between the Moon and Sun (center-to-center, after pre-clearing steps) as 85.0°. I also observe the altitude of the Moon's center as 45.3° and the altitude of the Sun's center as 49.9°. What is the cleared lunar distance? You set up the Sun-Moon-Zenith triangle:
  cos(LD) = sin(h1) · sin(h2) + cos(h1) · cos(h2) · cos(DZ).
Plug in the values for LD, h1, h2, and we get 
  cos(DZ) = -1.0076739.
Looks bad. But as long as we don't try to get the angle DZ out of this, it will work out fine. We can go on to the next step and get the cleared distance. The math doesn't mind. The cleared distance will turn out to be something like 84.365°. All good.

But let's back up a step. What has gone "wrong" here? What's the problem with the observations as listed at the top of this example? You should be able to see that the inputs cannot describe a real situation. And yet real observations with a little noise in the data might produce these numbers. The math can handle the noise, if we let it. Noise in the data should not crash the computation. The math is robust enough to handle it.

Note: this doesn't apply to all cases where the cos value comes out greater than one, but it applies to some important cases.

Frank Reed