# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Lunar4.4. vs Frank's online calculator**

**From:**Frank Reed

**Date:**2023 May 14, 10:31 -0700

Htom Trites, you wrote:

"Ahh ... sin 85° should identically equal sin 95° "

Yes. Of course this is a well-known property of the sine function in general. Apps that need to get inverse sine functions have to check the "quadrant" for sines near 90°, but this rarely arises in practice. An example anyway... Suppose I have an equation of this form:

sin(y) = a **·** cos(x),

where a is some constant that we're given and x is an input. Let's consider an example solution of this equation. Suppose a=0.99652 and suppose x=2.281°. We do the numbers and find that the right-hand side [a**·**cos(x)] yields 0.9957, rounded to four digits. What then is the value of y?

We actually have to worry about two issues here. One is the quadrant ambiguity. If you try this on a calculator, taking the arcsine (or inverse-sine) of 0.9957, you'll get 84.6847°. But of course if we flip that over to the other side of 90°, which is 95.3153°, we find that this angle also has a sine of 0.9957. No surprise. And once upon a time, this mattered. But in the modern world? No, not unless a coder has made a very basic mistake.

There's a second issue. If we graph out the sine function, the curve, as we all know, rises up from lower angles, reaches a peak with a value of exactly 1 at 90°, and then falls away symmetrically for angles after 90. That symmetry is the origin of the quadrant ambiguity above. The curve hits 0.9957 on both sides of 90°. But the graph reveals another source of uncertainty. It is very nearly "flat" at the peak. The numbers tell the same story: over a range of 10° in the angular argument, the sine rises from 0.9957 to 1 and then back to 0.9957 --very small changes over a fairly large range of angles. Thus we need much finer resolution on the right-hand side of our equation to "pick off" the correct inverse sine value.

I suggested at the top that the value on the right-hand side was rounded to four digits. We can calculate the implications of that by nudging that value up and down by half of that fourth digit, 0.00005, and seeing how that affects the value of y:

if RHS = 0.99565, then y = 84.6539°

if RHS = 0.99575, then y = 84.7157°

That's a difference of more than +/-0.3° which would be counted as a serious error in many applications. Historically --decades and centuries ago-- this latter issue mattered even more than the quadrant ambiguity since tables of natural trigonometric functions as well as logarithms were published with moderate numbers of digits (fundamentally to save paper and weight, but also to save on printing errors). I say that this issue mattered "more". How could that be? The quadrant ambiguity in this example yields a difference in the result of over 10°. The error resulting from uncertainty in the argument is only 0.3°. The latter, nominally smaller, source of error is worse because 1) the first problem is easily resolved with a rule for a quadrant check, and 2) because a smaller error is more insidious. It's more difficult to spot and debug.

Today? Neither of these issues matters in the vast majority of cases. Quadrant ambiguity is resolved by standard rules. Argument precision is resolved by the usual math operations on nearly all devices working to 15 digits (or more) past the decimal point in every implementation in the modern world. I'm puzzled that Antoine brought this up, especially for the sine function.

Frank Reed