NavList:

   

Reply

Recently there has been discussion of sight reduction accuracy with
single precision floating point values. On the principle that an ounce
of experimental data is worth a pound of theory, I ran a Monte Carlo
simulation. In my C# source code the formulas are:

Hc = (float)Math.Asin(sinL * sind + cosL * cosd * cosLHA);

Z = (float)Math.Atan((cosd * sinT) /
             (cosL * sind - sinL * cosd * cosT));

if (Z < 0.0)
    Z = (float) (Z + Math.PI);

The (float) keyword forces a value into single precision. It's necessary
because the arc sine and arc tangent functions return a double precision
result. All variables are single precision.

The test for negative Z is necessary because the Atan() function returns
a result in the interval [+pi/2, -pi/2] radians, i.e., [+90, -90]
degrees. So if it's negative, the angle is converted into the equivalent
in the range 90 to 180 degrees.

To generate a sight reduction problem, the program randomly selects a
latitude, azimuth, and altitude. Latitude and altitude can be restricted
to less than the full 0 and 90 degree range if desired, but I didn't do
so in this test.

With simple random selection of altitude, half the stars will be above
45°. But that's less than half the area of the visible hemisphere. So
altitude selection is weighted to generate constant star density
throughout the sky.

 From latitude, azimuth, and altitude the program computes LHA and
declination. These data are passed to the sight solver routine. Up to
this point all computations are in double precision. The solver computes
altitude and azimuth with the desired reduction method. Its results are
compared to the correct answers, and the errors recorded.

For this test I used 100 000 simulated observations. Root mean square
error in azimuth and altitude was .000667′ and .000296′ respectively.
Max azimuth error was .0878′ at altitude 89°40.2′. Max altitude error
was .0132′ at altitude 89°40.9′.

In production code I wouldn't use the above formulas. The one for Hc
loses accuracy the higher the body, and the one for Z fails if Z = 90°
since the tangent of that angle is undefined. Trouble isn't likely with
real world data, but a user experimenting with the program can create a
degenerate case. You can blow up the Z formula with zero latitude and
declination.

   

Reply