NavList:

From: Antoine Couëtte
Date: 2025 Feb 19, 10:19 -0800
In further reference to :
R Gleason's initial query
including ref /RG1/ : 156889.open.cpn.altitude.calculation.error.pdf.pdf

I have checked the calculations in the PDF with my Lunar program. The first is the Sun LL observation.

Ho
11 30.6 manual calculation with almanac
11 30.8 Open CPN
11 30.7 Antoine
11 30.6 me

Hc
11 31.5 manual calculation with almanac
11 38.2 Open CPN
10 52.9 Antoine
10 52.8 me

Not good. Antoine and I agree, but we differ from both Hc values in the PDF. Let's see if the Hc discrepancy is due to inaccurate Sun coordinates.

Sun GHA
128 20.5 manual calculation with almanac
128 20.5 Open CPN
128 20.5 Antoine
128 20.5 me

Sun declination
-18 15.2 manual calculation with almanac
-18 15.2 Open CPN
-18 15.2 Antoine
-18 15.2 me

The Sun coordinates from the plug-in are accurate, so the Hc discrepancy is a mystery. There may be a typo in the document.

Example 2 in the PDF is a Venus observation.

Ho
42 14.1 manual calculation with almanac
42 14.0 Open CPN
42 14.1 Antoine
42 13.9 me

Hc
42 15.5 manual calculation with almanac
42 18.8 Open CPN
42 15.4 Antoine
42 15.3 me

Venus GHA
111 54.9 manual calculation with almanac
111 55.0 Open CPN
111 55.0 Antoine
111 55.0 me

Venus declination (no values in PDF)
-14 53.9 Antoine
-14 53.8 me

Final example in the PDF is Regulus. As Antoine says, Hs 28 58.8 must be
a typo. Degrees should be 26.

Ho
26 52.5 manual calculation with almanac
26 52.5 Open CPN
26 52.5 Antoine
28 52.6 me

Hc
26 51.1 manual calculation with almanac
26 44.8 Open CPN
26 51.1 Antoine
26 51.1 me

Regulus GHA
80 31.4 manual calculation with almanac
80 31.2 Open CPN
80 31.4 Antoine
80 31.5 me

Regulus declination (no values in PDF)
11 50.6 Antoine
11 50.6 me

Summary: the corrections for IE, dip, refraction, SD are OK. The almanac function of the plug-in generates good GHA. For dec, only the Sun value could be checked, but it was OK The trouble seems to be in the "sight reduction table," i.e., the Hc computation. Given assumed position, GHA, and dec, the math is not hard, but somehow it's going wrong, and (weird!), the errors are only a few minutes.

I think Hc and Zn are calculated by the function at the end of this file:
https://github.com/rgleason/celestial_navigation_pi/blob/improve/src/Sight.cpp

The first thing is some calls to function "resolve_heading_positive()" to adjust arguments to the interval [0, 360]. (That function is defined on line 67.) Thus, -10° becomes +350°. OK, sines and cosines of both angles are identical. But why do the adjustment?

[P.J. Plauger, in his book "The Standard C Library" gives complete source code for the trig functions. He warns that you should not try to help a function by removing multiples of a whole circle from the argument. A properly written function will do that, and probably do a better job than you.]

The plug-in does a lot of conversion between degrees and radians. The constant 180 occurs more than 60 times in that file!

Hc is obtained from the familiar cosine formula for great circle distance between two points. However, I'm not absolutely sure this is the source of the inaccurate values in the PDF. The plug-in code is difficult to follow. There may be another Hc computation somewhere else. There's much repetition of code. For instance, refraction is computed three places in that file, not by calls to one routine, but by repeating the same lines of code. See lines 606, 744, 815.

The index correction uses some dangerous coding. It begins at line 580, where double precision variable IndexCorrection is initialized by the user's index error input, with no change of sign. But a few lines later we see that variable ApparentAltitude is initialized by subtracting indexCorrection. Thus, two wrongs make a right. And as I said earlier, the plug-in seems to get the correct values for Ho.

The reason for the inaccurate Hc is still a mystery. Maybe a 170 among all those 180s? That's a joke, but I suspect the Hc/Zn function is getting slightly corrupted input.

--
Paul Hirose
sofajpl.com