# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Sight Reduction Formula Question**

**From:**George Huxtable

**Date:**2005 Jan 12, 01:32 +0000

There are two separate topics merged into Peter Fogg's recent pair of mailings, for both of which George Bennett's "Complete on-board Celestial Navigator has provided the examples. The topics are 1. The effect of multiple roundings on precision of the result, which I will respond to under the threadname "On potential error introduced by rounded values." 2. The defects of the formula for obtaining azimuth from its sine and the errors that it gives rise to in Bennett's lookup table for azimuth, which I'll deal with here under "Sight Reduction Formula Question". I wrote earlier- "There's a third option, that for some reason doesn't find its way into >many textbooks. Get the azimuth from its tan! This formula is- Tan Z = sin >(hour-angle) / (cos (hour-angle) sin lat - cos lat tan dec) and the rules >for putting Z into the right quadrant, 0 to 360, clockwise fron North, are- >If tan Z was negative, add 180 deg to Z. >If hour-angle was less than 180 deg, add another 180 deg to Z." I should perhaps have added (which ought to be obvious), if the To which Peter Fogg replied- "Sounds great. When can we expect to see a production model? Could >this method be turned into a simple 'look up' table?" ================ I doubt if it could. It would need someone much more clever than I am. That is the one advantage of the sin Az procedure; it lends itself to a simple lookup table, such as Bennett has devised. Bennett's table provides a quantity X, which is sin B cos C. In the first lookup operation, the rows are regarded as LHA and the columns as Dec, so this provides X = sin LHA cos dec. In the second operation, the rows are of the azimuth that will result and the columns are Altitude, so this provides X = sin Az cos Alt. And the double lookup operation gets to a solution for Az by equating the two values of X, so sin LHA cos dec = sin Az cos Alt, or Sin Az = (sin LHA cos Dec) / cos Alt which was what we were after. The errors result from an unfortunate combination of using the arc-sine function (which requires very precise numbers to work from, when angles are near East or West) with the rounding of all quantities to the nearest degree (which implies that those numbers aren't very precise at all). By the way, if anyone is interested in the Bennett sight-reduction table for altitude, which tabulates three numbers for each minute of arc between 0 deg and 90 deg, those three quantities appear to be, in order- -13,030 log cos (lat or dec) -13,030 log haversine LHA 200,000 haversine (lat ~ dec) where the haversine of an angle is (1-cos (angle))/2 ==================== Peter has provided the following information- >George Bennett has made a preliminary examination of the probability of >error using these azimuth tables. The following is taken from that work. >Examples of Rounding-off errors in Sight Reduction Tables and Azimuth Tables Now that's interesting! I don't remember seeing that work elsewhere. Was it posted to Nav-L? No warning, or analysis, about azimuth errors seems to exist on the Bennett website. I would like to read the full text rather than extracts. Where can I do so, Peter? I would like to understand Bennett's analysis methods. Bennett adds- >I have added a footnote in the new edition which warns the user to take care >when observations have been made near the PV and substantial rounding off >must be made. When in doubt, interpolate. Good. I raised the subject in the first place because no such warning had been given in my copy. And in that edition, the user was clearly told that interpolation was unnecessary. Peter Fogg commented- >The highly selective >example showing a 15 degree error is too remote to be of practical concern. It was deliberately chosen to be an extreme example (but not a unique one) to show the maximum errors that could occur. There's a wide range of combinations of LHA, dec, and alt which can give rise to errors in azimuth up to, but less than, that 15 degree limit. George. ================================================================ contact George Huxtable by email at george@huxtable.u-net.com, by phone at 01865 820222 (from outside UK, +44 1865 820222), or by mail at 1 Sandy Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK. ================================================================