NavList:

   

Reply

Bill has brought to my attention that the table that I posted earlier may be misleading
in so far as it does not directly answer the question he asked, namely, "what is the
maximum error resulting from linear averaging of a given data set of altitude
observations?"

I had tabulated the difference between the arithmetic mean of two altitudes and the
actual altitude at the middle of the interval. In other words, the maximum error due to
linear interpolation. My purpose was to demonstrate the non linearity of the curve. To
find the answer to Bill's question, you have to apply the table in the data to your
actual sample. For instance, if you take three sights, each 2 minutes apart, you have 2
sights that each contribute an error of 14" and one that contributes 0". So the total
error is 66% of what you find in column 4 m.

For this purpose, the table was incomplete as it lacked the column for 2 minutes
separation; so I add it here (and remove the one for 5).

   Alt             2m             4m             6m
60 deg         0'03"          0'14"          0'31"
75 deg         0'07"          0'30"          1'06"
80 deg         0'11"          0'45"          1'40"
85 deg         0'22"          1'30"          3'21"

Now, if you take five sights in 1 minute intervals, you add the entries in the
appropriate line from columns 2m and 4m  and multiply with 0.4 to find the error. For
seven sights you add the entries from all three columns and multiply with 2/7 or 0.3.

But remember, this is the MAXIMUM error. How bad it really is, say 30 deg of the
meridian, I addressed in my previous message.

Herbert Prinz

   

Reply