NavList:

   

Reply

Please find attached a corrected version: SAMT 1.1.

George's comments made me look again more in detail in the evaluation
part of the conversion functions. There I could find a small bug in
the sheet "Integral". Due to this bug the sub-ranges were not
calculated quite correct and led to small errors in the calculation of
the conversion functions and thus resulting in slightly higher
off-sets from (0,0). With this correction the constant values of the
linear and quadratic function are zero within the limits of the
overall standard deviation:

I try to explain what I mean with "being zero within the error
limits". I do this on the (now new) result for the Linear fit as shown
in sheet "Conversion":
StdDev to Sub-Range values in sheet Integral: +/-0.06 moa
The Sub-Range values were however obtained by integration from the
calibration function having itself a StdDev of +/- 0.15 moa with
respect to the measured data (see sheet Cal_Poly).
The total error for the linear fit is therefore expected to have a
StdDev of +/- 0.16 moa.
In the linear fit the constant is 0.12 moa this corresponds to zero
within the error limits of
+/- 0.16 moa.

Now regarding the quadratic fit used as conversion formula as shown in
sheet "Conversion".

George comments on this:
"But when arriving at the three possible formulae on his "conversion"
page, and his words under "conversion function" in his "integral"
page, I get confused. If he is trying to obtain such a fit for the
conversion function itself, then as explained in my recent posting,
the quadratic version is unphysical."

I tried to fit the following function which George considered to be
more physical:
a Px^3 + b Px
The correlation is about the same as for the linear fit (StdDev also
+/- 0.6). With the "unphysical" quadratic function the StdDev can at
least be reduced by a factor of 6. The "wrong" constant off-set
(-0.03) is however well within the expected error limits for zero
(+/-0.15).

I don't really understand why George thinks that this function can't
be quadratic. The origin of the dataset, which is used to fit this
function, is derived from the calibration function which is also a
second order polynomial. Why should then the pixel ranges and the
corresponding integrated angles obtained from this second order
polynomial, i.e. from the calibration function not also possibly be
quadratic? I however agree that the constant off-set is
*theoretically* wrong. However, as long as the resulting off-sets are
within the expected error limits they are very likely to be
*physically* correct.

By far the best fit is obtained with the arc-tan-function, which you,
George, proposed. The StdDev is by a factor of about 1000 smaller than
for the linear fit. This fit requires however the tool Solver. The
linear and the quadratic fits are shown for those who can't use
Solver. But for the reasons mentioned earlier, these fits may
eventually be completely sufficient when looking at the attainable
accuracy resulting from the calibration function.

Marcel

   

Reply