NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
Refraction - Comparing results
From: Marcel Tschudin
Date: 2005 Sep 7, 23:06 +0300
From: Marcel Tschudin
Date: 2005 Sep 7, 23:06 +0300
Refering a comparison of results on calculated refraction values, Frank replied: > Interest?? Yes, yes, yes. Can you tell us what you found out and how you > went about it, please. OK, here we go. It seems that USNO published apparently recently new refraction values. (That is at least what I understood from some communication done here.) The data for altitudes between 0deg and 3deg were also communicated here. The intention was to see how other results compare with these data, such as Bennett's formula or results from integration using the U.S. Standard Atmosphere or the the atmosphere models for summer and winter as described in U.S. Standard Atmosphere Supplements, 1966 for different latitudes. The calculations were done using the USNO standard conditions, i.e. pressure 1010hPa, temperature 10C (=283.15K), wavelength 501.69nm. The refraction values from USNO are for 80 percent humidity. Since the contribution of the humidity is almost neglectable, I calculated finally only the results for the Standard Atmosphere twice, once wet and once dry. All the other calculations were done dry. The radius of the earth was taken as 6378390m. In order to make the results compareble to nautical observations and experiences, a height of 10m (a.s.l.) has been chosen. (The values from USNO and Bennett are scaled to the slightly different conditions at 10m height than those of the standard conditions at sea level.) For an altitude of 0 deg (slightly above the horizon) the following results were obtained (all values in arc min.): USNO (wet): 33.75 Bennett: 34.43 U.S. Standard Atmosphere (wet): 33.96 U.S. Standard Atmosphere (dry): 34.01 Average for January (all latitudes): 36.11 Average for July (all latitudes): 34.54 January detailed: 15 deg lat: 34.79 30 deg lat: 35.56 45 deg lat: 35.43 60 deg lat: 37.25 75 deg lat: 37.50 July detailed: 15 deg lat: 34.08 30 deg lat: 34.09 45 deg lat: 34.29 60 deg lat: 34.66 75 deg lat: 35.56 For an observer at an elevated location, the altitude of the horizon depends on the atmosphere itself, it varies with the dip. In order to calculate the value of the dip a regula falsi iterative procedure was implemented, which, depending of the visibility of the light ray (light ray below or above the radius of the earth) converges to the location of the horizon. Using this procedure the dip and the refraction value at the dip could be calculated. The calculated dip and the refraction at the dip allowed now also to compare the factor in Bowditch's formula for calculating the refraction at the dip, i.e. the value of 1.76 in the formula Refr in arc min. = 1.76 * SQRT(of observer's height in m) The refraction at the horizon alows also to calculate the factor K used for the calculation of terrestrial refraction. K expresses the bending of the light ray in multiples of earth radii. K is also hidden in the Bowditch factor. The results at the horizon are as follows: Refraction ----------- USNO (wet): not available Bennett: 35.64 (interpolated for a dip of -0.092 arc deg U.S. Standard Atmosphere (wet): 35.23 U.S. Standard Atmosphere (dry): 35.23 Average for January (all latitudes): 37.58 Average for July (all latitudes): 35.81 January detailed: 15 deg lat: 36.09 30 deg lat: 36.97 45 deg lat: 36.80 60 deg lat: 36.88 75 deg lat: 39.16 July detailed: 15 deg lat: 35.31 30 deg lat: 35.32 45 deg lat: 35.53 60 deg lat: 35.94 75 deg lat: 36.96 Altitude at horizon in deg arc (Dip) ----------------------------------- USNO (wet): not available Bennett: not available U.S. Standard Atmosphere (wet): -0.0921 U.S. Standard Atmosphere (dry): -0.0921 Average for January (all latitudes): -0.0903 Average for July (all latitudes): -0.0917 January detailed: 15 deg lat: -0.0915 30 deg lat: -0.0907 45 deg lat: -0.0910 60 deg lat: -0.0892 75 deg lat: -0.0890 July detailed: 15 deg lat: -0.0920 30 deg lat: -0.0920 45 deg lat: -0.0919 60 deg lat: -0.0916 75 deg lat: -0.0908 Dip-factor (Bowditch = 1.76) ------------------------------ USNO (wet): not available Bennett: not available U.S. Standard Atmosphere (wet): 1.747 U.S. Standard Atmosphere (dry): 1.747 Average for January (all latitudes): 1.713 Average for July (all latitudes): 1.739 January detailed: 15 deg lat: 1.735 30 deg lat: 1.722 45 deg lat: 1.726 60 deg lat: 1.692 75 deg lat: 1.689 July detailed: 15 deg lat: 1.746 30 deg lat: 1.746 45 deg lat: 1.743 60 deg lat: 1.738 75 deg lat: 1.723 K-factor --------- USNO (wet): not available Bennett: not available U.S. Standard Atmosphere (wet): 5.673 U.S. Standard Atmosphere (dry): 5.673 Average for January (all latitudes): 4.839 Average for July (all latitudes): 5.453 January detailed: 15 deg lat: 5.337 30 deg lat: 5.000 45 deg lat: 5.110 60 deg lat: 4.402 75 deg lat: 4.345 July detailed: 15 deg lat: 5.637 30 deg lat: 5.633 45 deg lat: 5.560 60 deg lat: 5.411 75 deg lat: 5.023 Some general observations of the results: The refraction values fom USNO are by tendency smaller than the others. Bennetts Formula seems to fit best to the stable atmospheric conditions of July. It seems to be advisable to increase slightly the refraction values from standard tables or formulae in winter and at higher latitudes. If the observed differences are in a size where they could or should be considered in nautical navigation, I suggest that 1) the calculations are redone and confirmed by an other person (Frank?) 2) the atmospheric models of 1966 are veryfied, this e.g. by statistical analyses of balloon data, other measurements of the atmosphere and by comparing the values with results of actual observations over a longer time periode. All input data and the results obtained are compiled in an Excel file. If someone should be interested in it, then please let me know it. It is a way of saying thankyou for all the help I received here. Marcel