NavList:

Gary, you wrote:
"But, give that 100 meter number (only one significant figure) to a guy and hand him a calculator and he converts it to 325 feet and, apparently a different guy, comes up with 328 feet! "

Yeah, this happens all the time in the literature of popular astronomy. There ought to be a "style manual" entry on this somewhere. It's not a particularly complicated matter, and yet we see these endless extra digits all the time. Some of the older publications do clearly have style advice for their authors. For example, in the pages and web site of Sky & Telescope, one of the oldest publications in popular science of any sort, you will typically see 100 meters translated for us imperialists (imperial units users, that is) as 300 feet.

Normally, this conversion issue comes up in the meters to feet department. That's common. But I especially liked your first example in this thread where the error was made in the opposite direction.

Now for an interesting issue... If I say that asteroid A is about 30km in diameter and asteroid B is 50km in diameter, then I can safely assume that these numbers are accurate to one significant digit and then convert to miles with an accuracy of one significant digit so 30km converts to 20 miles and 50km converts to 30 miles (what would happen to 40km, btw?). There's a small chance that the figures are accurate to two sig figs, but not likely. But now consider another case. If I say that asteroid A has a diameter of "about 10km" and asteroid B has a diameter of "about 1km", how many significant figures do we have? These are very common estimates where the size is only estimated to the nearest order of magnitude which means we only have "half" a significant figure. What THEN would be the correct conversion to miles?? Converting 10km to 6 miles and 1km to 0.6 miles implies a higher accuracy than the original numbers --we've gone from half a significant figure to one significant figure. In this case, converting to "about 5 miles" and "about half a mile" (spelled out that way) would be closer to the original estimates' meaning. Naturally the only safe solution is to attach some sort of error bars to any estimated number.

And yes, this does have significance for a celestial navigation issue that has come up again and again in NavList messages: any position fix should include an estimate of the size and shape of the error ellipse. Also, with GPS positions or positions derived from digital products like Google Maps, it's all too common to see latitudes and longitudes quoted to a thousandth of a second of arc --meaningless digits.

-FER

----------------------------------------------------------------
NavList message boards and member settings: www.fer3.com/NavList
Members may optionally receive posts by email.
To cancel email delivery, send a message to NoMail[at]fer3.com
----------------------------------------------------------------