NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
Re: Rounding decimal fractions
From: Stan K
Date: 2013 Jul 20, 20:38 -0400
From: Stan K
Date: 2013 Jul 20, 20:38 -0400
Herbert,
Is this an example of what you call "double rounding"?
The Power Squadrons has students display azimuth angle, Z, to tenths of degrees, but when it is converted to azimuth, Zn, only to whole degrees. For example, if Z = N 49.51000º W, this would be displayed as Z = N 49.5º W. Using this rounded value to calculate Zn you would get Zn = 360º - 49.5º = 310.5º which would round to 311º (USPS rounding). However, using the unrounded value of Z to calculate Zn to whole degrees you would get Zn = 360º - 49.51000º = 310.49º, which would round to 310º (USPS rounding).
This always bothered me, and when I grade an exam where Z is xxx.5º, I give full credit for either possible answer.
Stan
Is this an example of what you call "double rounding"?
The Power Squadrons has students display azimuth angle, Z, to tenths of degrees, but when it is converted to azimuth, Zn, only to whole degrees. For example, if Z = N 49.51000º W, this would be displayed as Z = N 49.5º W. Using this rounded value to calculate Zn you would get Zn = 360º - 49.5º = 310.5º which would round to 311º (USPS rounding). However, using the unrounded value of Z to calculate Zn to whole degrees you would get Zn = 360º - 49.51000º = 310.49º, which would round to 310º (USPS rounding).
This always bothered me, and when I grade an exam where Z is xxx.5º, I give full credit for either possible answer.
Stan
-----Original Message-----
From: Herbert Prinz <herbert@heavenlysoftware.org>
To: slk1000 <slk1000@aol.com>
Sent: Fri, Jul 19, 2013 1:25 pm
Subject: [NavList] Re: Rounding decimal fractions
----------------------------------------------------------------
NavList message boards and member settings: www.fer3.com/NavList
Members may optionally receive posts by email.
To cancel email delivery, send a message to NoMail[at]fer3.com
----------------------------------------------------------------
From: Herbert Prinz <herbert@heavenlysoftware.org>
To: slk1000 <slk1000@aol.com>
Sent: Fri, Jul 19, 2013 1:25 pm
Subject: [NavList] Re: Rounding decimal fractions
Paul,
You say
"I have to admit I never pay attention to these fine points in my own
software. Since Tinyac and Lunar3 both have selectable precision, I
assume the user will use, say, .01 precision if "tenths" are critical."
software. Since Tinyac and Lunar3 both have selectable precision, I
assume the user will use, say, .01 precision if "tenths" are critical."
This may be beneficial in some cases, but it also has the potential of aggravating an error through the effect of double rounding when a user selects .01 precision at first ("to be safe", or "because it does not cost anything") and then finally decides to round the result to 0.1 precision because that is what he really needs.
Herbert Prinz
----------------------------------------------------------------
NavList message boards and member settings: www.fer3.com/NavList
Members may optionally receive posts by email.
To cancel email delivery, send a message to NoMail[at]fer3.com
----------------------------------------------------------------
