NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
Re: Rounding decimal fractions
From: Stan K
Date: 2013 Jul 18, 16:12 -0400
From: Stan K
Date: 2013 Jul 18, 16:12 -0400
Paul,
"Round to even", also known as "banker's rounding", has, IMHO, no place other than a bank. However, the Nautical Almanac rounding scheme is neither what I called "standard" rounding (as taught by the Power Squadrons) nor banker's rounding. Banker's rounding makes sense when you are trying to balance the books where there are half cents involved, but I don't see it as being the proper thing to do in anything "scientific". What I call "standard" rounding has five values that round down and five that round up, i.e. half down and half up. That seems more sensible to me.
When I was writing Celestial Tools I was learning Visual Basic 6 from a book called "Visual Basic 6 from the Ground Up". In it the author writes about "The (New) Round Function", saying "One of the pains of previous versions of Visual Basic was the contortions you had to go through to round off a number. In VB6 all this pain is eliminated by using the nifty new Round function..." Nifty my butt. Half the time I could not get the same answer as I got doing a problem manually. After doing some research I found that the nifty new Round function used banker's rounding, so I had to go back to doing the contortions (simply making my own function).
I have seen other programs (Excel perhaps?) that give the user the choice of the kind of rounding to use, which seems to be the way to go. I guess the .NET Math.Round class also gives the programmer options other than "round to even".
Celestial Tools is intended to do problems as they would be done by Power Squadrons students, within its accuracy limitations. My "tweaked" Increments and Corrections rounding routine, with the seven hard-coded corrections that do not fit in, seem to reproduce the "modern" Nautical Almanac Increments and Corrections tables accurately. I say the "modern" tables because, as we have recently discussed, there are four correction values that changed in 2002. (To me, that is the most mind-blowing part of this entire discussion.) I have chosen to leave Celestial Tools that way, with a note about the change in the Help.
Stan
"Round to even", also known as "banker's rounding", has, IMHO, no place other than a bank. However, the Nautical Almanac rounding scheme is neither what I called "standard" rounding (as taught by the Power Squadrons) nor banker's rounding. Banker's rounding makes sense when you are trying to balance the books where there are half cents involved, but I don't see it as being the proper thing to do in anything "scientific". What I call "standard" rounding has five values that round down and five that round up, i.e. half down and half up. That seems more sensible to me.
When I was writing Celestial Tools I was learning Visual Basic 6 from a book called "Visual Basic 6 from the Ground Up". In it the author writes about "The (New) Round Function", saying "One of the pains of previous versions of Visual Basic was the contortions you had to go through to round off a number. In VB6 all this pain is eliminated by using the nifty new Round function..." Nifty my butt. Half the time I could not get the same answer as I got doing a problem manually. After doing some research I found that the nifty new Round function used banker's rounding, so I had to go back to doing the contortions (simply making my own function).
I have seen other programs (Excel perhaps?) that give the user the choice of the kind of rounding to use, which seems to be the way to go. I guess the .NET Math.Round class also gives the programmer options other than "round to even".
Celestial Tools is intended to do problems as they would be done by Power Squadrons students, within its accuracy limitations. My "tweaked" Increments and Corrections rounding routine, with the seven hard-coded corrections that do not fit in, seem to reproduce the "modern" Nautical Almanac Increments and Corrections tables accurately. I say the "modern" tables because, as we have recently discussed, there are four correction values that changed in 2002. (To me, that is the most mind-blowing part of this entire discussion.) I have chosen to leave Celestial Tools that way, with a note about the change in the Help.
Stan
-----Original Message-----
From: Paul Hirose <cfuhb-acdgw@earthlink.net>
To: slk1000 <slk1000@aol.com>
Sent: Thu, Jul 18, 2013 3:33 pm
Subject: [NavList] Rounding decimal fractions
From: Paul Hirose <cfuhb-acdgw@earthlink.net>
To: slk1000 <slk1000@aol.com>
Sent: Thu, Jul 18, 2013 3:33 pm
Subject: [NavList] Rounding decimal fractions
Stan K wrote: > A couple of years ago a friend of mine and I were discussing how to implement Increments and Corrections in my Celestial Tools program and his Navigation Calculator Workbook spreadsheet. Looking at the output of his spreadsheet, we noticed that the Almanac values apparently did not use what we call "standard" rounding, in this case, for instance, 0.00 through 0.04 would round down to 0.0 and 0.05 through 0.09 would round up to 0.1. I thought the convention, when a decimal can be rounded up or down with equal accuracy, is "round to even." E.g., 1.05 rounded to the nearest tenth is 1.0, but 1.15 is 1.2. But I see that my HP 49G calculator rounds up in these borderline cases. With 2 decimal point precision selected, 1/8 = .13, 3/8 = .38, 5/8 = .63, 7/8 = .88. The composite formatting feature in Microsoft's .NET Framework does the same thing as the calculator. But I discovered the .NET Math.Round class gives the programmer a "round to even" option. Results from a test program: 0.125 0.13 0.12 0.375 0.38 0.38 0.625 0.63 0.62 0.875 0.88 0.88 The first column is the fraction to full accuracy. In the second and third columns I used composite formatting with precision of two decimal places. The difference is that in the third column the value was first rounded with Math.Round. The sum of the second column is a little high (1/8 + 3/8 + 5/8 + 7/8 = 2 exactly) because rounding all borderline cases up introduces a small systematic error. I have to admit I never pay attention to these fine points in my own software. Since Tinyac and Lunar3 both have selectable precision, I assume the user will use, say, .01 precision if "tenths" are critical. --
