NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
Re: Round-off
From: Frank Reed
Date: 2009 May 14, 13:23 -0700
From: Frank Reed
Date: 2009 May 14, 13:23 -0700
Peter, you wrote: "If the round-off errors are a type of a random walk, then we should expect the error in the final result accumulate only with the square-root of the number of steps, times 0.05'." That's right except for one little detail. Round-off error is a random number taken from a uniform sample distribution. What you want for the step size if not the maximum size of the round-off error, but the standard deviation of that. You'll find that this amounts to 1/sqrt(12) times the width of the round-off band. So if you have a dozen numbers added up where you've dropped the tenths, your expected error is only ONE tenth. Lars B. wrote a nice post about this a while back: http://fer3.com/arc/m2.aspx?y=200610&i=101438 -FER --~--~---------~--~----~------------~-------~--~----~ Navigation List archive: www.fer3.com/arc To post, email NavList@fer3.com To , email NavList-@fer3.com -~----------~----~----~----~------~----~------~--~---
