Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Name or NavList Code:
    Re: Round-off
    From: Frank Reed
    Date: 2009 May 22, 12:33 -0700

    John, you wrote:
    "But remember, that's only one of many different probability estimates.  The max error is still L*N/2."
    You're right that there are many different ways of describing probability 
    limits and estimating the odds of extreme outcomes. But the key is that you 
    should always compare apples with apples. If we're looking at ONE standard 
    deviation error in one variable, then we should never be comparing that to a 
    TWO standard deviation measure of error, or yet some other measure of error 
    (like max error), in another variable. 
    So what about that maximum potential error in round-off? If we're dropping 
    tenths, each round-off could be as large as 0.5 minutes of arc, and if there 
    are a dozen round-offs, then that might lead to an error as large as +6.0 or 
    -6.0, equal to the L*N/2 you gave. But this just isn't relevant in practice 
    because we have no means of distinguishing round-off error from observational 
    error (except, of course, by the obvious approach of running the calculation 
    again without rounding off). To put it differently, we already should 
    consider the possibility, rather low in probability, that our observations 
    are off by +/-6.0 minutes of arc, and unlike round-off error, this is not an 
    upper limit. Then we should ask whether the probability of being that far off 
    is significantly increased by the possibility of round-off error. And that 
    all depends on how the standard deviation of observational error compares 
    with the standard deviation of round-off error. As noted in my post a little 
    earlier today, if the standard deviation of the observational error is just a 
    little above 1 minute of arc, the increase in the net standard deviation from 
    rounding off (dropping the tenths) is not that great, less than 33%. It is 
    nearly identical to assuming that your observational error is moderately 
    higher and the round-off error is zero. 
    Navigation List archive: www.fer3.com/arc
    To post, email NavList@fer3.com
    To , email NavList-@fer3.com

    Browse Files

    Drop Files


    What is NavList?

    Get a NavList ID Code

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    A NavList ID Code guarantees your identity in NavList posts and allows faster posting of messages.

    Retrieve a NavList ID Code

    Enter the email address associated with your NavList messages. Your NavList code will be emailed to you immediately.

    Email Settings

    NavList ID Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site