NavList:

The modern angular choice of tenths of a minute of arc for celestial navigation was a 20th century compromise. It was --and is-- a "funny" way of specifying angles: a little bit sexagesimal and a little bit decimal!

Celestial navigation is near its systematic accuracy limit at one (whole) minute of arc. Using manual sextant observations and a sea horizon, we can determine positions to the nearest mile or slightly better under excellent conditions, and, in fact, nearest two or three miles is common and nearest five miles is sometimes as good as it gets. Those "nearest mile" expectations should imply that we need our input data only to the nearest whole minute of arc. Or if we're using decimal degrees, we only need the nearest two digits past the decimal point, which is actually slightly better than a whole minute of arc since 0.01° is about half a mile (0.01° = 0.6').

We could leave it at that, and many navigators do so, but there's always that nagging feeling that we're losing something to "round-off" error, so it became customary to tabulate and work celestial navigation numbers to the nearest tenth of a minute of arc in the middle of the 20th century. Those "tenths" give us that little extra bit of computational "safety", at least in our heads. For example, imagine a noon Sun sight... If I add a ZD of 25°20.4' and a Dec of 12°11.4' to get latitude at local noon, then the sum is 37°31.8' ...or nearly 37°32'. But if my ZD and Dec had been rounded to whole minutes of arc before I did the addition, my result would have been 37°31'. Oh, the horror!! Of course that panic is irrational since the systematic process is +/- one mile anyway, and also those details often cancel out instead of adding up. Still... don't we worry?? Tenths are not necessary, but they are a balm for that nagging feeling...

Tenths also suggested future improvement. There are indications of a belief in the mid-20th century, especially in USN circles, that celestial navigation could be more accurate if navigators would just try a little harder, use the best sextants, and chase all those tenths. This was a fantasy for manual celestial navigation, but chasing details did become important when machines began taking sights. The automated systems that are found in high-altitude military aircraft (apparently... because details are "classified") can get celestial position fixes accurate to a fraction of a second of arc. But that's far outside our concerns here.

Before the invention and adoption of double-reflecting navigation instruments c.1735-1750, there was little reason to seek more than a half a degree accuracy. This was the end of, what I call, the pre-scientific era of celestial navigation (or "nautical astronomy" as it was known). The available almanac data barely supported any higher level of accuracy. After c.1735, those early "reflecting quadrants" (octants, sextants) could measure angles to a minute of arc precision or a little better, and it was hoped that this would make "lunars" possible, but none of the other data were up to par yet, except the Declination of the Sun, and even that was only just good enough. There were no useful coordinates for stars of sufficient accuracy, no almanac data for the planets that came close enough to navigation quality, and of course only crude tables of the Moon's motion. But by 1750, a tenth of a degree was reasonable, and instruments were readable to the nearest minute of arc.

With the release of data for stars, followed a few decades later by better Sun tables and especially the Moon tables developed by Tobias Mayer, celestial navigation entered its scientific era. From c.1767 (first year of the "Nautical Almanac & Astronomical Ephemeris") navigators could begin to worry about fractions of a minute of arc. Sextants appeared with finely engraved scales and verniers. Some of their calculations jumped in precision from minutes of arc to seconds of arc. This seems to imply much higher accuracy (it's a factor of 60, after all), but this is an illusion. They jumped to seconds of arc because that was how the numbering system for angles worked: first degrees, then minutes, then seconds, then thirds (thirds were never significant and finally banished by the early 19th century). Thus even in the 1890s, navigators would calculate the Sun's polar distance for their time sight work carried out to seconds of arc. This was "ritual" calculation precision, unjustified by the data, but that's what they did.

So what do we make of Bligh's work in that little latitude calculation in January of 1788? Bounty was at anchor off Tenerife in the Canary Islands near the beginning of the long voyage to Tahiti and the well-known drama that would follow. Bligh recorded his computations for latitude with every detail, each step listed to the nearest second of arc! Does this prove that his work was accurate to that level? No, of course not. His sextants were not magic (they had several aboard Bounty). It does tell us something about William Bligh though... He was having a little fun on a sunny afternoon. He was presumably getting some degree of pleasure from working those numbers in such detail. And he probably hoped that his detail might nail down the latitude of Tenerife better than any navigator before him had ever done. This was a common conceit among navigators in this period. Bligh had great experience in nautical astronomy already having sailed on Cook's first Pacific voyage, and he was skilled in lunars. He knew how to do his work to seconds of arc, and he did the calculations correctly. No doubt --no doubt at all, in my opinion-- he took pride in all those little seconds. What he demonstrates in that obsessively detailed computation most of all... is that he probably would have joined the NavList community if he were alive today... :)

You also asked:
"Weren't charts (often or always) in degrees-minutes-seconds?"

First, charts were rare. But tables of important locations in navigation manuals did sometimes list key points with coordinates that included seconds of arc. Only a foolish navigator would have trusted those seconds for navigation before the late twentieth century. Most tables of positions and charts were accurate to a mile.

You asked about bubble sextants, too. I hope some of the air navigators will describe the reading of those instruments. My own experience is limited, but I will say that 20th century bubble sextants were radically different in design from marine sextants and the "readouts" were more like the display of an early digital clock. You would turn the knob and typically look through a little window to read the degrees and minutes. Reading was "easy" and precision was limited to whole minutes of arc with expectations of nearest five minutes normally. Given the much lower intrinsic accuracy of bubble sextant observations (except in "laboratory" conditions), seconds of arc from a bubble sextant never came up in real practice.

Frank Reed
Clockwork Mapping / ReedNavigation.com
Conanicut Island USA