NavList:

You are correct in that Melville was on a whaler and according to wikipedia, also spent time on a merchant ship.   The indication was ~ 5 years at sea.  At no point was he ever a ship's officer, nor did he hold any licenses.  You cannot write about a technical subject if you were never exposed to it.

I agree with you that the navigation suite was off limits to common sailors.

As to the navigation itself.  The novel is set in the 1830's or 1840's, with the novel itself written in 1850-1.  The navigation would likely not have included lunars, as the era was mainly over by 1850.  For the enthusiasts, it is possible to include lunars if we assume Ahab to be an old school navigator.  The possibility also exists that Ahab was using a chronometer.  Whaling was a profitable industry but chronometers were expensive.  One can assume a wealthy ship owner furnishing such a device or pecuniously withholding one as an unnecessary expense.  It is a novel after all.  I think it most likely that the navigation would be a regime of latitude by meridian crossing and longitude by account (dead reckoning).  The Ethiopics (sailing directions) of the time all had tabular lists of features, like harbors, along with their latitude and longitude.  As does Bowditch and others.  The whalers were chasing whales, not navigating to minimize time, distance and cost.  

In our 'modern' era, it seems crazy to sail the oceans without a crisp idea of exactly where we are.  Yet ships crossed oceans well before the longitude problem was resolved.  Yes, it more efficient and much safer with knowledge of longitude, but it is possible without it.

Brad