Still studying navigation and am trying to consolidate my knowledge of the navigational triangle. Current interest in particular is how to establish the distance of the unknown side of the PZX triangle. From the assumed position of the ship (Z) to the geographical location of a star (X).

Lets throw in a couple of positions to work with:

ASSUMED POSITION Z
54 degrees 13'.45N
003 degrees 14'.62E

POSITION OF STAR X
47 degrees 17'.95N
019 degrees 37'.82W

I am familiar with the following formula as a result of help received on Jiskha, thanks Damon.

cosp = cosx x cosz + sinx x sinz x cosP

and a minute or so on the calculator will yield an answer.

I have now become curious about another formula I have come across to calculate the same information and need a lot of help AGAIN! applying this new formula.

New formula as follows:

Distance=60.ACOS(sin(latX)x sin(latZ)+cos(latX)x cos(latZ)x cos(longX - longZ))

Too many brackets for my liking. Obviously if both formulas are for the same purpose they should yield the same answer working with positions given above.

Please can someone have a go at the calculations using both formulas to see if it is possible to achieve the same answers.

I would also be interested to learn of any other formulas suitable for calculating distances between 2 positions on earth.

Thanks a million

Mike

4 answers

The first one is the formula I used yesterday and agreed with your answer. I also checked it against the online navigation calculators.

However I remember you did something like take 90 - the latitude before you ran your formula.

If you did that, then since cos (90-x) = sin (x) and sin(90-x) = cos (x)
the two formulas would be the same.

The 60 is to get nautical miles instead of degrees.

For the data you gave above the online software gives 956.7 nm

I will calculate roughly using your data and the second formula since I did not use that yesterday. I am dividing minutes by 60 to get fractional degrees for my calculator which does not do base 60 arithmetic.:
sin Lat X = sin 47.299 = .7349
sin Lat Z = sin 54.224 = .8113

cos Lat X = .6782
cos Lat Z = .5846

cos (long X -long Z) = cos (19.630+3.244) = cos(22.874) = .9214
note: The Plus sign here is because the Z longitude is East - - =+
so
cos^-1[ .7349 * .8113 + .6782 * .5846 * .9214 ]

cos^-1 [ .5962 + .3653 ]

cos^-1 [ .9615 ]

15.95 degrees

times 60 = 957 nm

So the second equation agrees with the online software navigator
Go to wikipedia on the subject of great circle navigation. They have several of the equations, the one you just gave being on the top. They give examples of where one might be accurate and another inaccurate.
Here is an excerpt from the online aviation navigation primer. If it prints here, it also shows your second formula as first choice and the haversine formula second:

Some great circle formulae:
Distance between points

The great circle distance d between two points with coordinates {lat1,lon1} and {lat2,lon2} is given by:

d=acos(sin(lat1)*sin(lat2)+cos(lat1)*cos(lat2)*cos(lon1-lon2))

A mathematically equivalent formula, which is less subject to rounding error for short distances is:

d=2*asin(sqrt((sin((lat1-lat2)/2))^2 +
cos(lat1)*cos(lat2)*(sin((lon1-lon2)/2))^2))
No one is listing the one I had from my old books and I can not get an accurate result from it on your second problem. I suggest using the ones I am seeing in the present day texts.
The Wikipedia article on marine navigation and the aviation source agree on the
cos^-1 [sin sin + cos cos cos] form, so I would stick to that.
They also agree on the haversine form, but I do not think you really need that unless just for fun. You can always carry more places with your calculator if you want more accuracy for special cases.
By the way, the online calculators allow you to use different earth models. We are assuming a perfect sphere as have navigators for centuries. However greater accuracy is possible using more accurate models for the geometry of earth, but no way you are going to figure them out with a pocket calculator.