Great-Circle Distance

Great-circle distance is the shortest path between two points on a sphere — the arc of the unique great circle through them. On the WGS-84 ellipsoid the analogous shortest path is the geodesic, computed by Vincenty's (1975) or Karney's (2013) algorithm; it differs from the spherical haversine by up to 0.5%.

The great-circle distance is the foundation of every long-haul navigation calculation and the answer to questions like "how far is it from here to there." Yet on a curved Earth the answer is non-trivial: there are multiple defensible formulas, multiple reference surfaces (sphere versus ellipsoid), and multiple non-obvious choices (which surface? at which epoch? for what use case?). This article runs the math, compares the haversine, Vincenty and Karney algorithms with worked numbers, explains when great-circle differs from rhumb-line, and shows why Vincenty-on-WGS-84 is the modern operational default. The companion supports /learn/haversine-formula-explained, /learn/vincenty-formula-explained, and /learn/great-circle-vs-rhumb-line go deeper.

Great circle, geodesic, rhumb line: three different things

The "shortest path" question has three plausible answers depending on whether the Earth is treated as a sphere or an ellipsoid and whether the path is constrained to a constant compass bearing.

On the sphere, the great-circle is the shortest path. On the ellipsoid, the geodesic is the shortest path; it differs from a "great circle on the average sphere" by up to 0.5%. The rhumb-line is never the shortest path between two distant points (except along a meridian or along the equator) — it appears straight on a Mercator chart, but it spirals around the globe and is significantly longer than the geodesic for long routes.

The haversine formula: spherical great-circle distance

The haversine formula gives the great-circle distance between two points on a sphere of radius R, using the latitude (φ) and longitude (λ) of each point:

a = sin²((φ₂ − φ₁)/2) + cos(φ₁) × cos(φ₂) × sin²((λ₂ − λ₁)/2)
c = 2 × atan2(√a, √(1 − a))
d = R × c

The formula is numerically stable for small distances (unlike the spherical-cosine formula, which loses precision when the distance is small) and computationally cheap (six trig calls, one square root).

Most web-mapping APIs, fitness trackers, and "distance between two GPS points" implementations use haversine because it's fast and the ~0.5% error is invisible at consumer scale (a 5 km hike is within 25 m of the truth). For scientific or operational navigation, Vincenty or Karney is required.

Vincenty's formula: ellipsoidal geodesic

Vincenty (1975) published an iterative algorithm that solves the inverse geodesic problem on an ellipsoid — given two latitudes and longitudes, return the geodesic distance and initial/final bearings. The algorithm uses nested loops with corrections for ellipsoidal flattening and converges to sub-millimetre accuracy in 1-3 iterations for non-antipodal pairs.

Karney (2013) published a closed-form series that supersedes Vincenty for new code: it handles all cases including nearly antipodal points (where Vincenty fails to converge), runs faster, and reaches sub-nanometre accuracy. The PROJ library and most modern geodesy software use Karney internally.

The JFK-to-London worked example

A standard worked example: the great-circle distance from John F. Kennedy International Airport, New York to London Heathrow.

A long-haul JFK−LHR flight follows close to the geodesic route — New York to London by great circle passes over Greenland and crosses the North Atlantic at ~58° N (well north of the rhumb-line course at ~45° N). The geodesic saves about 1,655 km (~30%) over the rhumb-line; in fuel terms, that's ~10 tonnes of jet fuel per flight.

Why flight paths look curved on web maps

A great-circle route is straight on the globe but curved on most flat-map projections. The "curve" is the projection's response to flattening a 3D arc onto a plane.

The gnomonic projection has the special property that all great circles are straight lines — useful for plotting initial trans-oceanic routes. Marine and aviation practice plots the great circle on a gnomonic chart, reads off waypoints at convenient intervals (every 5° or 10° of longitude), then sails or flies short rhumb-line legs between waypoints on a Mercator chart.

When the formula choice matters

The Coordinately distance calculator uses Vincenty on WGS-84 by default, with haversine available as a comparison option. For the JFK-LHR example, the difference is ~30 km — visible in the answer.

Common misconceptions

Related pillars

The other seven pillar concepts on Coordinately:

• What is latitude and longitude? — the angular coordinates a distance is computed between
• Coordinate formats explained — format choice affects whether distance is direct or computed
• How GPS works — produces the coordinates these formulas consume
• What is a map projection? — why the great-circle path looks curved on most maps
• What is a geodetic datum? — the WGS-84 ellipsoid Vincenty and Karney run on
• Time zones explained — the time-difference half of "how far is it?"
• History of latitude and longitude — from Eratosthenes to Vincenty