The forecast that missed by more than the lake has ever moved
Showing the misleading chart
A harbour planning note we drew fits a least-squares line to eight annual mean water levels at a Lake Huron gauge — 2013 to 2020, every observation plotted, linear unbroken axes, nothing smoothed — and carries it forward in the same ink to 2035, where it reads 180.0 m. Five years of data have since arrived. The line said 178.23 m for 2025 and the gauge read 176.29 m: out by 1.94 m, which is wider than the entire 1918–2025 range of annual means at that gauge, and below the fit’s own 95% prediction interval in all five years. The window is a knob as well — 2013–2020 is the steepest of the 101 eight-year windows in the record, and running the same recipe on every one of them projects a value fifteen years out that spans 6.49 m, with 63% of them landing somewhere the lake has never been. The best-fitting window of all, R² = 0.984, points down.
01The claim
Eight annual means, one line, every observation on the page: the mean of the twelve published monthly mean water levels for each year at the Harbor Beach gauge on Lake Huron, in metres above the International Great Lakes Datum of 1985. Nothing is smoothed, interpolated, indexed or rebased. No year has been dropped and none is missing. The years are plotted where they fall, the axes are linear, evenly ticked and unbroken, and the trend is an ordinary least-squares line through all eight points, none excluded and none weighted — a rise of 17.8 cm a year, R² = 0.944, residual standard deviation 11.4 cm. The lake stood at 175.914 m in 2013 and 177.326 m in 2020, a rise of 1.413 m in seven years. Carried forward at the same rate, the annual mean reaches 178.23 m in 2025, 179.12 m in 2030, 180.01 m in 2035 and 180.90 m in 2040. Read-out for the board: the 2035 level stands 4.01 m above chart datum, against 1.33 m in 2020. Raise the dock walls and the sheet-pile design elevation to the 2035 line, re-survey bridge and berth clearances, and defer the maintenance dredging programme — the channels gain 2.7 m of water by themselves.
02The trick
Every figure on that note is right, and the picture is fiction from 2020 onwards, because that is where the data stops and the line does not. The eight observations occupy the bottom-left corner; the rest of the panel — most of its width and nearly all of its drama — is one straight line drawn in the same colour, at the same weight, with nothing at the boundary to say that the part on the left was measured and the part on the right was computed. Five years have since arrived to settle it. The line said 178.23 m for 2025; the gauge read 176.29 m. That is a miss of 1.94 m, and the useful way to size it is against the series itself: the highest annual mean in the 108 complete years from 1918 is 177.326 m, in 2020, and the lowest is 175.676 m, in 1964, so the whole range this gauge has ever occupied is 1.650 m. The five-year error is wider than the entire record. It is also the largest five-year miss that any eight-year window in the record produces, and every one of the five years since 2020 has fallen below the fit’s own 95% prediction interval — an interval that is itself too narrow, because least squares assumes each observation is an independent draw and this record is nothing of the kind: the annual means carry a lag-1 autocorrelation of 0.845, and still 0.83 once a linear trend is taken out of the whole century, a wet year following a wet year, and serial correlation like that shrinks the computed standard errors below what the data supports. The band being narrow is the smaller half of it, though. The larger half is that there is no trend here to project, so no width of interval around this line would have contained the answer. The projection to 2035, 180.01 m, sits 2.68 m above anything the lake has done since 1918, and its whole 95% band lies above the record. Then there is the window, which is the second knob and the quieter one. There are 101 eight-year windows in this record, and 2013–2020 is the steepest of them. Fit each one the same way and carry each fifteen years past its last observation, and the answers run from 173.51 m to 180.01 m — a span of 6.49 m, nearly four times the 1.65 m the lake has actually moved between its record years — with 34 landing above the record high, 30 below the record low and only 37 inside the range the lake has ever occupied. Across the 86 windows whose target year has since arrived, the median miss is 1.35 m. And R², the statistic doing the persuading, measures none of this: it says how close the eight fitted points sit to the line through them, computed from those eight alone. The best-fitting eight-year window anywhere in this record is 1952–1959, at R² = 0.984 — tidier than the note’s 0.944 — and it points the other way, giving 173.51 m for 1974 where the gauge read 177.08 m, a miss of 3.57 m. What the series actually does is oscillate, and always has: the 1.413 m rise of 2013–2020 is the largest seven-year rise in the record, but 1964–1971 gave +1.128 m and 1952–1959 gave −1.079 m, and by 2025 the lake had given back 1.035 m of the rise the note extended. (Both drawings are ours; the board, the read-out and the recommendation on the first are invented, and every measurement on both is real.)
03The fix
Stop the ink where the data stops. Past the last observation the mark should change — dashed, paler, on its own labelled band, with a rule and a word at the boundary — so that a reader glancing at the picture can see which part was measured before reading a syllable. On the redraw the eight fitted years are a gold segment inside a marked column, the projection is a red dashed line, and the boundary between them is 2020. Then say what the projection is made of, in the caption, the way you print a unit: the window that was fitted, the shape that was assumed, and the horizon. Draw the interval, widening with distance, and say what it assumes — here it is drawn, it already failed in all five years, and it is still too narrow, which is more useful for a reader to know than a clean line with no band at all. The cheapest honest check is to run the same recipe on other windows and put the results together: all 101 of this record’s eight-year windows, projected fifteen years, fan across 6.49 m, and that fan is the answer the single line was never going to give. Ask what the quantity is allowed to do, too, because the mechanism is usually known before the chart is drawn — a lake level is set by precipitation, evaporation and outflow that have oscillated within about a metre and a half for a century, and a straight line has no way to do that. Prefer a projection with a mechanism behind it to one with a slope behind it, and where a published forecast already exists, use it and credit it rather than fitting your own. Then keep the two claims apart in the words as well as in the ink: what a series has done between two named dates is a measurement and survives every redraw, while what it will do next is a model — so where a decision turns on the second, quote the range rather than the line, and say whose model it came from. The harbour question underneath is a good example of why it matters in both directions, since the note’s recommendation to defer dredging and raise the walls was written off a rise that had already begun to reverse before the ink was dry.