Cherry-picking

The immature cohort

Every bar is a whole year. Only the old ones have had time to finish.

a.k.a. right-censoring · incomplete follow-up · cohort maturity · seasoning · vintage curves · loss development · the lag triangle · unequal exposure time · immature vintages · the unfinished bar

Some events do not arrive with the thing they belong to. A recall attaches to a model year built years earlier, a citation to a paper published a decade ago, a claim to the policy year it happened in, a default to the quarter the loan was written, a complaint to the product that shipped last spring. Group those events by the cohort they belong to and the chart looks like a trend, but it is not one measurement repeated: each bar is a count of everything that has arrived so far, and how long “so far” has been running is set by the date somebody pulled the data. The oldest cohort has been collecting for a decade and the newest for a few months, so the x-axis is doing two jobs at once — it is a category axis, and it is a clock running backwards. The damage is always the same shape and always in the same direction. The right-hand end of the chart falls away, the fall gets steeper towards the edge, and the shortfall reads as good news where the events are bad ones: fewer defects, fewer claims, fewer defaults, a safer product, a quieter year. Nothing has been excluded and no axis is bent. The newest bars are simply unfinished, and a bar has no way of saying so.

How to spot it

Ask how long each bar has been watched. Subtract the cohort’s start from the date the data was pulled; if that number changes along the x-axis, the x-axis is also a clock, and no two bars are the same measurement.
A decline that begins near the right-hand edge and steepens towards it. That is the shape of a censoring curve rather than of a trend — a real change in the world does not know where the file ends.
Ask whether the event can arrive late, and how late. Recalls, citations, claims, defaults, revisions, refunds, reviews, diagnoses, prosecutions and repairs all attach to a cohort long after the cohort closes, and the lag is usually measured in years.
Check the oldest cohort. If it is still gaining events, then no bar on the chart is final — and the young ones are barely started.
Count the same events by the date they happened instead of by the cohort they belong to. If that flow is flat while the per-cohort chart collapses, the collapse is the clock.
Redraw it next month. Any chart whose past values grow when you re-run it is censored; the only question left is by how much, and the newest bars carry nearly all of it.
Rates do not save you. Per thousand vehicles, per paper, per policy — the denominator is fixed early and the numerator goes on arriving for years, so a rate censors about as hard as a count does.
The vocabulary is a warning in itself: seasoning, maturity, vintage curves, loss development, lag triangles, right-censoring, time-to-event. Credit risk, insurance and epidemiology all named this, because all three have been bitten by it.

The fix

Give every cohort the same clock. Pick an observation window that all of them have completed — twelve months from the start of the model year, two calendar years, five years from publication — count only the events that arrived inside it, and say in the caption what the window is, the way you print a unit. The honest chart then stops earlier than the data appears to allow, and that is the feature rather than the cost: the missing bars are the finding, because the alternative is a bar that says “fewer” when it means “not yet”. Draw the development curves beside it, one line per cohort, cumulative against age, each stopping where the file does. It is the single most useful picture in this whole family — it shows the lag, shows how far each cohort has got, and makes the age-matched comparison something the reader can see rather than take on trust. Where an immature cohort has to appear, give it its own style and label it incomplete, or project it from the mature curves and call the projection a projection: actuaries have done this for a century with development factors, and a dashed bar with a stated method beats a solid one that is quietly wrong. Print the as-of date on the chart, since every number on it is “as at” something. And never take a percentage change between two cohorts of different age — not in a headline, not in a read-out, not in a footnote — because the difference in their ages is doing the arithmetic. Two neighbours are worth keeping it apart from, and the first of them is close enough to be worth a sentence. The unfinished period buckets events by when they happened, so only the last bin is short and the fix is to drop or mark that bin; this page buckets them by which cohort they belong to, so every bin is calendar-complete, the shortfall reaches back a decade rather than one bin, and no amount of waiting for the current period to end will settle it. Where a reporting lag is the whole story and the buckets are dates, that is the other page. And survivorship is about cases that left the dataset, where here nothing is missing at all: every cohort is present, and the young ones are showing a number that is still on its way up.

In the gallery

Dark navy briefing slide, masthead “Vehicle safety desk · fleet defect monitoring” on the left and “NHTSA Part 573 recall campaigns · by model year · file to 9 Sept 2026” on the right, headlined “Recalls on new model years have fallen by nearly three quarters.” Beneath the headline sits a bordered gold chip reading “Demonstration · drawn by us from published figures”. A standfirst says every figure is a straight count out of NHTSA’s own recalls flat file — the manufacturers’ Part 573 defect and noncompliance reports, one source, one field, one definition; that the axis is linear, zero-based, evenly ticked and unbroken; that every vehicle campaign in the file is counted and nothing is sampled, smoothed, indexed or rescaled, with every value printed above its own column; that NHTSA publishes the file and the bars are ours; and that the series peaks at 2,163 campaigns on model year 2019 and stands at 582 on model year 2026. The main panel, headed “Vehicle recall campaigns affecting each model year” and subtitled “Distinct Part 573 campaigns naming that model year · vehicles only · model years 2012 to 2026 · zero-based linear axis, evenly ticked, unbroken · every campaign in the file, nothing smoothed, dropped or rescaled”, is a column chart of fifteen gold columns on an axis running 0 to 2,500 in even steps of 500: 2012 at 1,809, 2013 at 1,913, 2014 at 1,969, 2015 at 2,102, 2016 at 2,085, 2017 at 2,039, 2018 at 2,059, 2019 at 2,163, 2020 at 2,081, 2021 at 2,121, 2022 at 2,012, 2023 at 1,715, 2024 at 1,421, 2025 at 1,090 and 2026 at 582. The shape is a long plateau across the middle of the chart and then a staircase down over the last four columns, the final one just over a quarter the height of the peak. A caption reads “Campaigns as filed · every vehicle campaign in the file · one scale throughout”. Five cards run down the right. “The fall, in one figure” prints 73% in large type over the words “fewer campaigns” and says model year 2019 is the peak of the series at 2,163 while model year 2026 stands at 582, that model year 2025 at 1,090 is already 49.6% below the peak, and that the three newest model years are the three smallest bars on the chart. “Read-out for the desk” says the series turns over after model year 2019 and falls away steeply, seven consecutive model years below the peak, ending 1,715, 1,421, 1,090 and 582 — and recommends treating the defect rate on new vehicles as falling, holding the campaign-management headcount flat into next year rather than growing it with the fleet, and moving the recall-readiness budget to the model years that still carry the volume, the 2015 to 2022 cohorts. “What one bar counts” explains that a bar is distinct NHTSA recall campaigns naming that model year, from manufacturers’ Part 573 defect and noncompliance reports, vehicle campaigns only so tires, equipment and child seats are out; that a campaign usually covers several model years, 2.76 on average over the campaigns that name one, and is counted once in each one it names, so a bar is campaigns rather than vehicles and the bars do not sum to the number of campaigns filed; and that a manufacturer has five working days to file once it decides a defect is safety-related, the date used here being the date NHTSA received the report. “Every value on the chart” tabulates all fifteen model years in two monospaced blocks with their campaign counts and the number of days each model year has been on file, from MY2012 at 1,809 campaigns and 5,365 days to MY2026 at 582 campaigns and 251 days. “Basis of measurement” credits the National Highway Traffic Safety Administration, Office of Defects Investigation, recalls flat file (FLAT_RCL_POST_2010), downloaded 12 September 2026, latest Part 573 report received date in it 9 September 2026, 13,608 vehicle campaigns; notes that model years before 2012 are left off because the file carries reports received from 1 January 2010, that model year 2027 has barely begun, and that the 117 campaigns with no model year are excluded; and says NHTSA publishes the file while the counts and the bars are ours. A method line runs along the foot: published records, as filed; one source, one field, one definition; zero-based linear axis, evenly ticked, unbroken; every vehicle campaign in the file; nothing smoothed, dropped or rescaled. A footnote credits the source and states that this slide is a demonstration drawn by us in the manner of an internal fleet safety briefing — the desk, the read-out and the recommendation are invented, and every figure behind them is real.

The 73% fall that is 15% once the cohorts are the same age

The immature cohort