MisleadingCharts
Back to the gallery

The 19% switchboard comeback that started at the lowest of eight estimates

Showing the misleading chart

Be the first to star this exhibit

An open-day flyer for a switchboard course (one we drew ourselves, for a school we invented) announces that telephone-operator jobs are up 19% in two years, and the BLS figures agree: 3,870 in May 2021, 4,600 in May 2023. But the window opens on the lowest of the eight May estimates since 2016. Zoomed out, the series fell from 8,860 to 4,600, and the two-year rise is about the size of the survey’s own sampling error.

01The claim

The switchboard is hiring again. The Bureau of Labor Statistics counted 3,870 telephone operators in May 2021 and 4,600 in May 2023: 730 more jobs, up 19% in two years, paying a mean $42,100 a year. Sign up for the 12-week certificate before the rush.

02The trick

Every number on the flyer is BLS’s, the axis starts at zero, and 4,600 really is 19% more than 3,870. The trick is where the window opens. When the flyer went out in spring 2024, the latest estimate was May 2023, and the occupation’s profile pages from May 2016 show May 2021 as the lowest of the eight: 8,860, 6,310, 5,160, 4,740, 4,630, then 3,870, 4,030 and 4,600. Start the chart there and the last two steps of a long slide become a comeback. Start it in 2016 and the series falls 48%. The 2016 estimate is the noisiest of the eight, with a relative standard error (the standard error as a share of the estimate) of 14.2%, but measured from May 2017 the fall is still 27%, and the flyer’s 4,600 is below every estimate from 2016 to 2019. A short window on a small occupation brings a second problem: sampling noise. Telephone operators are a few thousand people estimated from an employer survey, with relative standard errors of 7.3% in 2021 and 6.4% in 2023. Treat the two years as independent and the 730-job rise has a rough 95% range of about −70 to +1,530: the rise is about the size of the survey’s own error and could plausibly be zero. The survey’s design blurs the comparison further. Each May estimate pools six half-yearly panels collected over three years, so the 2021 and 2023 figures share two panels, November 2020 and May 2021, and the 730 really compares the four panels that differ. May 2021 is also the first estimate BLS made with a new model-based method, MB3, which BLS says “will also affect the comparability of estimates produced before and after the change.” The FAQ’s general advice is that the survey “is less useful for looking at changes over time”, and “The Bureau of Labor Statistics does not encourage the use of OEWS data for time-series analysis.” (The flyer and the school are invented; we drew both charts from BLS’s published figures.)

03The fix

The redraw plots the eight May estimates from 2016 to 2023, the latest available when the flyer went out, on the same zero-based axis. The flyer’s two years are shaded so you can see where its window opens, and a dashed line marks where BLS switched to its MB3 estimation method, so the long view crosses a method change too. Each dot gets a bar for its rough 95% range, the published relative standard error times 1.96 on either side. Those ranges and the range for the change are ours. The change range treats the two years as independent, which ignores the panels they share, so it is a rough guide, not a formal test. With the bars drawn, the two-year rise is no larger than the error around it and small next to the fall before it. A strip underneath shows the six survey panels behind each estimate, so readers can see that one year’s dot is not one year of data. The general habit has three parts. When a trend starts on a particular date, ask what the chart looks like from every other start date the data allows. When the series is a sample estimate, put an error range on the change itself before you read anything into a two-year move; overlapping error bars on the two endpoints are not that test. And when the agency says its survey is not built for time series, put that warning on the chart. BLS’s own outlook points the other way from the flyer. Its employment projections, which start from about 3,500 jobs in 2025, have telephone operators falling 27.6% by 2035, the second-fastest decline of any occupation after word processors and typists. Switchboard operators are third.