The leaderboard is the shortlist. It was never the decision.

There is a ritual that plays out in engineering channels every few weeks. A new model tops a board. Someone posts the screenshot. Someone else opens a ticket to migrate. And somewhere in the same company, quietly and without a screenshot, an agent pipeline keeps routing its two billion monthly tokens to a model that has never topped anything — because it is good enough, it is fast, and it costs a twentieth as much. The gateway operators who can see everyone's traffic at once report the pattern bluntly: what production actually runs is close to uncorrelated with what the leaderboards crown.

That gap is not irrationality. It is the market noticing something the screenshots have not caught up with: in 2026, a public leaderboard is a shortlisting instrument. The decision has to be made somewhere else — and pretending otherwise is how teams end up paying frontier prices for work a tuned small model does better.

Why the boards stopped deciding

Three structural facts, none of them scandals, all of them fatal to leaderboard-as-verdict. First, saturation. The top of SWE-bench Verified now sits within about one point, and the trackers disagree with each other by a full point on who leads — the ordering at the top is inside the measurement noise. The human-preference arenas have the same shape: a top tier packed inside the confidence interval, which is one tier, not a ranking. When the podium is a statistical tie, “#1” is a coin flip with a press release.

Second, the harness moves the number more than the model does. The same benchmark name run through two harnesses can differ by ten to twenty points; tool-assisted runs of the hardest exam-style suites land ten points above closed-book ones; and this year's most instructive dispute had a single model scoring 93% or 56% on the same science benchmark depending on whether safety refusals counted as correct. None of those spreads describe capability. They describe protocol — which means a score without its protocol attached is not information.

Third, contamination is now the default assumption, not the exception. The static suites the industry grew up on have leaked into training corpora, which is why the boards that still separate models are the ones engineered against it — problems collected after training cutoffs, question pools that rotate monthly, harder successors built when the originals compressed. The honest reading of any static benchmark in 2026 is: this measures, at minimum, familiarity as well as ability, in unknown proportion.

The half of the decision that isn't on any board

Meanwhile the axis the boards don't rank has become the interesting one. Sticker prices now span two orders of magnitude — from twenty cents per million input tokens at the efficient tier to ten dollars at the premium frontier — and the sticker is itself a poor predictor of the bill. Output tokens list at four to eight times input. Cached repeated input is discounted up to ninety percent, which means a long, stable system prompt changes the economics of the whole deployment. A single coding-agent task consumes hundreds of thousands to millions of cumulative input tokens across its loop. And models differ several-fold in how many tokens they spend to solve the same task — this August's trackers measured a 4× spread on one agentic benchmark — so the cheap-per-token model can be the expensive-per-task one.

The number that decides is dollars per task solved, under your traffic shape — your input-to-output ratio, your cache hit rate, your loop depth, your context tier. No public board can compute it for you, because no public board knows your workload. That is not a gap in the boards. It is the boundary of what a public instrument can ever tell you.

Size and openness sit on the same private axis. Regulated or proprietary data pushes toward open weights running inside your own boundary — and the open tier is no longer a concession: open models now clear scores the closed frontier held two release cycles ago, at a fraction of the serving cost. Latency and unit economics push toward small models tuned on your task. This is why the year's quiet consensus is that 2026 belongs to fine-tuned small models: the shortlist that matters often ends not at “which frontier API” but at “which open base, tuned on what, served where.”

What the decision actually is

So the discipline, in order. Use the aggregate boards for what they are excellent at: cutting three hundred models to a handful, reading clusters as tiers, at one effort setting, within one harness. Price the shortlist against your workload shape, not the price sheet. Let the boundary — privacy, residency, latency — trim it again. And then make the decision the only way it can actually be made: a few hundred probes drawn from your real task, scored deterministically, so the same output always yields the same verdict. Not an LLM judge grading vibes. A checker computing output-against-rule — and grading safety on the same turn as correctness, because the run that was capable and unsafe at once is the one you are liable for.

Everything upstream of that evaluation is shortlisting. Everything downstream of it is governance: the chosen artifact bound to the evidence that chose it, an approval by someone who is not the person who tuned it, a signed record you can re-verify later. That last part stopped being optional this month — the EU's enforcement powers over general-purpose model providers went live on August 2, and the state laws arriving behind it all ask for the same artifact: not a screenshot of a leaderboard, but a record of what you evaluated, what you decided, and who approved it.

The teams that internalize this stop asking “which model is best” — a question 2026 has made unanswerable in the general case — and start asking a better one: “which model survives our evaluation, at our price per task, inside ourboundary, with evidence we can show?” That question has an answer. It just was never going to come from a screenshot.