The public scorecard reaches the knockouts: the AI called its sharpest scores yet — and walked straight back into the trap it had just escaped

In the last article I ended on an admission, not a result. I’d gone to grade the half of the system that actually wins me the league — the part that guesses who’ll walk onto the pitch — and found I couldn’t, because my own record of who played was wrong in places. It had marked players as benched who’d started the whole game. So I said the honest thing: I can’t grade a forecast against a scoreboard I don’t trust, and before I trust that half of the system again, I’ll go do the dull work and get the record clean. And I said you could watch me do it in public.

The first knockout round is graded now. And this is the round where I did that dull work — and where doing it changed what I could say about everything else.

Two things happened, pulling in opposite directions, and the honest version of this round needs both. The scoreboard I said I couldn’t trust: I cleaned it, and now that I can read it, the quiet half of the system looks healthy again. And the loud half — the match predictions on the public page — called its sharpest scorelines of the whole tournament, then walked straight back into the one overconfidence trap it had spent the group stage climbing out of.

One thing to keep clear, because it’s the whole point of doing this in public: I’m not training an AI to predict football, and there’s no secret model being built here. I take existing models — the same ones any company can pick off the shelf — and give them the tools, the context and the instructions to do a specific job, then sharpen how they do it round to round. It’s closer to handing a sharp new analyst your data and a checklist than to building a new brain from scratch. You put a good one to work on your problem and keep a readable record of where it helps and where it hurts — and, as this round shows, sometimes the most valuable work isn’t sharpening the analyst at all. It’s fixing the ruler you measure them with.

The headline, both halves of it

The good news: I fixed the scoreboard. The record of who actually started each game — the thing that was quietly wrong last round — is now sourced, cleaned, and honest. And with a scoreboard I can trust, the who-starts half of the system reads healthy again: right about four times out of five, back near where it was before its wobble.

The good news, part two: the match predictions got sharper on the scoreline than in any round so far. Not just “did it pick the winner” — the actual final scores. More dead-on results, smaller misses when it missed.

The bad news, in the same breath: the one number I’ve told you matters more than accuracy — whether the system knows when it’s about to be wrong — slipped backwards. It had climbed to its best level of the group stage. This round it gave almost all of that back — landing about where it started — and missed the target the system had set for itself. And the reason is almost funny in how exactly it repeats an earlier mistake.

So it’s a split decision: the quiet engine got its gauge fixed and reads healthy, the loud engine got more precise, and the honesty of its confidence went the wrong way — at the exact moment the games started costing double.

The recap, in case you’re new here

Because the point of this series is that you can check me, here’s the setup in one breath.

I’m running a team in a fantasy World Cup, and a squad of AI agents does the daily homework. Every round they predict two different things: who will actually start each game, and how each game will end. Then reality grades both, out loud, and I write up the marks — the misses louder than the hits. The group stage was three rounds; I graded each one here. Now the tournament is knockouts: lose and you’re out. The first knockout round pairs the thirty-two survivors into sixteen one-off games, and that’s the round on the table today.

On the plain question — did it pick the right winner — the arc across the whole tournament now reads: 11 of 24 (46%), 18 of 24 (75%), 17 of 24 (71%), then 11 of 15 in the knockouts (73%). Steady in the seventies since the opening round’s wobble. (A caveat that runs through this piece: a knockout round is only fifteen games, half the size of a group round, so its numbers are noisier — read the direction, not the decimal.)

But picking winners was never the hard part. The hard part is the two things underneath it: can the system tell who’s going to play, and does it know when its own call is shaky? One of those I finally got a clean look at this round. The other one tripped.

The scoreboard I said I couldn’t trust — so I fixed it

Start with the scoreboard fix, because it’s the whole reason this round is worth your while.

Last round the problem wasn’t the forecast — it was the grading of the forecast. My record of who’d actually played had errors in it, so when the system’s who-starts numbers looked worse, I genuinely couldn’t tell whether the model had slipped or my scoreboard was lying. You can’t improve what you can’t measure, and worse, a broken measurement teaches the system false lessons — it “learns” from mistakes that never happened.

So the fix this round wasn’t a cleverer model. It was the dullest work there is, and there was a lot of it:

• Every actual is now sourced. Who started, who came off the bench, who never left it — each one checked against the public match record (the tournament’s Wikipedia match pages) rather than taken on trust. In the outlier list, every miss carries a source link you can click.
• Stale guesses are thrown out. If a prediction was made two weeks before kickoff and never updated when the team news landed, it doesn’t get graded — because grading a fortnight-old guess against reality just teaches the system nonsense. That alone excluded over a thousand stale predictions — more than half the records — leaving only the few hundred actually refreshed before kickoff to be marked.
• No player slips through. Before a round’s predictions can be submitted, every name in every nation’s official squad has to carry a number. No silent gaps where a regular quietly gets forgotten.
• The boring, checkable stuff gets counted. Suspensions from yellow-and-red cards. Rotation after a team has already qualified. A backup goalkeeper who took the starting job and never gave it back. These aren’t clever reads; they’re facts you can look up — and the system now has to look them up.

And here’s the honest reward for that work. With a clean scoreboard, the who-starts half reads healthy: right about four times out of five across the tournament, and back up to that level this round after its group-stage dip. When it’s very sure a man starts, he starts about eighty-two percent of the time; when it’s very unsure, he sits. The gauge points the right way.

That top row matters more than it looks. Last round I told a story about a nailed-on starter who the record claimed had sat — and I had to walk it back, because the record was wrong. This round, with the scoreboard fixed, the same kind of case is unambiguous: France’s midfield anchor, rated a 93% lock, played zero minutes — because France had wrapped up qualification and rested him. That’s a real miss, and a specific, fixable one. It’s now a rule: when a nation has all-but-secured the next round, drop its regulars’ start-odds, because that’s exactly when coaches rotate. Same for the suspended South Africa midfielder the system had at 88% (it hadn’t counted his card), and the regular Morocco defender it had absurdly low (a plain research gap). Every miss became a checklist item. That’s what a fixed scoreboard buys you: misses stop being mysteries and start being to-do lists.

The scores got sharper — genuinely, this time

Now the loud half — the match predictions on the public page — and here the news is good.

Round after round I’d flagged that the system was fine at who wins but soft on how much. It would tip a polite 2–0 on a game that finished 5–0. This round the scorelines tightened hard. The average miss on the goal difference — how many goals apart the final score was — has fallen every single round: about 1.4 goals, then 1.3, then just under 1, and now 0.73 across the four rounds so far. More than half a goal sharper than where it started. And the exact-score hit rate roughly doubled: four of fifteen games called dead-on — Brazil–Japan 2–1, USA–Bosnia 2–0, Ivory Coast–Norway 1–2, Colombia–Ghana 1–0. On paper, the weak spot is closing.

Two rounds ago I was careful to say a similar improvement was partly luck — a kinder schedule with fewer blow-outs. This round it’s more real: the knockouts had their share of routs (France put three past Sweden, Spain three past Austria) and the system stayed close on the scoreline anyway. When your margin error drops while the games get more lopsided, that’s the model genuinely reading games better, not the fixtures doing it a favour. Fifteen games is still a small pile — I’m reading the direction, not banking the decimal — but the direction has held for four rounds now.

Then the old trap walked back in

Here’s the number I care about most, and the one that went the wrong way.

Across the group stage that gap did something I was proud of: it started at about eight points, collapsed to nothing in the middle round (the alarm), then healed to its widest of the whole stage — thirteen points — because the system had stopped tipping confident wins on games that were really coin-flips. Those are figures I read straight out of the raw prediction logs, round by round; the dashboard itself only shows the blended all-rounds number, which sits at 7.7. This round the per-round gap fell back to under eight — right about where it began, and short of the target of ten the system had written for itself after the group stage.

And the reason it slipped is almost comically on-brand. It’s the exact same mistake, one round later, with a bigger price tag.

The signature failure of the middle group round was the confident favourite held to a draw — tipping a strong side to win comfortably against an opponent who parks the bus. That’s football slang for a team that gives up on attacking, packs every player back in front of its own goal, and plays purely not to lose; against a side willing to do that, a favourite can dominate for ninety minutes and still not score. The system had learned to be wary of exactly that and gone quiet on those games. Then the knockouts arrived, and it forgot.

The reddest example: Germany, tipped to beat Paraguay at 76% confidence, drew 1–1 — and went out on penalties. Paraguay set up a deep, disciplined block, their goalkeeper was man of the match, a German goal was chalked off by video review, and the favourite simply couldn’t break them down. It’s a near-exact rerun of the mistake from two rounds ago, and it wasn’t alone — three of this round’s four wrong calls were the same shape: a favourite tipped to win, a stubborn underdog, a game that finished level. The system had climbed out of that hole and strolled right back into it.

And there was a second, opposite error that gives the whole thing away. On Spain against Austria the system got the winner right but throttled its own scoreline — tipping a cautious 2–1 because it had flagged Austria as a park-the-bus side. But Austria wasn’t one; they’d scored freely to even get here. Spain won 3–0 at a canter. So that one wasn’t a wrong call at all, just a margin left on the table — and taken together with Germany it’s the tell: in the same round the system both failed to be cautious where it should have been (Germany, a wrong call) and was over-cautious where it shouldn’t have been (Spain, a right call under-priced). It wasn’t applying a rule, it was applying a reflex — and a reflex fires at the wrong times.

Two honest caveats before I lean too hard on any of this, because fifteen games is a thin pile to draw a hard line through. One: an eight-versus-thirteen swing in that confidence gap is noisy at this sample size — the same caution I put on the win rate. Two, and sharper: knockout teams are, by construction, closer in quality — everyone here already survived a group. So the favourites genuinely are less of a sure thing than they were early on, which would squeeze that confidence gap on its own, with no “forgetting” required. I don’t think that’s the whole story — the Germany call and the Spain over-cap are a repeat-and-mirror of a mistake the system had a named fix for, which reads more like a reflex misfiring than games merely being tighter. But the honest split is that some of the slip is genuinely harder games and some is the old trap, and I can’t cleanly separate the two on fifteen matches. I won’t pretend I can.

The knockout twist: a draw is now a coin-flip

There’s a reason this old mistake costs more in the knockouts than it did in the group stage, and it’s worth spelling out because it’s the useful part.

In the group stage, a confident favourite held to a draw just drops two points and moves on. In the knockouts, a draw doesn’t end the game — it goes to a penalty shootout, and a shootout is close to a fifty-fifty. So when the system says it’s 76% sure Germany beats Paraguay, it’s quietly making two bets at once: that Germany won’t be held (they were), and that if they somehow are, they’ll win the shootout anyway (they didn’t). Both halves of an over-confident call get exposed. A 76% that would have been merely wrong in the group stage becomes a team knocked out of the tournament.

The system has since written itself a hard rule for exactly this, and it’s the plan for the next round: in a knockout, never put more than about seventy percent on a favourite to win outright, because the penalty-shootout risk has to be priced into the number. And the “park the bus” caution now has to pass an explicit three-point check — is the underdog actually sitting deep, do they have recent proof of it, is their keeper in form — instead of firing on a hunch. Whether that discipline holds when the next favourite meets the next stubborn underdog is precisely the thing you’ll be able to watch on the page.

The standings: still leading, but the cushion just halved

So where does all this leave the actual team? Still in front — but by less than it looks, and by less than it was.

I’m first, on 292 points. But second place is on 287 — a lead of just five points, where a round ago it was eleven. The gap has closed by more than half. At the same time I’m comfortably the most valuable squad in the league — roughly 8.7 million clear of the next most valuable squad, players and cash together. So it’s a lopsided lead: a fat cushion on paper value, a thin one on the thing that actually decides the league.

And that split is the whole tension of the run-in. The squad value came from the quiet who-starts engine — the one I just spent the round proving I can trust again — buying nailed-on starters cheap. But points come from surviving these one-off games, and that’s where the loud engine’s overconfidence can hurt: back a favourite too hard, watch them go out on penalties, and a rival closes the gap. A comfortable-looking lead built by the reliable half of the system can be nibbled away by the unreliable half. That’s not a football problem. It’s the same mix-up I’ve watched flatter real companies — confusing “we own valuable assets” with “we’re winning this quarter.”

Strip the football out

Read this round again and delete the word “football.” What’s left is three things that happen to every system you’d trust with a real decision.

First — the highest-value work this round wasn’t improving the model. It was fixing the ruler. I spent the round cleaning the record the system grades itself against, and that single unglamorous job was worth more than any clever tweak, because until the measurement is honest, every “improvement” is a guess and every “lesson” might be learned from an event that never happened. If you take one thing to your own dashboards: an unaudited metric isn’t a metric, it’s a rumour. Fix the ruler before you trust anything it measures.

Second — one number improving tells you nothing about the others. The scorelines got sharper this round and the confidence honesty got worse, in the same system, in the same games. A single headline figure — “accuracy is up” — would have hidden the trade completely. The metric worth watching is rarely the loud one; it’s whether the system knows when it’s about to be wrong.

Third — a fixed failure mode doesn’t stay fixed. The system had genuinely learned to stop being confidently wrong on coin-flip games. Then the conditions changed — the knockouts, with new pressures — and it walked straight back into the same trap, because it had learned a reflex, not a rule. It’s the shape of every “rule” that’s really just a habit: the discount that always gets waved through for a certain size of customer, the credit check that gets skipped for “the good clients” — fine while the business looks the way it always has, then wrong the first time a deal walks in that doesn’t fit the old mould. Discipline that holds under old conditions quietly breaks under new ones. The only way you find out is by measuring the same weakness again, after the ground shifts — not assuming a past fix is a permanent one.

So before the accuracy number, before the dashboard’s green ticks, here are the two I’d check first on any system you’re leaning on: can I trust the scoreboard I’m grading it with — and is the thing it was confidently wrong about last time still fixed, now the situation has changed?

The first knockout round is graded — the good, the fixed, and the old habit that came back. The mistakes cost double from here, and I go into the next round freshly reminded that a lead built by the careful half of a system can be spent by the reckless half. I’ll grade that in public too. If you saw your own project somewhere in here instead of a football pitch — a metric you never audited, a fix you assumed was permanent — that’s the more useful place to be looking, and you know where to find me.