Methodology
How this works
Five frontier models were each given the identical brief and the identical prompt for every prediction. No model ran its own web search. All grounding came from the shared brief, so the comparison isolates reasoning rather than search tooling.
Predictions were committed before kickoff
The full set, meaning 72 group match distributions, the tournament awards and a complete bracket for each model, was generated once, serialized in a fixed canonical form, and hashed with SHA-256. The resulting fingerprint is public, so no pick can be quietly changed after the fact. Change a single probability anywhere in the file and the fingerprint changes completely.
sha256:b233af4fa5051a6d1990fb0d8d9ce03c01c4f9aa71629239dcc2d2a55a7b36e6
Verify it yourself
The commitment is a plain SHA-256 over the canonical serialization of the aggregate prediction file. If you have a copy of that file and the canonical serializer, recomputing the hash should reproduce the fingerprint above character for character. If it does not, something changed, and that is the entire point of publishing it.
Pinned model versions
Versions are frozen for the whole tournament. Silently upgrading a model partway through would invalidate the comparison, because you would no longer be scoring the same contestant.
| Entrant | Lab | Frozen version |
|---|---|---|
| Claude | Anthropic | claude-opus-4-8 |
| GPT | OpenAI | gpt-5.5 |
| Gemini | gemini-3.1-pro-preview | |
| Grok | xAI | grok-4.3 |
| DeepSeek | DeepSeek | deepseek-v4-pro |
Scoring rules
- Brier score. The squared error between each predicted probability and the actual outcome, summed over all graded matches. Lower is better. This is the headline ranking metric because it rewards calibration, not bravado.
- Log-loss. Punishes confident wrong picks more sharply than Brier does. It serves as a secondary tiebreaker.
- Pick accuracy. The share of matches where a model's single most likely outcome turned out to be correct. Useful, but on its own it hides how confident a model was.
- Consensus. Computed by averaging the five probability distributions, not by counting votes.
Why rank by Brier and not by win rate? Because a model can pick more winners and still be worse calibrated. Brier captures whether a model's stated confidence actually matched how often it was right. Early in the tournament the sample is small, so treat the ordering as provisional until enough matches are graded.
A note on Grok
Grok has live data access in normal use. That edge is deliberately neutralised here. Every model reasons only from the shared brief, with web search turned off, so no entrant gets to peek at live information the others cannot see.
Data and revisions
Hard facts, meaning fixtures, groups, rankings and the knockout template, come from api-football. Narrative context came from a research step prompted for facts rather than opinions. Results are graded within roughly 10 minutes of full time and marked provisional, then finalised at a fixed cutoff against one canonical source. Revisions are logged.