Gen 9 Random Battle

PokéBench

AI models and coded players, ranked by benchmark Elo.

Company / associationShowing 13 of 13

#	Model	Effort
1	GPT-6.1 SolOpenAI · Low	Low	15881528–1698 95% CI 1528–1698	96.20	$0.1381	6.79	111	132	85.6%
2	Foul PlayOther	—	15601513–1623 95% CI 1513–1623	—	$0	0	0	791	92.9%
3	Claude Opus 5.5Anthropic · Low	Low	13521262–1473 95% CI 1262–1473	70.34	$0.3552	6.09	178	84	63.1%
4	Claude Fable 5.1Anthropic · Low	Low	13391247–1443 95% CI 1247–1443	64.74	$1.0577	9.33	280	84	61.9%
5	GPT-6 SolOpenAI · Low	Low	13301243–1429 95% CI 1243–1429	71.46	$0.1754	5.80	137	132	65.2%
6	GPT-6 LunaOpenAI · High	High	11981112–1331 95% CI 1112–1331	67.05	$0.0107	8.11	313	40	70.0%
7	Claude Sonnet 5.5Anthropic · Low	Low	11221032–1223 95% CI 1032–1223	53.11	$0.2268	5.76	249	100	38.0%
8	Heuristic botOther	—	10981078–1124 95% CI 1078–1124	63.54	$0	0.001	0	1,628	55.0%
9	GPT-6 LunaOpenAI · Low	Low	1022949–1086 95% CI 949–1086	55.13	$0.0075	3.98	34	132	34.8%
10	Max-damage botOther	—	10001000–1000 95% CI 1000–1000	56.25	$0	0.002	0	1,323	54.5%
11	Claude Haiku 4.5Anthropic · High	High	967895–1003 95% CI 895–1003	37.01	$0.4176	22.75	2.1k	20	45.0%
12	Claude Haiku 4.5Anthropic · None	None	816695–932 95% CI 695–932	33.38	$0.1639	7.31	456	112	14.3%
13	Random botOther	—	244114–335 95% CI 114–335	0.00	$0	0.002	0	1,028	1.0%

Elo with 95% intervals · Select a metric to sort · Swipe for more statistics

Head-to-head matrix

Each cell shows wins–losses from the row player's view, against the named column opponent. Game counts include ties. Unplayed and self comparisons have no result. Select Replays to watch a published game. Player order follows the initial Elo ranking. Only the published snapshot is included. Same-lab model games remain excluded.

The filter selects row players. All opponent columns remain available. Scroll sideways to compare opponents. Player labels stay fixed.

Row player	H2H vsGPT-6.1 SolLow	H2H vsFoul PlayReference	H2H vsClaude Opus 5.5Low	H2H vsClaude Fable 5.1Low	H2H vsGPT-6 SolLow	H2H vsGPT-6 LunaHigh	H2H vsClaude Sonnet 5.5Low	H2H vsHeuristic botReference	H2H vsGPT-6 LunaLow	H2H vsMax-damage botReference	H2H vsClaude Haiku 4.5High	H2H vsClaude Haiku 4.5None	H2H vsRandom botReference
GPT-6.1 SolLow	–Self	10–818 games	15–520 games Replays (1) Game 1: Row won	16–420 games Replays (1) Game 1: Row won	–Unplayed	–Unplayed	18–220 games	18–018 games	–Unplayed	12–012 games Replays (1) Game 1: Row won	–Unplayed	20–020 games	4–04 games
Foul PlayReference	8–1018 games	–Self	–Unplayed	–Unplayed	15–318 games Replays (1) Game 1: Row won	–Unplayed	11–112 games	473–27500 games Replays (11) Game 1: Row won Game 2: Row won Game 3: Row won Game 4: Row won Game 5: Row won Game 6: Row won Game 7: Row lost Game 8: Row won Game 9: Row won Game 10: Row won Game 11: Row won	18–018 games	192–15207 games Replays (4) Game 1: Row won Game 2: Row won Game 3: Row won Game 4: Row won	–Unplayed	18–018 games Replays (1) Game 1: Row won	–Unplayed
Claude Opus 5.5Low	5–1520 games Replays (1) Game 1: Row lost	–Unplayed	–Self	–Unplayed	10–1020 games	–Unplayed	–Unplayed	10–212 games Replays (1) Game 1: Row won	17–320 games	7–18 games	–Unplayed	–Unplayed	4–04 games
Claude Fable 5.1Low	4–1620 games Replays (1) Game 1: Row lost	–Unplayed	–Unplayed	–Self	10–1020 games Replays (1) Game 1: Row lost	–Unplayed	–Unplayed	10–212 games Replays (1) Game 1: Row won	18–220 games	6–28 games	–Unplayed	–Unplayed	4–04 games
GPT-6 SolLow	–Unplayed	3–1518 games Replays (1) Game 1: Row lost	10–1020 games	10–1020 games Replays (1) Game 1: Row won	–Self	–Unplayed	17–320 games	15–318 games	–Unplayed	11–112 games	–Unplayed	16–420 games Replays (1) Game 1: Row lost	4–04 games
GPT-6 LunaHigh	–Unplayed	–Unplayed	–Unplayed	–Unplayed	–Unplayed	–Self	–Unplayed	13–720 games	–Unplayed	15–520 games Replays (1) Game 1: Row lost	–Unplayed	–Unplayed	–Unplayed
Claude Sonnet 5.5Low	2–1820 games	1–1112 games	–Unplayed	–Unplayed	3–1720 games	–Unplayed	–Self	8–412 games Replays (1) Game 1: Row won	12–820 games Replays (1) Game 1: Row won	8–412 games	–Unplayed	–Unplayed	4–04 games
Heuristic botReference	0–1818 games	27–473500 games Replays (11) Game 1: Row lost Game 2: Row lost Game 3: Row lost Game 4: Row lost Game 5: Row lost Game 6: Row lost Game 7: Row won Game 8: Row lost Game 9: Row lost Game 10: Row lost Game 11: Row lost	2–1012 games Replays (1) Game 1: Row lost	2–1012 games Replays (1) Game 1: Row lost	3–1518 games	7–1320 games	4–812 games Replays (1) Game 1: Row lost	–Self	12–618 games	327–173500 games Replays (11) Game 1: Row won Game 2: Row won Game 3: Row won Game 4: Row won Game 5: Row won Game 6: Row won Game 7: Row lost Game 8: Row won Game 9: Row won Game 10: Row won Game 11: Row won	–Unplayed	15–318 games	497–3500 games Replays (10) Game 1: Row won Game 2: Row won Game 3: Row won Game 4: Row won Game 5: Row won Game 6: Row won Game 7: Row won Game 8: Row won Game 9: Row won Game 10: Row won
GPT-6 LunaLow	–Unplayed	0–1818 games	3–1720 games	2–1820 games	–Unplayed	–Unplayed	8–1220 games Replays (1) Game 1: Row lost	6–1218 games	–Self	6–612 games	–Unplayed	17–320 games	4–04 games
Max-damage botReference	0–1212 games Replays (1) Game 1: Row lost	15–192207 games Replays (4) Game 1: Row lost Game 2: Row lost Game 3: Row lost Game 4: Row lost	1–78 games	2–68 games	1–1112 games	5–1520 games Replays (1) Game 1: Row won	4–812 games	173–327500 games Replays (11) Game 1: Row lost Game 2: Row lost Game 3: Row lost Game 4: Row lost Game 5: Row lost Game 6: Row lost Game 7: Row won Game 8: Row lost Game 9: Row lost Game 10: Row lost Game 11: Row lost	6–612 games	–Self	11–920 games	10–212 games	493–7500 games Replays (11) Game 1: Row won Game 2: Row won Game 3: Row won Game 4: Row lost Game 5: Row won Game 6: Row won Game 7: Row won Game 8: Row won Game 9: Row won Game 10: Row won Game 11: Row won
Claude Haiku 4.5High	–Unplayed	–Unplayed	–Unplayed	–Unplayed	–Unplayed	–Unplayed	–Unplayed	–Unplayed	–Unplayed	9–1120 games	–Self	–Unplayed	–Unplayed
Claude Haiku 4.5None	0–2020 games	0–1818 games Replays (1) Game 1: Row lost	–Unplayed	–Unplayed	4–1620 games Replays (1) Game 1: Row won	–Unplayed	–Unplayed	3–1518 games	3–1720 games	2–1012 games	–Unplayed	–Self	4–04 games Replays (1) Game 1: Row won
Random botReference	0–44 games	–Unplayed	0–44 games	0–44 games	0–44 games	–Unplayed	0–44 games	3–497500 games Replays (10) Game 1: Row lost Game 2: Row lost Game 3: Row lost Game 4: Row lost Game 5: Row lost Game 6: Row lost Game 7: Row lost Game 8: Row lost Game 9: Row lost Game 10: Row lost	0–44 games	7–493500 games Replays (11) Game 1: Row lost Game 2: Row lost Game 3: Row lost Game 4: Row won Game 5: Row lost Game 6: Row lost Game 7: Row lost Game 8: Row lost Game 9: Row lost Game 10: Row lost Game 11: Row lost	–Unplayed	0–44 games Replays (1) Game 1: Row lost	–Self

Anthropic vs OpenAI

Lab summary

4 Anthropic models and 3 OpenAI models, at their lowest thinking settings. This is one Pokémon benchmark, not a measure of provider intelligence.

The ratings below use all benchmark opponents, including reference bots. They are relative to the Max-damage bot, not Showdown ladder ratings. Bars show 95% intervals.

Anthropic

Highest benchmark rating

Claude Opus 5.5 · lowest thinking (low)

1352 95% interval 1262⁠–⁠1473

OpenAI

Highest benchmark rating

GPT-6.1 Sol · lowest reasoning (low)

1588 95% interval 1528⁠–⁠1698

240 games between Anthropic’s and OpenAI’s models: Anthropic won 88, OpenAI won 152. The aggregate is weighted by games: each completed cross-lab game counts once. It is not an equal average of models or pairings. Model pairs with more games have more weight. The bar shows the share of points (a win is 1 point, a tie ½).

Gen 9 Random Battle

How well do AI models play Pokémon?

PokéBench ranks AI models by how well they play Pokémon Showdown random battles. Every model gets the same open prompt and the same teams, and every rating comes with its 95% interval.

See the leaderboard Read the method

GPT-6.1 Sol · lowest reasoning (low) chose Keldeo-Resolute.

“Keldeo resists both of Bisharp’s STAB types and threatens it with a quadruple-effective S…”

Each model gives a reason for each move. Select the text box to watch the battle.

Method

The method in one screen

Fixed seeds

A seed fixes both teams and every random roll of a battle. Anyone can play the same games again.

Mirror matches

Each pair of teams is played twice, with the players swapped. A lucky team then helps each side once.

Ratings with intervals

One Bradley–Terry model, fitted to all games at once, gives the Elo ratings. 200 bootstrap samples give each 95% interval.

One open prompt

Each game is one conversation, in the same plain text for every model: what a player sees on screen, and the legal actions. No tools, no hints.

The cost of each game

We count the tokens and the price of each answer, so every rating comes with its cost per game.

The full method

The prompt, the rating model, the limits, and how to repeat a run.

Read the method

Enter the game

Walk around PokéBench Town. The League holds the leaderboard, the TV in your house shows battles, and the Professor in the lab explains the method.

What the people in the town say

In the town: A RATING is a number for skill. We use the ELO scale. Chess uses it too. Say you are 200 points above me. On average, you win 3 games in 4.
In the town: Luck matters in a battle. One side may get a better team. So we play each battle twice. The two games use the same teams. In game 2, they swap teams. That is a MIRROR MATCH.
In the town: Here we play RANDOM BATTLES. Each side gets 6 random POKéMON. Nobody can plan a team. So it tests how well you play!
In the town: A model sees what you see. It gets no damage calculator. It picks a move or a switch. And it gives a short reason.
In the League: Welcome to the POKéBENCH LEAGUE! A RATING is a number for skill. A longer bar is a higher rating.
In the lab: Hello, this is my lab! We test how well AI models battle. A SEED is a number. It fixes all the luck. Same seed, same teams. We play each battle twice. In game 2, they swap teams. Wins and losses give a RATING. A RATING is a number for skill. A model reads the battle as text. It sees its team and legal moves.

2,803 games of Gen 9 Random Battle, singles · data of 6 October 2026 · prompt version 3 · Showdown 3661ce40

These Elo ratings are not directly comparable to Pokémon Showdown ladder ratings. We played no human opponents and set the Max-damage bot’s Elo to 1000. Intervals are 95% confidence intervals. Read the method.

Win rate counts wins / games. Opponent strengths vary, so aggregate win rates do not determine Elo.

Price/game uses saved token-rate accounting per completed game, including saved retry usage. Subscription-backed gateway costs are not necessarily cash charges. Coded players have no model API cost.

Tokens/turn averages recorded output tokens per saved decision with usage. Input and cached input are excluded. Reported thinking is already included in output and is counted once. Saved reask/retry usage and simulator-rejected decisions are included, unlike Seconds/turn. Separate thinking counts may be unreported; we do not estimate unreported thinking or unsaved failed-attempt usage. Coded bots use 0 API tokens.

Seconds/turn averages saved latency per accepted decision, including moves, switches, and retries within that decision. Simulator-rejected decisions are excluded. Foul Play timing is shown as 0 by convention; its live decision time was not measured. Error and incomplete games are excluded.

Score v1: formula and inputs

Score is 0–100, relative to this snapshot. Elo still sets the default order. Max-damage anchors benchmark Elo at 1000. Score uses the translated Random rating as its strength baseline, so a change of Elo origin does not change Score. Top Elo is the highest fitted rating across all published entrants, before sorting.

q = clamp((Elo − Random Elo) / (top Elo − Random Elo), 0, 1)
w = 0.40 + 0.50 × q²
C = 1 / (1 + cost per turn / $0.01)
T = 1 / (1 + seconds per turn / 5)
Score = 100 × q × (w + (1 − w) × (0.80 × C + 0.20 × T))

Quality, cost and speed weights are 90% / 8% / 2% at the top rating, and 52.5% / 38% / 9.5% at the midpoint between Random and the top rating. The $0.01 and 5-second references are fixed v1 constants.

Cost per turn divides total recorded API usage cost by accepted decisions. Its numerator includes simulator-rejected usage and recorded retries, counted once per saved decision. Tokens/turn instead averages all saved decisions with usage. Timing uses the published millisecond average per accepted decision, before display rounding.

Coded players have zero API usage cost. Foul Play has no Score because its live decision time was not measured. Other missing inputs or a missing or mismatched snapshot cost supplement also make Score unavailable.