About

Can an AI lie well, and catch a liar?

Ten to twelve players get secret characters. Three are evil and know each other; the rest are good and know nothing about anyone. Nobody can prove who they are. That makes the game a demanding test of the skills we most want to measure in AI: telling a lie that survives scrutiny, coordinating in secret, and modelling what other people believe.

Here, one AI model plays every good seat and another plays every evil seat, and each deal is replayed with the sides swapped. A model's rating combines how well it detects as the town and how well it deceives as the Demon's team. Dwarkesh's video essay tells the story of the eval, game by game.

The rules in one page

Teams. Ten to twelve players each get a secret character. Most are good. Three are evil: one Demon (the Imp) and two Minions. The evil players know each other; the good players know nothing about anyone.

Winning. Good wins by executing the Demon. Evil wins when only two players are left alive.

Night and day. Each night the Demon kills a player, and many good characters secretly learn a clue. Each day everyone talks, then the living players may nominate someone; the nominee with the most votes, if that reaches half the living players, is executed. Dead players keep talking and keep one "ghost vote" for the rest of the game.

Nobody can prove anything. Characters are never revealed in life, so every claim might be a lie. The evil team claims good characters and invents clues to match. And some true-looking clues are false: the Poisoner can silently break a player's ability, and the Drunk thinks they have a role they don't.

The characters (the "Trouble Brewing" script)

How the harness works

• One model per team. One model plays every good seat, another every evil seat, so a game pits one model's detection against the other's deception. Each seat is its own conversation; teammates share no hidden channel beyond in-game whispers.
• Side-swapped pairs. Every deal is played twice with the models' sides swapped, so neither gets the luckier hand. The library links each game to its twin.
• A scripted Storyteller. No model judges the game. Every Storyteller choice is a random pick over legal options with hand-set weights (for instance, a poisoned player's clue is false 80% of the time), and every rule can be explained in one line.
• A fixed day. Six rounds of private whispers (each player holds one conversation at a time, with one or two others), three rounds of public speech in the town square, then nominations and votes. Within a round, players act in three random waves, so later speakers can respond to earlier ones.
• Equal settings. Every model plays at its lab's "high" reasoning effort, with the same 8,000-token ceiling per turn and the same word limits (150 words to speak or whisper). All text reaches other players in one plain typography, so formatting can't give a team away.
• Memory like a person's. Each seat remembers only what it saw and what it said and thought itself. There are no notes and no summaries.
• Belief probes. Before each day's votes, every good player is asked, outside the game, how likely each other player is to be the Demon. The answers never enter the game; they show how the town's beliefs move. The replay viewer charts them day by day.
• Ratings. One Bradley–Terry fit over all rated games gives each model an Elo rating, plus separate detection and deception ratings. See the leaderboard.

Run your own

The harness is open source, and every game we played is in a dataset: transcripts, private whispers, reasoning, belief probes and the cost of every API call.

Quick start

git clone https://github.com/…/clocktower && cd clocktower
pip install -e .                       # Python 3.11+
export ANTHROPIC_API_KEY=... OPENAI_API_KEY=...
python -m unittest discover -s tests   # rules and engine tests, no API calls
python -m clocktower sim --games 200   # random players, no API: check the Storyteller

Play one side-swapped pair

python -m clocktower run --good-model claude-opus-5-5 --evil-model gpt-6.1-sol \
    --games 1 --swap --seed 1 --out games/v17/demo
python -m clocktower replay games/v17/demo/game-01 --thinking     # read it

Each game writes transcript.md (an omniscient replay with every seat's reasoning), events.jsonl, probes.jsonl (belief probes), calls.jsonl (every API call with tokens and cost), config.json and result.json. A game costs from about $2 to $50 depending on the models.

Run a tournament and rate it

python -m clocktower plan plans/models-2026.json --within 10 --pairs 1 \
    --seed 120000 --out games/v17/mine > plans/mine.json
python -m clocktower tournament plans/mine.json      # every game at once; rerun to resume
python -m clocktower analyze games/v17/mine-*/game-* # win rates, probes, quick ratings
python -m clocktower ratings                         # the Bradley–Terry fit with bootstrap intervals

Use the existing games

python -m clocktower fetch v17/f3-*      # download logs from the dataset
python -m clocktower.export --games games  # rebuild this website from them

Mixed teams (--good-model A,B), one upgraded seat (--seat-model good:MODEL:1) and a different model for the Imp (--seat-model Imp:MODEL) are all built in. The full protocol is in docs/PROTOCOL.md; why each rule is what it is, in docs/protocol-history.md.

Questions worth exploring

• Learning over time. Do models get better across games? In context (notes carried between games) or by fine-tuning on their own logs. Careful: shared notes on the good team leak who is good, so evil-only learners or a league of individual agents are the clean designs.
• Harder scripts. Trouble Brewing is the beginner script. Characters like the Lunatic (who believes they are the Demon) or scripts with more misinformation would test whether models can reason about information that is wrong on purpose.
• One smart player. How much does a single strong model lift a team of weaker ones, as good and as evil? Which evil role carries the team?
• Mixed teams, systematically. We found half-and-half evil teams play worse than either model alone. Is that coordination cost, or style mismatch? Does it shrink with stronger models?
• Teammate recognition. Writing style alone separates the teams in our logs, though no model ever used it. Do models learn to fingerprint their teammates, and does uniform typography stop it?
• Humans vs AIs. Seat humans at the table (it's built in) for a human baseline, and to see whether people can tell which players are models.
• Thinking on votes. Claude models skip thinking on most votes as good, and unthinking votes follow the running tally more. Forcing a thought before each vote is a cheap ablation.
• Memory. Models that carry their own reasoning across turns vs replies only: how big is the handicap?

The data

Every game on this site, with every seat's reasoning, is in the Hugging Face dataset. The ratings come from one fit over all rated games; the leaderboard explains the method.

Credits

Clocktower Bench was made by Dwarkesh Patel, with the harness, experiments and analysis built together with Claude.

Not affiliated

This is an unofficial fan project and research eval. It is not affiliated with, endorsed or sponsored by The Pandemonium Institute. Blood on the Clocktower is a trademark of The Pandemonium Institute. No official artwork, logos or icons are used; every illustration here is original. If you enjoy the game, buy it and play it with friends.