Drawmaha Solver Open the solver

Wiki

Everything about the solver, in one place

Overview · Getting started

Introduction

What the Drawmaha solver is, the rules of the game it is built for, and the terms used throughout this wiki.

The Drawmaha solver is a game-theory solver for heads-up pot-limit Drawmaha, a split-pot hybrid of draw poker and Omaha that no existing solver plays. It does not attack the full game head-on. It is built as a validation ladder: a sequence of games, each one adding a single new idea to the last, and each one graded against a known answer before the next is attempted. By the time the method meets the real game, every piece of it has been checked somewhere a mistake would have shown.

The question every step answers is the same: how much would a perfect adversary win against this strategy? That number, exploitability, is zero exactly at a Nash equilibrium and positive everywhere else, and it is computed exactly by a grader that shares no code with the solver. Training loss is never used as a progress meter, because in self-play it can fall while the strategy gets no better.

The game

If you have never played Drawmaha, this is the minimum you need to follow the rest of the wiki. The target game, as the README's "The target game" section defines it:

  • Each player is dealt 5 hole cards.
  • It is a bomb pot: both players ante and the flop comes out with no preflop betting. The order is ante → flop (3 cards) → betting → one draw (discard 0–5 cards and replace them) → turn → betting → river → betting. Three betting rounds, not Omaha's four.
  • The draw-1 rule: a player who draws exactly one card is dealt it face up; they may keep it or reject it and take the next card face down.
  • Split pot: half the pot goes to the best Omaha hand (exactly 2 hole cards + 3 board cards), half to the best 5-card poker hand made of the hole cards alone.

The solve's defaults, all configurable: pot-limit with the action set {fold, check/call, pot}; 100 big blinds effective; both players ante, so there are no blinds and no positional asymmetry from them. Bet sizes are a config, so a multi-size solve is a re-solve, not a rewrite.

Why it is hard: a hand is judged two ways at once, the draw changes both of them, and the draw-1 rule leaks information publicly. A strategy has to balance what it holds for the Omaha half against what it holds for the draw half, and the size of the full game rules out the table-based methods that solve the small games below. See Abstraction for how the project shrinks it, and Full Drawmaha for what is still to be done. The live rules page is at /rules.

Poker glossary

TermMeaning
Heads-upExactly two players. Every game in this project is two-player and zero-sum: what one wins the other loses.
AnteA forced bet every player puts in before the cards. In every game here both players ante, which is what makes the pot worth fighting for.
Bomb potA hand where everyone antes and the first board cards come out with no betting before them.
Hole cardsA player's private cards, seen by them alone.
BoardShared face-up cards both players use.
DrawDiscarding some hole cards and receiving replacements from the deck.
Street / roundOne betting round, after a set of board cards or a draw.
Pot-limitThe largest bet allowed is the size of the pot. "Pot" in the action set means a pot-sized bet.
Fold / check / call / bet / raiseGive up the hand / pass with nothing to call / match the bet in front of you / put chips in first / increase a bet.
ShowdownWhen betting ends with two players left, the hands are compared.
Split potThe pot is divided into halves, each won by a different hand ranking. Winning both halves is a scoop.
Omaha handA five-card hand built from exactly two hole cards and exactly three board cards.
Big blind (bb)The unit stack sizes are quoted in. With antes only, it is a unit, not a blind anyone posts.
Chips per handThe unit every grade is quoted in: chips won or lost on average per hand dealt, with the ante as one chip in the small games.

Game-theory glossary

TermMeaning
Strategy (σ)For every situation a player can be in, a probability for each legal action. A strategy that mixes is a mixed strategy; one that always picks one action per situation is pure.
Information set (infoset)Everything the acting player knows at a decision: their own cards and the public history, never the opponent's cards. A strategy is defined per infoset, because a player cannot act differently in two situations they cannot tell apart.
Chance nodeA point where the deck acts rather than a player: a deal, a board card, a draw.
Nash equilibriumA pair of strategies where neither player gains by changing their own. In a two-player zero-sum game, playing one guarantees at least the game's value against anyone.
Game valueWhat a seat wins per hand when both play an equilibrium. Kuhn poker's first seat is worth −1/18.
Best response (BR)The strategy that wins the most against one fixed opponent strategy.
ExploitabilityThe average of the two seats' best-response winnings against a strategy pair, (BR₀ + BR₁) / 2. Zero at equilibrium. The project's only grade.
RegretHow much better an action would have done than what the strategy actually did, accumulated over iterations.
Regret matchingPlay each action in proportion to its positive accumulated regret.
CFRCounterfactual regret minimisation: regret matching at every infoset of a game tree at once, with each update weighted by how likely the others make it to reach that infoset.
Average strategyThe reach-weighted average of every iteration's strategy. It is CFR's answer: the average converges to equilibrium; the current strategy need not.
MCCFRMonte Carlo CFR: CFR on sampled hands instead of the whole tree, for games too large to enumerate.
Deep CFRCFR with the per-infoset regret tables replaced by a neural network that generalises across infosets.

Key concepts

These are the ideas the solver is built from. Each links to its full treatment in the Theory tab.

ConceptWhat it is
Nash equilibrium and exploitabilityThe target, and the meter. Exploitability measures distance from equilibrium in the units that matter, chips an adversary takes. See Equilibrium and Exploitability.
Regret matchingThe learning rule at the bottom of every game here, unchanged from rock-paper-scissors to mini-drawmaha. One ledger per decision; only what feeds it changes. See Regret matching.
Counterfactual regretRegret at one infoset in a tree, weighted by the probability that chance and the opponent bring play there. The two reach weights, own and counterfactual, are the classic place to make a silent mistake. See CFR.
Exact best responseA grader that walks the whole game, sums over the hidden cards first and takes the adversary's best action second. Reversing the order describes an adversary who sees your cards. See Exploitability.
External-sampling MCCFR, run in lockstepSample one hand per seat per worker, bank every worker's regret against the same frozen strategy, and get bit-for-bit reproducible parallel training. See MCCFR.
Regret rulesVanilla, LCFR (linear weighting), CFR+ and DCFR: different ways of weighting and flooring regrets over time. LCFR won the race on mini-drawmaha. See Regret rules.
Deep CFRReplace 6.2 million regret ledgers with one network, refit each iteration from a reservoir of sampled regrets. See Deep CFR.
AbstractionMini-drawmaha keeps Drawmaha's structure (a bomb pot, a draw, a pot that pays twice) on a deck small enough to grade exactly. See Abstraction.

Overview · Getting started

Method

The validation ladder, the pre-registered evaluation protocol, and what each rung found.

The full game is too large to solve on the first try, and a solver that is wrong does not announce it: a swapped weight or a peeking grader still produces a strategy, a falling curve and a confident number. The project's answer is to never take a step that cannot be checked. Each rung is a game that adds exactly one new idea to the rung below it, and each is graded against an answer that was known before the solver ran.

Each rung adds one idea and is graded before the next

RungGameWhat it addsGraded against
0Rock-paper-scissorsthe regret-matching ledger itselfthe uniform Nash, closed form
1Kuhn pokera game tree, hidden cards, CFR's two reach weightsthe closed-form equilibrium family; all 64 pure strategies of each seat enumerated
2Leduc pokerchance in the middle of the hand; actions that differ by spotan OpenSpiel solve, frozen as tests/leduc/referee.json
3Mini-drawmaha, 6.2M infosetssampling (MCCFR), parallel lockstep, four regret rules racedthe exact best-response walk; no outside engine plays this game
4Mini-drawmahaa neural network in place of the regret tables (Deep CFR)rung 3's grader and rung 3's frozen answer
nextFull DrawmahaDeep CFR at scalea head-to-head checkpoint ladder and a trained exploiter

The learning rule never changes as the games grow. src/drawmaha_solver/regret_matching.py is the same ledger at every rung; what changes is how the utilities it is fed are computed — a payoff column in rock-paper-scissors, a tree walk in Kuhn and Leduc, a sampled walk in mini-drawmaha, a network's output in Deep CFR. See The ledger.

Exploitability is the only grade

The evaluation protocol was fixed in the README before the results existed ("Evaluation protocol (pre-registered)"). The reasoning: in self-play, training loss is not a progress meter. Both players move together, so a loss can fall while the pair stays easy to beat. The only honest question is what an adversary wins.

exploitability(σ) = (BR₀(σ₁) + BR₁(σ₀)) / 2
  • BR₀(σ₁) is what seat 0 wins per hand by playing its best response against seat 1's strategy, and BR₁(σ₀) the same for seat 1.
  • Why the average of two: the game is zero-sum, so at an equilibrium the two best responses are the game value and its negative and the sum is exactly zero. Any leak in either seat's strategy pushes one of them up, so the average is positive everywhere else.
  • What moves it: a strategy that leaks more chips to a perfect opponent in either seat raises it; it cannot be lowered by playing well against a weak opponent. That is what makes it a meter and not a score.

How it is computed at each rung:

  • Toys (rungs 0–2): exactly, and checked against Kuhn's closed form and an OpenSpiel solve of Leduc.
  • Mini-drawmaha (rungs 3–4): the exact best-response walk in src/drawmaha_solver/minidrawmaha/exploitability.py as ground truth, plus the Deep CFR versus tabular gap.
  • Full scale: a duplicate-dealt head-to-head ladder of checkpoints plus a trained reinforcement-learning exploiter against the frozen network, reported as a lower bound on exploitability, because no exact walk is possible there.

Units are total chips across both pot halves, per hand, with per-half EV and scoop rate kept as split-pot diagnostics. The grader never imports the solver — see Onboarding.

Results, rung by rung

From the results table at the top of README.md; every number there is recomputed by a test from a committed file.

RungGameMethodResult
0Rock-paper-scissorsregret matching0.0009 chips/round exploitable after 100k rounds
1Kuhn pokervanilla CFR0.00063 chips/hand; game value −0.05555 against −1/18
2Leduc pokervanilla CFR0.011 chips/hand; game value −0.0874 against −0.0856
3Mini-drawmahalockstep MCCFR, four regret rulesLCFR wins: 0.061 chips/hand at 20M hands per seat
4Mini-drawmahaDeep CFR0.068 chips/hand at 10M hands per seat, where LCFR is at 0.126; still 0.068 at 20M, where LCFR has reached 0.061
nextFull DrawmahaDeep CFR at scalenot started; the range interface is built ahead of it

What each rung taught, in a sentence:

  • Rung 0 — the ledger converges to the uniform Nash in self-play and to pure paper against a 50%-rock opponent (+0.24/round against a best response of +0.25).
  • Rung 1 — CFR rediscovers Kuhn's bluffing ratio unprompted: the jack bluffs at α = 0.221 and the king value-bets at 0.661 ≈ 3α. Swap the two reach weights and the run still "converges", to a king that folds 18% of the time.
  • Rung 2 — a jack on a jack board checks and raises rather than leading, a check-raise nobody encoded.
  • Rung 3 — on 6.2M infosets, LCFR's linear average beats vanilla, CFR+ and DCFR at every checkpoint of the race; it is frozen as the release rung3-lcfr-2m. See Regret rules.
  • Rung 4 — at 1M hands per seat the best net is about a tenth as exploitable as the table (0.108 against 1.131), but the nets flatten past 1M hands and the table passes the ÷ pot net between 10M and 20M. One seed; see Status.
Exploitability by hands per seat for the four regret rules
Exploitability by hands per seat for the four regret rules

Overview · Dive deeper

Architecture

How the solver packages, the graders, the analysis commands, the committed figures and the site fit together.

The project is one Python package that solves games, one directory of committed outputs, and one website that draws those outputs. Data flows in one direction: Python computes, writes a file into the repository, and every downstream surface reads that file. Nothing on the site re-implements the solver, and nothing in a grader imports it.

Data flows one way: solve, grade, export, draw

  1. Rules — each game's game.py (for mini-drawmaha, the rules/ subpackage) defines the state machine, the infosets and the payoffs, with no solver in it.
  2. Solver — CFR (cfr.py, rungs 1–2), sampled lockstep MCCFR (minidrawmaha/solver/, rung 3) or Deep CFR (deepcfr/, rung 4) trains the regret ledgers and accumulates the average strategy.
  3. Grader — each rung's exploitability.py imports the rules alone and computes the exact best response against the solver's average.
  4. Analysis and export — entry points registered in pyproject.toml (kuhn-analysis, minidraw-analysis, deep-analysis, minidraw-deals, minidraw-ranges, …) write figures to figures/<rung>/ and JSON exports to the apps' public/ folders.
  5. Site — five Vite + React apps and five static pages read those files. The static pages are written by scripts/site/make_static_pages.py from the same committed numbers; Vercel builds and serves everything.

Heavy runs (the rung 3 race and the rung 4 arms) train on Modal or on a Mac, write checkpoints to Modal volumes or a local directory, and come back into the repository only as graded JSON under figures/ and as release assets on GitHub.

Components

ComponentWhereTechnologyRole
Shared ledgersrc/drawmaha_solver/regret_matching.pyPython + NumPyRegret matching: the one learning rule every rung trains.
Graded gamessrc/drawmaha_solver/{rps,kuhn,leduc}/PythonThe three toys, each with rules, solver, exact grader, analysis, terminal play; Kuhn and Leduc also an exploit server.
Mini-drawmahasrc/drawmaha_solver/minidrawmaha/Python + NumPy, multiprocessingrules/ (the game, 6,220,050 infosets), solver/ (lockstep MCCFR, packed table, regret rules), answer/ (frozen strategy, analysis, exports, release catalogue), and exploitability.py.
Deep CFRsrc/drawmaha_solver/deepcfr/Python + torch (optional deep extra)Nets, reservoirs, the collector, the train loop, the exact running average and the policy net.
Refereescripts/leduc/generate_referee.pyOpenSpiel (never a dependency)Writes tests/leduc/referee.json once, by hand.
Training launchersscripts/minidrawmaha/modal_rung3.py, scripts/deepcfr/modal_rung4.py, scripts/deepcfr/run_deep_cfr.pyModal, or localThe rung 3 race and the rung 4 runs. Started by a person, never by an agent.
Committed outputsfigures/PNG + JSONEvery analysis command's output, committed as a regression fingerprint.
Static pagesweb/cover/HTML generated by scripts/site/make_static_pages.pyHomepage, Rules, Benchmarks, Releases, Wiki; the archived Cover.
Appsweb/rung0-viz, rung1-viz, rung2-viz, rung3-viz, solver-vizVite + React + TypeScriptThe three benchmark demos, the research publication with Learn and Play tabs, the range explorer.
Deployvercel.json, scripts/site/vercel_build.shVercelBuilds the apps into public/, copies the static pages, fonts and theme.
Teststests/pytest; vitest per appThe suite is the CI: no workflow runs tests.

What the site reads, and from where

SurfaceRouteReads
Homepage, Benchmarks, Releases/, /benchmarks, /releasesthe README's results tables, figures/rung3/ and figures/rung4/ grades, answer/releases.py — baked in at generation time
Research, Learn, Play/research, /learn, /playweb/rung3-viz; the Play tab's deal pack under web/rung3-viz/public/deals/ (written by minidraw-deals)
Range explorer/solverper-decision-point exports under web/solver-viz/public/mini*/ (written by minidraw-ranges)
Kuhn and Leduc demos/rung1, /rung2the solves exported by kuhn-analysis --json and leduc-analysis --json; the Exploit tabs POST to a local exploit server
Rock-paper-scissors demo/rung0runs the 63-line ledger in the browser, the one place a learner is ported

Overview · Dive deeper

Quickstart

Install the package, run the suite, solve a game, play against it, and see the site locally.

Everything below runs from the repository root with uv. Python 3.12 or later; NumPy and matplotlib are the only heavy dependencies of the default install, and torch comes in only with the deep extra.

Install and run the suite

uv sync                  # Python 3.12+; add --extra deep for torch (rung 4 training only)
uv run pytest            # the whole suite; needs no torch, Modal, OpenSpiel or network

The full default suite takes about seven minutes on a laptop (tests/AGENTS.md). The fast loop is one rung's directory with -x:

uv run pytest -x tests/kuhn

Solve a game and redraw its figures

Each rung has an analysis command that solves it and rewrites its figures under figures/<rung>/. Rungs 0 to 2 run unseeded and deterministically, so a clean run leaves the working tree clean; a changed number shows up in git diff.

CommandWhat it doesTime
uv run rps-analysisrock-paper-scissors self-play and the biased-opponent run → figures/rung0/seconds
uv run kuhn-analysisKuhn to 100k iterations, 60 checkpoints → figures/rung1/, prints the answer sheetseconds
uv run leduc-analysisLeduc to 2,000 iterations → figures/rung2/about two minutes
uv run minidraw-analysisreads the committed grades and the frozen release → figures/rung3/ and answer_sheet.jsonabout two minutes
uv run minidraw-exploit-by-reach --release tabularwhere mini-drawmaha's exploitability lives, binned by reachminutes; downloads the release
uv run deep-analysisthe rung 4 price figure → figures/rung4/price.png and answer_sheet.jsonquick; reads committed grades

minidraw-analysis downloads the frozen strategy (rung3-lcfr-2m, 18 MB compressed) once and checks it against a SHA-256 pinned in src/drawmaha_solver/minidrawmaha/answer/strategy.py.

Play against the solver in the terminal

uv run rps-play          # against the regret-matching learner
uv run kuhn-play         # against Kuhn's equilibrium; seats alternate each hand
uv run leduc-play        # against Leduc's solved strategy
uv run minidraw-play     # against the frozen mini-drawmaha strategy

Every bot plays the average strategy, reads only its own infoset, and reveals both hands at the end, folds included.

Lock a strategy and watch CFR exploit it

The Kuhn and Leduc pages have an Exploit tab that posts your locked strategy to a local Python server and draws the counter-strategy CFR finds.

uv run kuhn-exploit-server            # the page POSTs locked spots to this
cd web/rung1-viz && npm install && npm run dev   # then http://localhost:5173/rung1/#exploit

leduc-exploit-server with web/rung2-viz works the same way.

See the site

An app alone:

cd web/rung3-viz && npm install && npm run dev

The whole site, wired the way Vercel wires it:

bash scripts/site/vercel_build.sh          # builds public/: five npm ci builds, then the static pages
uv run python scripts/site/serve_site.py   # http://localhost:4321

After changing anything a static page reads — including this wiki — regenerate the pages:

uv run python scripts/site/make_static_pages.py

Overview · Working in the repo

Onboarding

The rules every change in the repository follows, why each exists, and how to add a new surface.

The rules below come from the repository's AGENTS.md files: the root one, which holds what applies to every change, and nested ones loaded when you work inside a subtree (src/drawmaha_solver/, src/drawmaha_solver/minidrawmaha/, src/drawmaha_solver/deepcfr/, tests/, scripts/, web/, docs/). Each rule exists because breaking it fails silently: the code still runs and still produces a confident number.

The rules that hold everywhere

A grader never shares code with what it grades

Every rung's exploitability.py imports the game rules and nothing from the solver. A bug shared between the two would appear in both and cancel, and the meter would certify the thing that broke it. A review of a grader starts with its import block.

Inside a grader, the order of operations is fixed: sum over the hidden worlds first, then take the responder's max. A max inside the sum is an adversary who sees face-down cards. At Kuhn, a test pins that a peeking meter scores a genuine Nash equilibrium at 0.278 chips/hand. At mini-drawmaha, grouping the responder's max by the table's key instead of by physical state under-reports by about 0.012 chips/hand (src/drawmaha_solver/minidrawmaha/AGENTS.md). See Exploitability.

CFR's two reach weights are keyword-only

RegretMatcher.update(utilities, *, regret_weight, strategy_weight) in src/drawmaha_solver/regret_matching.py. Regret is weighted by the counterfactual reach π₋ᵢ (chance and the opponent), the average by the player's own reach πᵢ. Swapped, every rung still converges — to a wrong answer, with no warning. At Kuhn the swapped run folds the king to a bet 18% of the time. Never make them positional. See CFR.

Committed fixtures, figures and exports are regenerated, never hand-edited

tests/leduc/referee.json, tests/minidrawmaha/census.json, everything under figures/, the packs under web/rung3-viz/public/deals/ and web/solver-viz/public/mini/, and the copies under web/*/src/data/. Each names the command that writes it. The tests re-derive the numbers in them a second way, so a fixture that merely asserted itself would pass while the solver rotted. The static pages, this wiki included, are regenerated by scripts/site/make_static_pages.py.

CI runs no tests

The only GitHub workflow is an automated review. uv run pytest is the gate, and you run it yourself before every PR, plus npm test in any app you touched. The default suite must pass without torch, Modal, OpenSpiel or a network download.

Docstrings and the README are the design record

The reviewer re-derives the math a docstring states. A wrong claim in a docstring or in README.md is a bug, fixed in the same change as the code. Convergence numbers quoted in either are recomputed by a test from committed data, never transcribed from a session.

PRs branch from their predecessor's head and target main

Never stack a PR on another's branch, and never rebase a branch onto main to clean its diff: that deletes the modules it imports. Commit subjects state the finding:

  • feat(rung4): <the finding>
  • fix(web): <the finding>

Modal costs money

The launchers under scripts/ that run on Modal are started by a person, never by an agent on its own initiative. Deploy first (free), then run. See Modal.

Never git stash

Several agents work in parallel worktrees that share one stash stack; a bare git stash pop can take another session's changes.

Rules inside the code

WhereRuleWhy
Every rungLoops over a ledger's columns pair enumerate(legal_actions()) with the ledgercolumn 0 means call at the open and fold facing a raise; nothing else keeps them in step
Rungs 1–2Run unseededvanilla CFR enumerates the whole tree, so two runs agree bit for bit and figures/ is a fingerprint
Rungs 3–4Every sampled quantity is a function of (seed, iteration, seat, walk index), never of the worker countparallel must equal serial bit for bit; a run resumes exactly
Mini-drawmahaDo not simplify DCFR's deferred discount or CFR+'s per-bank floor to the textbook forma row can be banked twice in one traversal through the suit relabelling
Deep CFRtorch stays optional; torch tests importorskip itthe default suite runs on uv sync alone
Deep CFRThe oracle pin is the correctness gate: with OracleLearner the loop reproduces rung 3's LCFR table bit for bitif it breaks, the loop is wrong, not the test
Deep CFRA field added to DeepConfig after runs were saved goes in ADDED_LATER with the old valuea saved run must resume as the run it was
Tests--import-mode=importlib; test modules never import one anotherevery rung has its own test_game.py
TestsSlow work is gated by MINIDRAWMAHA_FULL_TABLE, MINIDRAWMAHA_FULL_GRADE, MINIDRAWMAHA_FULL_CALIBRATION, MINIDRAWMAHA_FULL_DEEPthe default suite stays fast
TestsLoosening a convergence threshold to pass is a finding to report, not a fixthresholds are seeded and justified in a comment
WebNo CFR in TypeScriptone implementation of the walk, in Python
WebFive chrome files are byte-identical across the five appstests/test_shared_chrome.py holds it

Adding a surface to the site

From web/AGENTS.md. A new app or static page touches:

  1. vercel.json — a rewrite for its route.
  2. scripts/site/vercel_build.sh — the rungs list for an app, or the page loop for a static page.
  3. tests/test_shared_chrome.py and tests/test_ui_standard.py — the APPS tuple for an app; for a page, PAGES there and in tests/test_cover_page.py.
  4. If the nav names it: NAV and the Surface type in site.tsx (copied to all five apps), and NAV in scripts/site/make_static_pages.py; then regenerate the pages.

Read web/DESIGN.md before touching any stylesheet, and afterwards run:

uv run pytest tests/test_ui_standard.py tests/test_shared_chrome.py tests/test_cover_page.py

Before you open a PR

  1. uv run pytest passes, and npm test in every app you touched.
  2. Any figure, export or static page your change affects is regenerated and committed.
  3. Docstrings and README.md say what the code now does.
  4. If you added a path to an AGENTS.md, it exists: tests/test_agent_docs.py checks every backticked path and every CLAUDE.md shim.
  5. The commit subject states the finding.

Overview · Working in the repo

Reading map

Which document in the repository answers which question, and which of them are living and which are dated records.

The repository keeps three kinds of writing, and knowing which is which saves reading the wrong one. Living documents are corrected in place when the code or a run changes them. Dated records say why the code looks the way it does on the day they were written and are never rewritten. Instructions tell an agent or a contributor what to do in a subtree.

Which file to read

When you want to…Read
Understand the whole project, rung by rungREADME.md, the long-form narrative, kept current
Know the rules for any changethe root AGENTS.md, then the nested AGENTS.md of the subtree you are in — summarised in Onboarding
Touch a stylesheet or a component's classesweb/DESIGN.md, the enforced UI standard
Edit the homepage, Rules, Benchmarks, Releases or this wikiweb/cover/README.md and docs/wiki/README.md; the pages are generated by scripts/site/make_static_pages.py
Continue rung 4 (Deep CFR)docs/superpowers/specs/2026-10-07-deep-cfr-m1-design.md, the PR ladder and its gates
Plan a rung 4 run, or try a regret rule other than lineardocs/notes/rung4-run-log.md (every arm and its grades) and docs/notes/regret-rules-dcfr-cfrplus.md (why DCFR and CFR+ lost twice)
Know why a design decision was madedocs/superpowers/specs/ for designs, docs/superpowers/plans/ for the dated plan each PR was built from
Change mini-drawmaha's deck, draw cap, betting tree or infoset keythe rung 3 plan, which lives outside the repo on Alec's machine (~/Desktop/Claude/poker/rung3-delta/rung3-delta-plan.md, §3 rules, §4 census)
Read the literature the solver draws onthe ML survey in paper/ (paper/drawmaha-ml-survey.pdf) and the nine notes in docs/research/ — see Literature
See how /solver's panes were placeddocs/research/range-ui-studies/
Read a code walkthrough of a merged PRdocs/notes/walkthrough-pr61-mps-refit.md, docs/notes/walkthrough-pr62-regret-rule-screen.md
Know what a committed figure is and which command writes itfigures/AGENTS.md and the rung's section of README.md
Run something on Modalthe launcher's docstring under scripts/, and Modal; it costs money

Living vs dated documents

KindWhereRule
LivingREADME.md, docs/notes/, docstrings, this wikiCorrect a wrong claim in place, in the same change as the code. Every number in docs/notes/ is read from a committed file under figures/, and the file wins when they disagree.
Dated recorddocs/superpowers/specs/, docs/superpowers/plans/Never rewritten to match what shipped. A change of direction gets a new document; the old one stays as the reason.
Finished referencedocs/research/, docs/writeups/rung0-rps/, paper/Not updated as the code moves.
Instructionsevery AGENTS.md (with its one-line CLAUDE.md shim)tests/test_agent_docs.py checks every backticked path exists.

Where this wiki sits

The wiki explains; it is not a second source of results. Its numbers are quoted from the README and from figures/, and it names where each comes from. When the two disagree, the committed file is right and the wiki is the bug.

Theory · Equilibrium and regret

Nash equilibrium in two-player zero-sum games

What the solver is trying to find, why it is the right target for heads-up poker, and how the repo measures distance from it.

Every game in this repo is two-player and zero-sum: whatever one seat wins, the other loses. In that setting there is one well-defined thing a solver can aim for — a Nash equilibrium, a pair of strategies where neither seat can gain by changing its own play while the other holds still. This page says what that object is, why it is the right target for heads-up Drawmaha, and what it does not promise, using the solved games the repo grades against.

A strategy is a probability for every action at every spot

Poker is a game of imperfect information: you see your own cards and the betting, not your opponent's cards. So a strategy cannot depend on the full state of the deal. It depends on an information set (an infoset, or spot): everything the acting player can see at a decision.

  • In Kuhn poker an infoset is the actor's own card plus the public betting history — K with history pb (check, bet) is one spot. Two deals that differ only in the opponent's card are the same infoset, and must map to one strategy entry.
  • A behavioural strategy σᵢ for seat i assigns, at each of its infosets I, a probability σᵢ(I, a) to every legal action a. Probabilities at one infoset sum to 1.
  • A strategy profile σ = (σ₀, σ₁) is one strategy per seat.

The expected payoff of a profile is uᵢ(σ): average over the deck's deals and both seats' action probabilities, in chips per hand. Zero-sum means u₀(σ) = −u₁(σ) for every σ.

Equilibrium means no unilateral improvement

u₀(σ₀*, σ₁*) ≥ u₀(σ₀, σ₁*) for every σ₀
u₁(σ₀*, σ₁*) ≥ u₁(σ₀*, σ₁) for every σ₁
  • σ₀\, σ₁\ are the equilibrium strategies.
  • The first line says: holding P1 at σ₁\, nothing P0 can switch to does better than σ₀\. The second says the same for P1.
  • Neither line mentions what the opponent should do — each is about one seat deviating alone.

Why this is the right target for heads-up poker

In a two-player zero-sum game the minimax theorem makes the equilibrium payoff a single number, the game value v. Every equilibrium earns exactly v for P0, and playing any equilibrium strategy guarantees at least v whatever the opponent does. That guarantee is the reason to want one: an equilibrium strategy cannot lose more than the game's built-in edge, against anyone.

The repo's own game values:

GameGame value to the first seatSource
Rock-paper-scissors0the uniform Nash (⅓, ⅓, ⅓)
Kuhn poker−1/18 ≈ −0.05556closed form; the solver reaches −0.05555
Leduc poker−0.08561OpenSpiel's sequence-form LP, frozen in tests/leduc/referee.json
Mini-drawmahaabout −0.0906 in self-play of the frozen answerthe README's rung-3 answer, read from the released rung3-lcfr-2m strategy

The negative values all have the same cause: the second seat acts last. In mini-drawmaha P1 acts last in both betting rounds and in the draw, and that is worth about a tenth of an ante.

The equilibrium can be a family, not a point

Kuhn poker shows that "the" equilibrium is often a set. The first player's equilibria form a one-parameter family over α ∈ [0, ⅓]:

  • with the jack, P0 bluffs (opens with a bet) with probability α;
  • with the king, P0 value-bets with probability 3α;
  • with the queen facing a check–bet, P0 calls with probability α + ⅓.

Every α in the range earns exactly −1/18. The solver lands wherever its iterates take it — α = 0.221 and K-open = 0.661 ≈ 3α in the README's run — and the ratio is discovered, never supplied. That is why tests/kuhn pins the nine infosets that are the same in every equilibrium and the two invariants (K-open = 3α, Qpb = α + ⅓), and never a hard-coded α.

Nash is unbeatable, not punishing

An equilibrium guarantees v; it does not collect what a weak opponent leaves on the table. The Kuhn exploit tab measures the gap. Take a P1 that plays equilibrium everywhere except two leaks — it calls a bet with the queen 90% of the time instead of a third, and folds the king 40% of the time:

chips/hand to P0
exact best response against the leaky P1+0.183334
CFR's counter-strategy, 20k iterations+0.183328
equilibrium play against the same leak−0.011502

Equilibrium play still loses a little (the seat is worth −1/18 at equilibrium, so the leaks have only partly offset it). The best response wins 0.195 chips/hand more. Equilibrium play doesn't make mistakes of its own, but it doesn't exploit yours either. Exploiting a known opponent means computing a best response, which is a different object (see Exploitability and Kuhn).

Units

QuantityUnit
Payoffs, game values, exploitabilitychips per hand, ante = 1 (per round for rock-paper-scissors)
The research page (/research)chips per hundred hands (the per-hand files × 100, in one place)
Full Drawmaha (planned)big blinds per hand, total across both halves of the pot

Theory · Equilibrium and regret

Regret matching

The learning rule at the heart of every solver in the repo — one ledger per spot, unchanged from rock-paper-scissors to mini-drawmaha.

Regret matching (Hart & Mas-Colell 2000) is how a player who repeats a game learns to play it without being told the answer. The repo uses one implementation of it, RegretMatcher in regret_matching.py, for every game. Rock-paper-scissors feeds it a payoff vector each round. Kuhn, Leduc and mini-drawmaha feed it one vector per infoset per visit. The rule itself is the same in all of them; only what feeds it grows.

The whole algorithm is three lines

  1. Record regret. After a visit, for every action a, add how much better always-a would have done than the mix you actually played.
  2. Play in proportion to positive regret. Next visit, play each action with probability proportional to its accumulated positive regret; uniform when none is positive.
  3. Answer with the average. The running average of the strategies played converges to equilibrium in two-player zero-sum games. The current strategy does not.
r(a) = u(a) − ⟨σ, u⟩
R(a) ← R(a) + r(a)
σ(a) = R⁺(a) / Σₐ′ R⁺(a′), R⁺ = max(R, 0)
  • u(a) is the payoff action a would have earned this visit. In RPS it is a column of the payoff matrix against what the opponent threw; in a tree it is a counterfactual value computed by the walk (see CFR).
  • ⟨σ, u⟩ = Σₐ σ(a)·u(a) is what the current mix earned on the same visit — the baseline every action is compared against.
  • r(a) is the regret of not having played a: positive if a beat the mix, negative if it did worse.
  • R(a) is cumulative_regret, the ledger's memory. σ is derived from it on demand and never stored.

Why positive regret, and why proportional

The rule puts probability where playing more would have helped. An action with negative cumulative regret has, on balance, done worse than the mix, and gets probability zero until it climbs back above zero. Among the positive ones, an action with twice the regret gets twice the probability. When nothing is positive there is no evidence to follow, so the ledger falls back to uniform.

What moves what. Feed in one large positive r for an action, and that action's probability jumps next visit. Feed in a long run of small negatives and its probability drops to zero, and stays there until enough positive regret accumulates to cancel the debt. Plain regret matching keeps that debt in full. CFR+, compared in Regret rules, floors it at zero instead.

The average converges; the current strategy cycles

The current σ chases whatever beat it last time, so against an adapting opponent it cycles: rock gets punished by paper, paper by scissors, and round it goes. The average of those strategies converges.

The ledger therefore keeps a second vector, strategy_sum, which banks σ after every visit. average_strategy() normalises it. In rock-paper-scissors self-play the average reaches (0.334, 0.333, 0.333) after 100,000 rounds, exploitable for 0.0009 chips/round, and its exploitability falls like 1/√T. Against a fixed opponent that throws rock half the time, the same ledger converges to pure paper and earns +0.24/round; the best response earns +0.25.

Self-play average strategy converges to the uniform Nash
Self-play average strategy converges to the uniform Nash

Two weights let the same ledger serve a game tree

In RPS every round counts the same. In a tree a spot is only reached some of the time, and the two vectors want different scales:

WeightMultipliesIsWhy
regret_weightthe regret r added to Rthe counterfactual reach π₋ᵢ: chance's and the opponent's probabilities of reaching the spot, leaving out your owna line you currently avoid must keep learning at full strength, or you could never discover it is good
strategy_weightthe σ added to strategy_sumyour own reach πᵢthe average must reproduce what you really played, so a spot you rarely steer into counts rarely

Both default to 1, so RPS callers never mention them.

The ledger contract

From mini-drawmaha on, many things touch a ledger: the walk, the regret rules, the checkpoint, the packed table that holds 6.2M of them in flat arrays. The module docstring fixes exactly which slots they may use:

SlotShapeHolds
cumulative_regret(n_actions,) float64R
strategy_sum(n_actions,) float64the run's primary average
extra_sums(k, n_actions) float64further averages of the same strategies at other weights (uniform, quadratic), banked for free
stamp(1,) int64the last iteration a lazily applied discount reached (DCFR)

All four can be views into a table's flat buffers (RegretMatcher.over), so writers update in place (+=, [:] =) and never rebind the attribute. Rebinding would silently cut the ledger loose from its table.

Theory · Equilibrium and regret

Counterfactual regret minimisation

How CFR turns a game tree into one regret-matching ledger per infoset, and why the two reach weights must never be swapped.

Counterfactual regret minimisation (CFR) runs regret matching at every infoset of a game tree at once, and its average strategy converges to a Nash equilibrium. RPS needs nothing more than the ledger: the opponent shows a move, and the payoff matrix already has a column of utilities for it. A tree doesn't give you that. What "bet with the king" is worth depends on everything both players do afterwards, so the utilities have to be computed, by walking the tree. That walk is the one new idea in Kuhn, in kuhn/cfr.py, and Leduc and mini-drawmaha keep its shape.

Two quantities travel through the walk in opposite directions

One CFR iteration is a recursive walk from the root. At every node it carries reach probabilities down and returns values up.

  • Down: reach. reach[p] is the product of player p's own action probabilities on the path so far, and nothing else. Chance is in neither entry: its probability travels separately and is multiplied into the counterfactual reach π₋ᵢ (chance × opponent) when a ledger is banked, as below.
  • Up: values. What the node is worth to each seat, as a plain conditional expectation: at a decision node, Σₐ σ(I, a) · v(child a). Values are not weighted by reach. Reach weights are applied once, when a ledger is banked.
  • The walk returns the value to both seats, so nothing is negated on the way up.

At an infoset I owned by player i, the walk has a value vᵢ(I, a) for each action and the mix's value vᵢ(I) = Σₐ σ(I, a)·vᵢ(I, a). It banks:

R(I, a) += π₋ᵢ(h) · ( vᵢ(h, a) − vᵢ(h) )
S(I, a) += πᵢ(h) · σ(I, a)
  • h is the concrete history being walked. Many histories share one infoset, because they differ only in cards player i cannot see. The bank adds across all of them.
  • π₋ᵢ(h), the counterfactual reach, is the probability that chance and the opponent bring play to h. Player i's own choices are left out on purpose: it is the reach i would have if i played to get there.
  • πᵢ(h) is player i's own reach, the probability i's own choices lead to h.
  • R is cumulative_regret, S is strategy_sum. The average strategy is S(I, ·) normalised.

Why the regret uses the opponent's reach and the average uses yours

Regret asks "had I played to get here, how much would always-a have beaten my mix?" Weighting it by your own reach would silence a line you currently avoid: its regret would be multiplied by roughly zero, and you could never find out it is good. The counterfactual weight keeps every line learning at full strength.

The average asks a different question: "what did I actually play?" A spot you almost never steer into should barely affect your average, so it is weighted by your own reach.

Why chance is in π₋ᵢ, and why it does not matter in Kuhn

The counterfactual reach is defined to include chance's probability. In Kuhn every deal is equally likely (1/6), so the chance factor scales every regret increment by the same constant. Regret matching reads only the ratio of positive regrets, so σ, the reach and the average are unchanged; the repo measures the difference at 9e-15 after 5,000 iterations, which is floating-point rounding. The factor is still carried because once a game deals non-uniform chance it stops being a common scale. It is also why the ledgers hold numbers 6× smaller than Neller & Lanctot's reference trace, whose pseudocode leaves the factor out.

The average converges; the current iterate never does

Kuhn's exploitability, from the README:

iterationsaveragecurrentgame value
100.091140.24191−0.03597
1000.023300.25000−0.05472
1,0000.006500.27885−0.05518
10,0000.001490.20359−0.05553
100,0000.000630.24649−0.05555

Two lessons, and the repo builds on both:

  • The average falls steadily; the current strategy sits around 0.2 forever. At 1,000 iterations it is briefly worse than at 10. Vanilla CFR's current iterate cycles. Every grader grades the average, and the live surfaces serve averages.
  • Game value is the softer grader. At 100 iterations the game value is within 0.0009 of −1/18 and looks solved, while the strategy is still 15× more exploitable than it will be at 10,000. Leduc repeats the lesson on a 460-times-larger tree: by 100 iterations its game value is within 0.004 of −0.08561, while the strategy is five times more exploitable than at 2,000.
Only the average strategy converges; the current one cycles forever
Only the average strategy converges; the current one cycles forever

The average falls along the 1/√T guide, which is CFR's convergence rate: exploitability of the average after T iterations is O(1/√T).

What changes at Leduc: chance in the middle, ragged action sets

Leduc keeps the walk and adds two things:

  • A chance node mid-hand. The board card comes after the first betting round, so the walk meets chance inside the tree rather than enumerating every deal at the top. A jack is trash until a jack hits the board, then it's the best hand in the game.
  • Spots with different action sets. Facing a raise you may fold, call or raise; at the open you may not fold. A ledger is 2 or 3 wide depending on where it sits, and column 0 means call at the open but fold facing a raise. Every loop pairs enumerate(legal_actions()) with the ledger's columns.

288 infosets instead of Kuhn's 12. One iteration walks 9,450 nodes in about 50 ms. Vanilla CFR enumerates the whole tree and never samples, so two runs agree bit for bit, and figures/rung1/ and figures/rung2/ double as regression fingerprints.

Locking a seat turns CFR into a best-response finder

The same walk can hold some infosets fixed. At a locked infoset it plays the supplied probabilities and banks nothing; elsewhere it learns as usual. The locked probabilities still advance the reach, so they still enter the learner's π₋ᵢ. The learner is never told the opponent is fixed. It is simply weighted by how often that opponent brings it to each spot. Minimising regret against an opponent that can't move back converges to a best response, not to Nash. Lock a whole seat and CFR becomes an exploit finder with no new algorithm. That is the Exploit tab at /rung1#exploit, graded against the exact best response (see Exploitability).

Past Leduc, the full walk does not run

Mini-drawmaha has 85.1 billion decision nodes and 6,220,050 infosets. A full walk per iteration is impossible, so rung 3 samples the walk instead. The regret weight then becomes the sampling frequency, which is the subject of MCCFR.

Theory · Equilibrium and regret

Exploitability and the exact best response

The one number every result in the repo is graded by — what a perfect adversary wins — and the rules that keep the grader honest.

In self-play, training loss doesn't tell you how close you are. Two players improving against each other can drift to somewhere comfortable for both and beatable by anyone else, and still report a fine game value to each other. The honest question is how much a perfect adversary would win. That number is exploitability. Every result in the README, on /benchmarks and on /research is exploitability, computed exactly and never estimated.

Exploitability is the mean of the two best-response values

exploitability(σ) = ( BR₀(σ₁) + BR₁(σ₀) ) / 2
  • BR₀(σ₁) is the most seat 0 could earn, in chips per hand, against seat 1 frozen at σ₁. BR₁(σ₀) is the same for seat 1 against seat 0's half.
  • Freezing one side and re-optimising the other removes the cancellation that makes self-play numbers flatter themselves.
  • At a Nash equilibrium the two terms are v and −v, the game value and its negative, so the sum is exactly 0. Any slack pushes it above zero.
  • Halving averages over the two seats, so one number grades the whole profile.

What the number does and does not mean

Exploitability is not what the strategy loses to any one opponent. It is the average of the two best responders' edges. For scale, uniform random play on mini-drawmaha is exploitable for 4.842 chips/hand (figures/rung4/answer_sheet.json). The frozen tabular answer is at 0.061346, with BR₀ = −0.034421 and BR₁ = +0.157114. P0's best response is still negative: even a perfect P0 loses against the solved P1, because the second seat's edge is built into the game.

It is also the stricter grader. In Kuhn at 100 iterations the game value is within 0.0009 of −1/18 while the strategy is still 15 times more exploitable than at 10,000 (see CFR).

Sum over hidden worlds first, then take the max

The responder plays from infosets, and must choose one action per infoset without seeing the cards hidden from it. So the best-response value at an infoset I of seat i is:

BRᵢ(I) = maxₐ Σ_{h ∈ I} π_c(h) · π₋ᵢ(h) · v(h · a)
  • h ∈ I ranges over the hidden worlds the infoset stands for: the deals that look identical to the responder.
  • π_c(h) · π₋ᵢ(h) weights each world by how often the deck and the frozen opponent produce it.
  • v(h · a) is what action a leads to in that world, with the responder playing its best responses deeper down.
  • The max is outside the sum. Score each action across all worlds, then pick one. The responder's own reach is the same for every world in one infoset under perfect recall, so it is left out: it would only rescale the argmax's inputs uniformly.

A grader never shares code with what it grades

Every game's exploitability.py imports the game's rules and nothing from the solver. If the walk and the meter shared a routine, a bug in it would appear in both and cancel, and the meter would certify the thing that broke it. Every result's ground truth is independent of the solver:

GameHow the best response is computedChecked against
Kuhnall 2⁶ = 64 pure strategies of each seat enumerated; mixed play and hidden cards handled by weighting, so the values are exact expectationsthe closed-form equilibrium family
Leducone max per infoset, deepest infosets first (tag, bucket, max), since a seat has about 6 × 10⁵¹ pure strategiesan OpenSpiel solve, frozen in tests/leduc/referee.json
Mini-drawmahaa walk over the 141-point public tree carrying vectors over private statesnothing outside: no engine plays the game, so this walk is the ground truth

Why enumerating pure strategies keeps Kuhn's adversary honest

A pure strategy names one action per infoset, fixed before any card is dealt. "Maximise per deal" can't even be written in that form. That is what made the 64-strategy enumeration a safe grader at Kuhn. At Leduc there are too many to enumerate, so the same structure is spelled out as the sum-then-max recursion.

Mini-drawmaha's meter walks the public tree with vectors

85.1 billion decision nodes rule out tagging every world. The meter moves along the public tree instead (141 public decision points), carrying one vector over the opponent's private states and one over the responder's:

v[b, r] = Σₕ w[b, h] · M_b[r, h] at a terminal, board b
v[b, r] = maxₐ vₐ[b, r] at a responder spot
  • r is one of the responder's private states, its physical (hole, thrown card): 455 × 16 = 7,280 entries.
  • h is one of the opponent's private states, indexed the same way.
  • w[b, h], the reach vector, says how often the deck and the frozen opponent bring exactly h here on board b.
  • M_b[r, h] is what r is paid against h on that board (the split-pot showdown), zero where the two hands share a card.
  • b is the physical board, a batch axis: 15 after the first card, 210 ordered pairs after the second.

One entry (b, r) is a responder infoset, and the opponent's hands were already summed inside it by the first line. So one max per entry is sum-first-max-second, and the peeking bug can't be written: no vector ever indexes the responder's state and the opponent's hand together. Vectors are indexed by physical cards, not canonical keys, because the two seats' canonical keys for the same deal usually sit in different suit frames and could not be dotted together.

Where the exploitability lives: a per-spot decomposition

A best responder's edge telescopes over its own decisions as one-step deviations. At each infoset, multiply how often the strategy's own play reaches it by what the best response earns there over the strategy's mix. Every spot gets a share, and the shares sum to the grade. uv run minidraw-exploit-by-reach --release tabular bins the 6.2M spots by how often self-play reaches them (figures/rung3/exploit_by_reach.json, which also checks the sum):

  • Per visit, a table row rots as the sampler sees it less. The commonest band leaks 0.4 chips per hundred visits, a spot reached once in ten million hands about 20, and the rarest band about 67. 72% of the total sits in round 2.
  • A net leaks the other way. On the two Deep CFR averages (figures/rung4/exploit_by_reach_deep-*.json) the net leaks about three times the table per visit at the commonest spots and a fifth of it at the rarest. A table is exact where it has samples and bleeds where it doesn't. A net fills the rare rows from the common ones, and pays for it in the common rows.
Where the exploitability lives
Where the exploitability lives

Theory · Scaling up

Monte Carlo CFR, external sampling and lockstep

How rung 3 keeps CFR's walk but samples the deck and the opponent, why the regret weight becomes exactly 1, and how W workers share one table.

Vanilla CFR walks the whole tree on every iteration. Leduc's tree is small enough for that. Mini-drawmaha's has 85.1 billion decision nodes, so a full walk isn't slow, it's impossible. Monte Carlo CFR (MCCFR) keeps the walk and samples the parts of it that are expensive to enumerate. Each iteration then costs one sampled hand instead of the whole game, and the expected update is still CFR's. The repo uses external sampling (mccfr.py), parallelised as lockstep (lockstep.py).

External sampling: sample what is outside the traverser

One traversal per seat per iteration, P0 then P1, each from a freshly dealt root. The traversing seat is the one whose regrets are updated. Rung 2's recursion stays — terminal, chance and decision branches, utilities indexed by position in legal_actions(), ledgers banked on the way back up — and two branches change:

NodeVanilla CFRExternal sampling
Chance (the deck)average over every outcomepick one outcome by its probability
The opponent's spotenumerate every action, weighted by σpick one action from the opponent's current mix
The traverser's spotenumerate every actionenumerate every action, because regret compares the actions and a comparison needs all of them

The weight is the sample

In vanilla CFR the regret bank is multiplied by π₋ᵢ · π_c, how often the deck and the opponent bring play here. Under sampling, that product is exactly the probability the walk arrives here at all. So the update lands with that frequency instead of being multiplied by that coefficient, and the regret weight at the traverser's spot is exactly 1:

at my spot: R(I, ·) += 1 · ( u − ⟨σ, u⟩ )
at their spot: S(I, ·) += t · σ(I, ·)
  • u[k] is what the traverser's k-th action earned down the sampled path; ⟨σ, u⟩ is what its current mix earned on the same path. Their difference is one unbiased sample of the counterfactual regret.
  • S is banked at the opponent's spots, where the walk samples from σ anyway. Arriving there already happened with the opponent's own reach, so adding σ once per arrival reproduces the own-reach weight. The traverser's own spots bank no strategy: they were enumerated, not played, and counting them would credit every action with having been chosen.
  • t makes the average linear: iteration t's strategy counts t times as much, on the premise that later iterates are better.
Why the linear average is not free under vanilla regret

On Leduc under vanilla regret, the uniform averaging column beats the linear one at every length: 1.5–2.7× at 10k iterations, and about 4× by a million (0.012 against 0.050). The docstring calls this a measurement, not yet an explanation. It is consistent with vanilla CFR's regret bound applying to the uniform average (linear averaging is backed when the regrets are weighted to match, as in LCFR and CFR+), and with a t-weighted average having a smaller effective sample. Under the other three rules the linear column is the better one. On mini-drawmaha the race found linear beats uniform for both vanilla (8.6 against 15.5 per hundred hands) and CFR+ (24.8 against 36.5). Extra averaging columns cost nothing to keep, because nothing reads an average while training.

Lockstep: W hands per seat against one frozen σₜ

A one-hand walk writes as it goes, so two processes can't share a table. A lockstep iteration walks W hands per seat (one per worker) and writes nothing while it walks:

  1. Freeze σₜ. Each of the W workers walks one sampled hand for seat s and records what it would write.
  2. Once all W are in, every touched row is banked once, with the sum of its workers' regrets, and the strategy mass is added. The rows are banked in a fixed order.
  3. The other seat does the same against the updated regrets (alternating updates).

The W walks of one seat are a phase. This is MCCFR with W samples per iteration. In the race W = 10, so one lockstep iteration is 10 hands per seat, and 2,000,000 iterations is 20M hands per seat.

Why lockstep is the right shape

  • No races. The table is read-only during a phase, so the walks can't race. The one write per row happens in a fixed order, so the float sums are the same however the walks were scheduled. parallel.py runs the W walks in W processes; lockstep.py runs them one after another and is the reference the processes must equal bit for bit.
  • Every regret rule is exact. CFR+ floors a row once per iteration, on the total of its W sampled regrets. That is the paper's rule with that total as the iteration's regret, not W separately floored one-hand banks.
  • Textbook semantics. A walk that reaches the same row twice (suit relabelling can do that) reads σₜ both times, instead of seeing its own first bank the second time.

Counter-based randomness makes resume exact

Worker w's walk for seat s at iteration t draws from default_rng([seed, t, s, w]). A run carries no RNG state, so resuming from any checkpoint is the same as never having stopped. Deep CFR inherits the rule: every draw is a function of (seed, t, seat, walk index), never of W, so arms with different worker counts see identical hands.

The memory: 6.2M ledgers in four flat arrays

6,220,050 infosets, each a RegretMatcher 2–4 wide, would be millions of Python objects. packed_table.py stores them as four flat arrays — the ledger contract's cumulative_regret, strategy_sum, extra_sums, stamp — and hands out ledgers as views into them, which is why ledger writers update in place and never rebind.

Theory · Scaling up

Regret rules — why linear weighting won, twice

Vanilla, LCFR, CFR+ and DCFR differ only in what a visit banks. Under sampling, the two rules that test a regret's sign lost to linear weighting, tabular and neural alike.

Once the walk has measured one sample of counterfactual regret, r = u − ⟨σ, u⟩, at one of the traverser's spots, something has to decide how r enters the stored total. That decision is the regret rule, and it is the only thing that differs between the four algorithms the repo raced. In the code a rule is a function bank(ledger, r, t) and nothing else (regret_rules.py). The average isn't the rule's business: every rule banks the same played strategies into the same averages.

The headline: on mini-drawmaha the two rules designed to beat linear CFR lost to it by 3–4× tabular and by about 17× neural. Both times the cause was the same. A rule that depends on a regret's sign, or runs on its own clock, stops meaning what its paper meant once rows are sampled. LCFR survives because its weight w(t) = t depends on neither. Linear weighting is the repo's default for anything sampled.

The four rules, stated on a running total R

RuleBankIn words
vanillaR ← R + revery iteration counts the same
LCFR (linear CFR)R ← R + t·riteration t counts t times iteration 1, so the early, noisy iterations fade; after T iterations the first tenth carries 1% of the weight. Nothing is erased and no sign is tested.
CFR+R ← max(R + r, 0)the floor is on the running total: a bad action stops piling up debt and can come back the moment it turns good
DCFR (α = 1.5, β = 0)R ← R + r, then positive entries × t^1.5/(t^1.5 + 1), negative entries × ½positive regret fades fast early (factor 0.5 at t = 1, 0.99 by t = 22, 0.99997 at t = 1,000); negative regret halves every iteration — a softer CFR+

All but vanilla are from Brown & Sandholm 2019 (arXiv 1809.04040). CFR+'s other half, alternating updates, is already how every iteration runs: P0 traverses, then P1.

DCFR discounted lazily, and why that is exact

As written in the paper, DCFR discounts every regret of the traverser on every iteration, visited or not. Under sampling a row is visited once in thousands of iterations, and touching all 6.2M rows per iteration is out of the question. So a row is discounted only when it is next visited, with all the owed iterations at once. That is exact because an unvisited row receives r = 0, and multiplying by a positive factor never changes a sign. Each entry stays positive or negative over the whole gap, so the per-sign factors simply multiply:

positive entries: ∏ₖ k^1.5 / (k^1.5 + 1) = exp( L(t−1) − L(s−1) )
negative entries: (½)^(t − s)

s is the row's stamp, the last iteration it was brought up to. L is the running log of the positive factor. The lazy rule equals the eager one bit for bit.

Rung 3, tabular: keeping negative regret wins by 3–4×

Each rule trained for 2,000,000 lockstep iterations (20M hands per seat) on Modal, graded by the exact walk. Chips per 100 hands, linear average, seed 0:

hands per seatvanillaLCFRCFR+DCFR
1M114.5113.1156.4156.0
10M16.712.636.937.0
20M8.66.124.825.0
Keeping negative regret wins
Keeping negative regret wins

DCFR collapsed onto CFR+ because of its clock

The implementation is correct. The problem is what "per iteration" means once rows are sampled. A row visited once every g iterations pays g iterations of discount between two pieces of evidence:

negative regret at the next visit = (½)^g × what was banked
  • g, the gap between visits, was large. At 2M iterations the median row was last visited about 37,000 iterations earlier (the reach-weighted median gap was 7,006).
  • Float64 reaches exactly zero after about 1,075 halvings. So every negative regret was gone before its row came round again, which is exactly CFR+'s floor.
  • The positive side failed too. DCFR's positive discount does its work in the first few dozen ticks of the global clock, and 73% of rows were first reached after that window, so their positive regret was never discounted.

In the limit of rare visits DCFR(α, β, γ) → DCFR(∞, −∞, γ), which Brown & Sandholm identify with CFR+. Both runs used the same linear average, and they were within 0.7% at every checkpoint. They aren't identical, though (24.8 against 25.0): DCFR also discounts positive regret.

LCFR was immune for two reasons

  • Sign-blind. Weighting by t treats positive and negative regret alike, so idle time can't tilt the balance between an action's good and bad history.
  • Scale-free. w(t) = t has no built-in time scale. A rare row's updates arrive at t ≈ j/p, and the ratio of two of its weights is tᵢ/tⱼ ≈ i/j whatever the visit rate p is. A rare row gets the same linear-in-evidence weighting as a frequent one.

Rung 4, neural: per-sample DCFR and CFR+ lose by about 17×

Deep CFR stores no running regret. Each refit learns the t-weighted mean of the regret samples in the reservoir, and regret matching reads that mean's sign pattern. So the only place a rule can act is on the individual samples. The screen (train.rule_targets, PR #62) rescales each stored sample by the weight the rule would give it at iteration now, using that sample's own sign:

DCFR, positive sample from iteration s: ∏ₖ₌ₛ^(now−1) k^1.5 / (k^1.5 + 1)
DCFR, negative sample from iteration s: (½)^(now − s)
CFR+: positive samples keep weight 1; negative samples count only when s = now

Exact average, chips per hand, 4 × 256 net, 4,000-step refits, seed 0 (figures/rung4/rule_*-256x4.json):

hands per seatlinearDCFR screenCFR+ screen
200k (t = 20)0.2942.2272.858
500k (t = 50)0.1732.8703.092

Both get worse with training (DCFR 1.51 → 2.23 → 2.87 over t = 10, 20, 50); uniform random is 4.84.

A sample's sign is not its cell's sign

figures/rung4/rule_sign_flips.json measures the mechanism. By t = 50 the screen keeps only 4.1% (DCFR) or 2.1% (CFR+) of the memory's negative regret mass, against 82–100% of its positive mass. A genuinely bad action still draws some positive samples from lucky walks. Once the negatives are discounted away and the lucky positives aren't, the fitted mean turns positive. 36–38% of seat 0's cells that are negative under linear weights turn positive by t = 50, up from 8–11% at t = 10. Regret matching plays every positive cell, so these are bad actions the net is being taught to play, and there are more of them every iteration. That fits the rising curve, but no ablation has isolated it as the only cause.

A worked case shows the shape. Two actions' samples: b = {+1, −3, +1, −3}, bad on average at −1 per sample, and c = {+0.5, +0.5, +0.5, +0.5}, mildly good. Clip per sample and both fit to +0.5, so regret matching plays them equally. CFR+ on the totals would floor b at zero and play only c. A per-sample rule inflates the noisy action, and sampling produces noisy actions.

Theory · Scaling up

Deep CFR — the regret table replaced by a network

Why full Drawmaha needs a net instead of a table, how rung 4 swaps one in without touching the walk or the grader, and what it has measured so far.

A table needs one row per infoset. Mini-drawmaha's 6.2M rows fit in memory; full Drawmaha's don't come close. Deep CFR (Brown, Lerer, Gross & Sandholm 2019) runs CFR with the regret table replaced by a neural network that predicts each spot's regrets from features of the spot. Rung 4 (deepcfr/) puts it on mini-drawmaha, a game where the exact answer and an exact grader already exist. The cost of the net can then be read off at every point:

price = exploitability(Deep CFR's average) ÷ exploitability(LCFR's linear average), at equal hands per seat
  • The numerator is graded by the same exact meter as rung 3 (see Exploitability).
  • The denominator is the rung-3 race's LCFR at the same number of sampled hands.
  • Equal hands: Deep CFR at K = 10,000 walks per seat per iteration has seen at iteration t the same hands LCFR had at lockstep iteration 1,000·t (LCFR's W = 10). So the eight marks t = 10, 20, 50, 100, 200, 500, 1,000, 2,000 land exactly on the race's 100k–20M checkpoints.
  • There is no pass mark. The ratio is recorded and read.

One iteration is rung 3's lockstep with the regret half swapped out

For each seat in turn (train.py):

  1. Materialise. Write both seats' nets into the packed table's regret slots (materialise.py). The walk then plays regret matching over the table as before, with RegretMatcher.strategy() unchanged.
  2. Average. Let the exact average see the iteration's strategy while it sits in the table (average.py).
  3. Walk. K hands on W processes (collect.py), returning the traverser's regret samples and the opponent's strategy samples as rows.
  4. Remember. Merge the rows into fixed-size reservoirs, 50M rows each (reservoir.py).
  5. Refit from scratch. Throw the seat's net away and fit a fresh one to its whole memory (fit.py).

Materialise writes eq. (3) into the slots

Deep CFR plays regret matching over the net's outputs when any output is positive, and the single best action when none is (eq. 3 of the paper). Plain regret matching would read all-negative as uniform, so materialise writes the verdict in a form the ledger already reads correctly:

Net outputs at a spotWritten to the slotsThe ledger then plays
some positivethe outputs as they areregret matching (negatives clipped)
all ≤ 0, not all equala one-hot at the argmax, shared by tiesthe best action(s)
all equal (an untrained net, an exact tie, no net)zerosuniform

It is 141 batched forward passes, one per public point, each written with one slice assignment.

The net and its loss

The net is a plain MLP (net.py): 136 input features describing what the actor can see, then hidden layers, then 7 signed outputs, one per action in a fixed order, so a column means the same thing at every spot. The head is linear because the outputs stand in for signed regrets; a ReLU or softmax would throw away the sign materialise needs. The default is 3 × 64 (17,543 parameters); the graded arms went to 3 × 128 and 4 × 256.

The refit minimises a t-weighted squared error:

loss = mean over rows of ( t / mean t ) · Σ_legal a ( net(I)[a] − target[a] )²
  • target is a sampled regret r = u − ⟨σ, u⟩ at row I's legal columns. Padding columns are masked out before the subtraction.
  • t / mean t is Linear CFR's weight carried into the fit: early iterations' regrets were sampled against near-random strategies and say the least about the game. Dividing by the batch's mean t keeps the loss on one scale as the run lengthens.
  • Squared error means the net learns the t-weighted mean regret of each spot, which is what regret matching needs.
  • Adam, gradient norm clipped at 1.0, rows drawn uniformly with replacement from the memory.
Why a reservoir, and why Algorithm R

Over 2,000 iterations the walks offer about 280M advantage rows per seat, and the memory keeps C = 50M. The kept rows should be a uniform sample of everything offered, so no iteration is favoured by when it happened. Algorithm R does that in one pass: the first C offers fill the slots, and offer g ≥ C draws j uniformly from [0, g] and overwrites slot j if j < C. By induction every offer is resident with probability C/(g+1). The draws come from a counter-based stream seeded by (seed, t, seat, memory), and the offer count n is saved with the memory, so a resumed run merges exactly as the uninterrupted one would. A row is 24 bytes: row (int32), t (int32), four float32 targets.

Three averages are graded, because they answer different questions

CFR's guarantee is about the average strategy, and Deep CFR throws the table away every iteration, so nothing keeps the average for free. Rung 4 keeps it two ways and grades both, plus the current iterate:

ObjectWhat it isWhy grade it
Exact averageΣₜ t · own_reachₜ(I) · σₜ(I, a), summed per column while each iteration's strategy sits in the table (Single Deep CFR)CFR's actual answer: no sampling or fitting error in the averaging itself
Π, the policy neta fresh net fit on the policy memory, the (row, t, σ) rows the opponent's walks copiedthe only average a game without a table can keep, so it is what full Drawmaha will use
Current iterateregret matching over the materialised tableCFR never promises it converges; graded to see by how much

The exact average's weight is the textbook own reach: the sum, over the histories a key stands for, of seat q's own action probabilities on the way there. The opponent's probabilities are left out because the average is of q's play, not of how the opponent happened to be playing. A post-draw key has three pre-draw predecessors, all of which count, so own reach is pushed through a predecessor table over the 141 public points.

Π gets the same weighting by regression. At a spot with memory rows σ₁, σ₂, σ₃ at weights t₁, t₂, t₃, the output y minimising Σ tᵢ·(y − σᵢ)² is Σ tᵢσᵢ / Σ tᵢ, the t-weighted mean, even though no single row held it. Reach is carried by the sample counts, since walks visit a spot in proportion to how often play arrives. Π carries two errors the exact average doesn't: the reservoir's sampling and the fit's. The exact average beat Π at every graded mark of every linear arm.

Dividing each target by the pot spreads the fit's accuracy by chip cost

One net fits every spot with one accuracy budget, and the loss decides where it goes. A regret is in chips, so a spot's regrets are about as large as its pot, and a relative error ε there costs (ε · pot)² per sample. Raw squared error therefore weighs a spot by how often × pot². The 52-chip spots count 676 times the 2-chip spots they are reached as often as, and the ante spots, where every hand begins, are fitted last.

Divide every target at a spot by pot^k before the fit:

  • It's free for the regrets themselves. The divisor is shared by every action and every sample at the spot, so the spot's mean regret is scaled by one constant. Regret matching reads only signs and ratios, so it plays the same σ from scaled regrets as from raw ones (pot_scaled in train.py; a test pins the invariance). Only the net's errors, and so the σ it actually plays, move.
  • What k does. The weight left is how often × pot^(2 − 2k). k = ½ weighs by how often × pot, the chips a σ mistake at that spot actually costs, and is the default. k = 1 weighs every spot alike.

Measured on one seed, 4 × 256 at 16,000 steps and lr 5e-4: at 2M hands raw 0.115, ÷ √pot 0.086, ÷ pot 0.084 chips/hand. The two pot weightings are tied within the noise.

The net leads early, flattens, and the table passes it

The ÷ pot arm, run to the race's endpoint (figures/rung4/pot_1-256x4-s16000-lr5e-4.json), exact average in chips per hand:

hands per seat1M2M5M10M20M
Deep CFR, exact average0.1140.0840.0710.0680.068
LCFR at the same hands1.1310.6300.2640.1260.061
  • Early, the net generalises. At 1M hands it is a tenth as exploitable as the table: one learned function covers spots the table hasn't banked yet. Equal hands isn't equal compute, though. Two large refits make an iteration cost tens of seconds to minutes, and the refits are about 95% of the time.
  • Then it flattens. The last 10M hands, 49 hours of training, moved it from 0.0679 to 0.0685. The table, still on its ~1/√N line, passes it between 10M and 20M hands. This is a measurement on one seed and one net, not an extrapolation.
  • Width was the lever. 3 × 64 → 3 × 128 cut the 1M-hand grade by 36%. Depth did 2–9%, and doubling the refit 12–24%.
Deep CFR ÷ LCFR at equal hands
Deep CFR ÷ LCFR at equal hands

The exploitability decomposition shows where the two differ. The net leaks about three times the table per visit at the commonest spots and a fifth of it at the rarest.

Theory · Scaling up

Abstraction — how mini-drawmaha shrinks full Drawmaha

Why the repo solves a smaller game first, what was cut and what was kept, and the one lossless collapse it allows.

Every CFR-family solver needs something per information set, and real poker has astronomically many. The literature's bridge is abstraction: build a smaller game whose solution says something about the big one. Mini-drawmaha is this repo's abstraction. It is not a bucketing of full Drawmaha's hands. It is a separate, smaller game that keeps every idea full Drawmaha adds over Leduc. It's small enough to solve exactly, so a method can be graded on it before it meets a game where no grade is possible.

Mini-drawmaha keeps the ideas and cuts the size

Full Drawmaha (the target)Mini-drawmaha (rung 3–4)
Deck52 cards15 cards: ranks 2–6 in three suits
Hole cards5 each3 each
Startbomb pot: everyone antes, no preflop bettingthe same: ante 1 each, no blinds, no preflop betting
Board before the drawflop, 3 cards1 card
Betting roundsthree (flop, turn, river)two
The drawdiscard 0–5 and replace; draw-1 is dealt face up (keep it or take the next face down)throw at most one, replaced face down; P1 learns only whether P0 drew
Inner half of the potbest 5-card hand of the hole cards alonebest 3-card hand of the hole cards alone
Outer half of the potOmaha: exactly 2 hole + 3 boardexactly 2 hole + both board cards
Bettingpot-limit, {fold, check/call, pot}, 100bb stacksthe same action set, 26-chip stacks
Sizefar beyond any table6,220,050 infosets, 85.1 billion decision nodes

What survives the shrink is the list of things Leduc does not have, each of which breaks a naive solver differently:

  • The chance node depends on the action. Stand pat and the deck never acts; throw a card and it deals one. A throw is two nodes, the decision and then the replacement.
  • Thrown cards leave the deck for good, and only their owner knows which. remaining_deck() is derived from the state, never counted down, so a discard can never be redealt.
  • The pot pays twice, by two rules over the same cards. Dealt trips is the inner half's top category and can bring only a pair outside. Strength is two-dimensional.
  • Suits carry information. Both halves have flushes, so Leduc's trick of throwing suits away is no longer lossless.

Every size number was chosen for a reason

  • Three suits because with two, a rank holds two cards and the inner half loses trips entirely.
  • Five ranks because at four, a run of four distinct ranks is the whole rank set, and high card and plain flush would have nowhere to exist.
  • 26-chip stacks because from the 2-chip ante pot an uncapped pot-raise war commits 2 → 8 → 26. The stack buys a bet, a raise and an all-in shove (truncated to the 25 left behind the ante), and nothing deeper. Round 1 then has 13 complete lines, and round 2 opens in one of four (pot, stacks) states, so the infoset key needs no explicit chip counts.
  • Hand rankings by measured rarity in this deck, not 52-card convention. A four-card straight (25.11%) ranks below two pair (16.38%) and trips (9.89%), because a run of four out of five ranks is easy to make. Outer high card is the rarest outer result (0.74%) and still ranks last, by definition.

The one collapse allowed is relabelling suits, jointly

Within the game, the only abstraction the solver uses is lossless: two positions that differ only in which physical suit is which are the same position. canonical() relabels the three suits to the lexicographically smallest image of the whole visible picture. That takes the 455 dealt hands down to 95 classes, a 4.79× saving that costs nothing.

The same lossless step scales to the target: the survey's notes brute-force full Drawmaha's 2,598,960 five-card hands down to exactly 134,459 canonical classes (19.3×) (docs/research/05-abstraction-bucketing.md).

Counting infosets by factorising the key

85 billion nodes can't be walked to count infosets, but an infoset factorises into two independent halves:

infoset = public point × private key
6,220,050 = Σ over the 141 public points of (private keys at that point's shape)
  • A public point is the betting so far, both draw counts, how many board cards are out, and whose turn it is. There are 141, and none depends on a card, so one representative deal walks them all.
  • A private key is the actor's own (hole, discarded, board) after the joint relabelling, with the board kept in the order it came. There are 970 or 10,170 with one board card out (by whether the actor has thrown), and 10,170 or 100,400 with two.

Keeping the board's order doubles the round-2 half: a sorted board would have given 3,142,290 infosets, merging histories the player can tell apart. The census splits 3,106,630 for P0 and 3,113,420 for P1, and tests/minidrawmaha/census.json holds it.

What the literature does at full scale, and where Deep CFR sits

The survey's abstraction lane (docs/research/05-abstraction-bucketing.md) traces a ratchet of answers to "which hands should share a strategy?":

LayerIdeaCost
Losslesssuit isomorphismnone, provably exact
Expectation bucketsrank hands by equity E[HS] and split into percentileshides shape: a made hand and a live draw can share an E[HS]
Distribution-awarek-means on equity histograms under earth mover's distanceimperfect recall can trade convergence guarantees for size
Potential-awarecluster on histograms over next-round bucketsbuilt bottom-up, so trajectory matters

Two warnings carry over. Refining an abstraction can increase real-game exploitability (Waugh et al. 2009), so an abstraction is validated by exploitability or head-to-head, never assumed. And Drawmaha's split pot makes strength two-dimensional, so a scalar equity is worse than usual.

Deep CFR claims to "obviate the need for abstraction". The survey's synthesis is narrower: nets replace the clustering, not the representation. The net's input is itself an abstraction. Rung 4's 136 features are four card groups of 23 floats each (multi-hot over the deck, plus rank counts and suit counts, a hand-built stand-in for learned card embeddings), plus 44 floats of public state. Two keys that share everything the player can see share a vector, and nothing in the vector names the table row. A net handed key ids would just be a table with extra steps.

Implementation · Shared

Package map

Where every piece of the solver lives, the five layers each game repeats, and the commands that run them.

Everything the solver does lives in one Python package, src/drawmaha_solver/, with one subpackage per graded game and three small modules they all share. The games grow from rock-paper-scissors to mini-drawmaha, but each subpackage is built from the same five layers, so once you can read one you can read them all.

One subpackage per graded game

SubpackageGameSolverGround truth it is graded against
rps/rock-paper-scissorsregret matching, self-playthe uniform Nash, closed form
kuhn/Kuhn poker, 12 infosetsvanilla CFRthe closed-form equilibrium family; all 64 pure strategies
leduc/Leduc poker, 288 infosetsvanilla CFROpenSpiel, frozen in tests/leduc/referee.json
minidrawmaha/mini-drawmaha, 6,220,050 infosetslockstep external-sampling MCCFRthe exact best-response walk (no outside engine exists)
deepcfr/mini-drawmaha againDeep CFRthe same exact walk, and the tabular answer as the yardstick

minidrawmaha/ is the only one split further, into rules/, solver/ and answer/, because it is the first game too large for one file per layer. See Mini-drawmaha rules.

The three shared modules

ModuleWhat it is for
regret_matching.pyRegretMatcher, the ledger every game trains. It stores how much better each action would have done than the mix actually played, and turns that into the next strategy. The learning rule never changes as the games grow; only what feeds it does. See The regret ledger.
play_session.pyThe bookkeeping around a hand that the terminal tables share. QuitGame is raised from any prompt where you type q, so the hand in progress is dropped rather than scored. Scoreboard tallies chips from the human's side, per seat as well as in total, because no poker game here is symmetric. verdict is the one line you read at the end of each hand.
plotting.pyThe house figure style. new_axes, legend and save give every game's figures the same axes, three categorical inks and neutral everything else, so you can compare a rock-paper-scissors curve with a Kuhn one without re-reading the axes. log_indices picks log-spaced checkpoints from a long trajectory.

Every game repeats five layers

The package's own AGENTS.md names the layering. The README section for each game explains the same layers in the same order:

LayerFileJob
1. The rulesgame.pyWhat the game is: the deck, legal actions, chance, payoffs and the infoset (what the player to act can see). No solver code at all.
2. The memoryinfoset_table.pyOne ledger per infoset, all allocated up front so that no code path can invent a new one.
3. The solvercfr.py (Kuhn, Leduc), mccfr.py + lockstep.py (mini-drawmaha)The walk that computes each action's utility and banks it into the ledgers.
4. The meterexploitability.pyHow much a perfect adversary wins against the strategy. It imports game.py and nothing from the solver.
5. Entry pointsanalysis.py, play.py, exploit_server.pySolve and draw the figures; play against the answer in a terminal; serve a live exploit to the site.

The commands, from pyproject.toml

Every entry point is registered under [project.scripts] in pyproject.toml and runs with uv run <name>:

CommandModuleWhat it does
rps-analysis, rps-playrps.analysis, rps.playRun both self-play experiments and draw figures/rung0/; play the learner yourself
kuhn-analysis, kuhn-play, kuhn-exploit-serverkuhn.*Solve Kuhn and draw figures/rung1/ (--json for the site); play the equilibrium; serve the Exploit tab
leduc-analysis, leduc-play, leduc-exploit-serverleduc.*The same three for Leduc and figures/rung2/
minidraw-analysisminidrawmaha.answer.analysis:mainGrade new race checkpoints (--grade-runs) and draw figures/rung3/
minidraw-playminidrawmaha.answer.playPlay the frozen tabular strategy in the terminal
minidraw-dealsminidrawmaha.answer.deal_packWrite the pre-dealt hand pack the /play trainer reads
minidraw-rangesminidrawmaha.answer.range_exportWrite the per-spot range export /solver reads
minidraw-exploit-by-reachminidrawmaha.answer.analysis:reach_mainBin a released strategy's exploitability by how often each spot is reached
deep-analysisminidrawmaha.answer.analysis:deep_mainDraw figures/rung4/price.png and write the Deep CFR answer sheet

The scripts under scripts/ are run by hand and are not registered: the OpenSpiel referee (scripts/leduc/), the census, freeze and grade tools and the Modal race (scripts/minidrawmaha/), the Deep CFR trainers and studies (scripts/deepcfr/), and the site generator (scripts/site/).

Tests mirror the package

tests/ has one directory per subpackage (rps/, kuhn/, leduc/, minidrawmaha/, deepcfr/), with roughly one test file per module, plus top-level checks on the shared ledger, the static site and the agent instruction files. The default uv run pytest needs no torch, Modal, OpenSpiel or network. The expensive whole-game checks sit behind environment flags (MINIDRAWMAHA_FULL_TABLE, MINIDRAWMAHA_FULL_GRADE, MINIDRAWMAHA_FULL_CALIBRATION, MINIDRAWMAHA_FULL_DEEP). See Testing.

Implementation · Shared

The regret ledger

RegretMatcher, the one learning rule every game in the repo trains, and why its two weights are keyword-only.

Every solver in the repo, from rock-paper-scissors to Deep CFR, learns through the same small class: RegretMatcher in regret_matching.py. One ledger sits at each decision point a player can face. It remembers how much better each action would have done than the mix actually played, and it turns that memory into the next strategy. The games around it grow by eight orders of magnitude, and this class barely changes.

What one ledger holds: two vectors, nothing derived

A ledger over n actions stores two persistent vectors and two extra slots:

SlotTypeWhat it is
cumulative_regretfloat64, nR(a): accumulated regret for each action
strategy_sumfloat64, nS(a): the banked strategy mass, the thing that becomes the answer
extra_sumsfloat64, k × nk further averages of the same strategies under other weightings (uniform, quadratic), banked beside the primary one; k = 0 unless a run asks
stampint64, 1the last iteration a lazily applied discount has been brought up to (only DCFR uses it)

The current strategy and the average strategy are never stored. Both are read off these vectors when asked for, so nothing inside a ledger is ever "already averaged".

The three methods

  • strategy(): what this spot plays right now. It keeps only the positive regrets and normalises them, so each action is played in proportion to how much it has been regretted. When no regret is positive, it plays uniformly.
σ(a) = R⁺(a) / Σₐ′ R⁺(a′), with R⁺(a) = max(R(a), 0)
  • average_strategy(): the answer. It normalises strategy_sum, and plays uniformly if nothing was ever banked. In two-player zero-sum games this average converges to equilibrium, while the current strategy cycles forever. Regret matching shows both behaviours on rock-paper-scissors.
  • update(utilities, *, regret_weight, strategy_weight): record one visit. utilities[a] is what action a would have earned. It is read off a payoff matrix in rock-paper-scissors and computed by a subtree walk in the poker games.
R += regret_weight · (u − ⟨σ, u⟩)
S += strategy_weight · σ

Read the first line as follows. ⟨σ, u⟩ is what the mix actually earned, so u − ⟨σ, u⟩ is how much better each pure action would have done. Positive entries are regrets, and negative ones say the action did worse than the mix. The second line adds the strategy that was played into the running sum the average is read from.

update validates its input before it mutates anything. A wrong-width vector, a NaN utility or a negative weight is refused and leaves the ledger exactly as it was. A NaN in particular would poison both accumulators for good, and every later strategy() would quietly return NaN.

Why the two weights are keyword-only

In rock-paper-scissors every round counts the same, so both weights default to 1. In a game tree a spot is only reached some of the time, and the two accumulators want different scales:

WeightIsWhy that scale
regret_weightthe counterfactual reach π₋ᵢ: how often chance and the opponent bring play here, excluding your own choicesa line you currently avoid must keep learning at full strength, or it can never be rediscovered
strategy_weightyour own reach πᵢ (times t in linear averaging)the average has to reproduce what you actually played

Pass them swapped and every game still converges, quietly, to the wrong answer. Kuhn with the weights transposed reports a converged-looking grid that folds the king to a bet 18% of the time. For that reason both weights are keyword-only, and the root AGENTS.md forbids making them positional. The derivation of the two reaches is in CFR.

over: a ledger that is only a window

At mini-drawmaha's 6.2 million infosets one Python object per ledger cost 3.4 GB, rising to 5.4 GB once the two extra slots were added. RegretMatcher.over(...) builds a ledger whose four arrays are views into somebody else's flat buffers, with no copy. That somebody is the packed table. An update therefore lands in the table itself, and the ledger object can be thrown away after the visit. over checks every view's shape and dtype, because a wrong-width view or a float32 buffer would not crash: it would bank into a neighbour's columns or lose precision without a word.

That is also why every writer changes the arrays in place (+=, [:] =) and never rebinds an attribute. Rebinding would silently cut the ledger loose from its table. Mini-drawmaha solver covers the packed table this serves.

Callers

CallerHow it feeds the ledger
rps/players.py → RegretMatchingPlayersamples from strategy(), then banks the payoff column of the finished round, both weights 1
kuhn/cfr.py, leduc/cfr.py → walkcomputes the utilities by recursion; banks at counterfactual and own reach
minidrawmaha/solver/mccfr.py, lockstep.pysampled walks; regret weight 1 (the sample is the weight), the regret rule decides how r is banked
deepcfr/materialise.pywrites a net's outputs into cumulative_regret so strategy() plays the net, without touching this class

Implementation · The graded games

Rock-paper-scissors

The ledger with nothing around it — the rules, the players, the two experiments and the terminal game in src/drawmaha_solver/rps/.

Rock-paper-scissors is the smallest game with a mixed-strategy equilibrium, the uniform (⅓, ⅓, ⅓). The regret ledger therefore has an exact answer to converge to before any game tree exists. There is no tree, no chance and no hidden information here, only the ledger playing itself. That isolates the one thing every later game inherits: regret matching's average converges and its current strategy does not.

Files

FileWhat it is for
game.pyThe rules and the meter in one place, because both are a few lines here
players.pyWho plays, and the loop that runs a match
analysis.pyThe two experiments that certify the game, and their four figures
play.pyPlay the learner from the terminal (uv run rps-play)

game.py: payoffs and the one-shot best response

  • Action: rock, paper, scissors.
  • payoff and winner: who beats whom, as a +1 / 0 / −1 payoff and as a winner index.
  • action_values: what each pure action earns against a mixed strategy. This is the utility vector the ledger learns from, and it doubles as the input to the meter.
  • best_response_value: rock-paper-scissors' exploitability, the most a perfect adversary who knows your mix earns per round against it. It is zero exactly at the uniform mix. See Exploitability.

players.py: the player interface and the match runner

  • Player: the interface a match needs. act chooses the next action and observe sees the finished round.
  • FixedStrategyPlayer: plays one mixed strategy forever; the opponent the learner is tested against.
  • RegretMatchingPlayer: the learner. It samples from its ledger's current strategy and feeds every finished round's payoff column back into update, with both weights at 1 because every round counts the same.
  • HumanPlayer: reads r / p / s from the keyboard. q raises QuitGame.
  • play_match: runs n rounds in which both players observe each one, and returns player 0's payoffs.

analysis.py: two experiments, four figures

uv run rps-analysis runs both and writes figures/rung0/:

  1. Self-play (run_self_play): two ledgers play each other. The average strategy must reach the uniform Nash, with exploitability falling like 1/√T, while the current strategy visibly cycles. After 100k iterations the average is (0.334, 0.333, 0.333) and is exploitable for 0.0009 chips a round.
  2. Against a biased fixed opponent (run_vs_fixed): the learner must find the best response. Against a 50%-rock opponent it converges to pure paper and earns +0.24 a round, against a best-response ceiling of +0.25.

SelfPlayTrajectories and VsFixedTrajectories record every iteration, so the four figure functions can subsample freely.

Self-play average strategy converges to the uniform Nash
Self-play average strategy converges to the uniform Nash
The current strategy cycles; the average settles
The current strategy cycles; the average settles

The app

web/rung0-viz, served at /rung0. It is the one app that ports its ledger to TypeScript. The 63-line regret-matching rule is safe to transcribe, which the reach-weighted tree walk of the later games is not. The rung-0 writeup (Markdown, LaTeX and PDF) is under docs/writeups/rung0-rps/.

Implementation · The graded games

Kuhn poker

The first game tree — twelve ledgers, vanilla CFR's two-direction walk, an exact meter over 64 pure strategies, and an exploit finder built from one extra parameter.

Kuhn poker is the first game with a tree, and so the first where the ledger's utilities have to be computed. The deck has three cards, J < Q < K. Both players ante 1, each gets one card, and one round of betting follows with a single bet size of 1. The whole state of a solve is 12 ledgers of 4 numbers each. Its equilibrium has a closed form, so every number the solver produces can be checked on paper.

Files

FileWhat it is for
game.pyThe rules: deal, betting, the tree, infosets, payoffs. No solver code.
infoset_table.pyThe solver's memory: twelve ledgers, pre-allocated
cfr.pyVanilla CFR, the recursive walk that feeds the ledgers
exploitability.pyThe exact meter; imports game.py only
exploiter.pyLock a strategy and let CFR find the most that beats it
analysis.pykuhn-analysis: one solve, three figures, the answer sheet, --json for the site
play.pykuhn-play: hands against the solved equilibrium
exploit_server.pykuhn-exploit-server: the local endpoint behind /rung1#exploit

The rules: four decisions, two actions, twelve infosets

game.py gives a solver its vocabulary:

  • Action is PASS or BET at every node, so a ledger is always two wide. action_label renders the poker word for display: PASS is a check with nothing to call and a fold facing a bet, and BET is a bet or a call.
  • KuhnState is one node of the tree: the deal plus the actions so far. It is frozen, so apply returns a new node and a walker never has to undo a move. returns pays in chips with the ante as 1 and sums to zero.
  • InfoSet is what the player to act knows: their own card and the public history, and deliberately not the opponent's card. Two deals that differ only in the opponent's card must collapse to one equal, equally hashing key.
  • all_infosets lists the 12 = 4 decision nodes × 3 cards. DEALS lists the six equally likely deals; chance acts only once, at the root.

The memory: a 4 × 3 grid of ledgers

new_infoset_table builds a dict[InfoSet, RegretMatcher] with all twelve entries, so no code path can invent a thirteenth. current_strategy and average_strategy read the whole table at once. format_strategy_grid prints P(BET) as the 4 × 3 grid, with one row per decision node and one column per card. Moving down a column is the same hand in a different spot. Moving across a row is the same spot with a different hand, which is exactly what the opponent cannot see.

The walk: reach goes down, values come up

cfr.py is the only new idea of the game. walk recurses through the tree carrying two quantities in opposite directions:

  • Down: reach. reach[p] is the product of player p's own action probabilities along the path, and nothing else.
  • Up: values. What the node is worth to each seat, as a plain conditional expectation with no reach weighting.

The weights are applied once, when a ledger is banked: regret at the counterfactual reach, the average at the acting player's own reach (see The regret ledger). walk returns the value to both seats, so nothing is ever negated on the way up. run_iteration walks the tree once per deal. train repeats that for the requested number of iterations and returns the table. The full derivation is in CFR.

The meter: all 64 pure strategies, exactly

exploitability.py measures what a perfect adversary wins, (BR₀ + BR₁) / 2, which is zero exactly at Nash. Each player owns 6 infosets with 2 actions, so best_response_value enumerates all 2⁶ = 64 pure strategies for the responder and keeps the best one. Mixed play is handled by weighting both branches at every node, and the hidden card by enumerating all six deals at ⅙, so every number is an exact expectation. best_response returns the counter-strategy itself, expected_value is what each seat earns when both play a given profile, and exploitability averages the two best responses.

Enumerating pure strategies over infosets is what keeps the adversary honest. A pure strategy fixes one action per infoset before any card is dealt, so an opponent who sees your card cannot be written down. A test pins that a "peeking" meter would score the true equilibrium at 0.278 chips a hand.

The exploit finder: CFR against something that cannot move

exploiter.py asks the opposite question, "what beats this?", with no new algorithm. LockedStrategy pins some infosets to probabilities you supply. They never bank an update, but their probabilities still drive the reach, so they still enter the learner's counterfactual weight. Minimising regret against a player who cannot move back converges to a best response instead of to Nash. exploit runs that training and samples Checkpoints along the way. ExploitResult carries the grade: gap is how far CFR still is from the exact ceiling the meter computes. locked_seat tells the two cases apart. A whole locked seat has a ground-truth ceiling. A partial lock leaves the rest of that seat learning, which gives a constrained equilibrium rather than a best response, so no ceiling is claimed.

Entry points

CommandWhat it does
uv run kuhn-analysisrun solves to 100k iterations, measuring both candidate answers at 60 log-spaced checkpoints (Trajectory); three figures go to figures/rung1/; to_json / write_json export the solve the /rung1 page draws
uv run kuhn-playmain solves, then play_hand deals hands against the average strategy, seats alternating because the first seat is worth −1/18; bot_action samples only from the bot's own infoset
uv run kuhn-exploit-serverstdlib HTTP; solve turns a POST of locked spots (parse_locked) into an exploit run and returns the trace; nash_value_against adds what equilibrium play earns against the same lock

The answer, graded: 0.00063 chips a hand after 100k iterations, game value −0.05555 against −1/18, with the jack bluffing at α = 0.221 and the king opening at 0.661 ≈ 3α.

Only the average strategy converges
Only the average strategy converges

Implementation · The graded games

Leduc poker

Chance in the middle of the tree, ledgers of varying width, a best-response recursion that replaces enumeration, and OpenSpiel as a frozen referee.

Leduc poker has six cards, two each of J, Q and K. Each player gets one, a betting round follows, then one public board card, then a second betting round at double the stake. Two things change from Kuhn, and between them they are the whole game. The deck now acts in the middle of the hand, so the walk meets a chance node mid-tree. And spots no longer all offer the same actions, so ledgers are two or three wide depending on where they sit. Leduc has 288 infosets against Kuhn's 12, and its equilibrium has no closed form, so ground truth comes from an outside implementation.

Files

FileWhat it is for
game.pyThe rules: two rounds, the board, legal actions, infosets keyed by rank
infoset_table.py288 ledgers, each sized by its own legal actions
cfr.pyVanilla CFR with a chance branch inside the walk
exploitability.pyThe exact meter, now a recursion rather than an enumeration
exploiter.pyLock a strategy, find its best response — Kuhn's module at Leduc's scale
analysis.pyleduc-analysis: one solve, three figures, --json for /rung2
play.pyleduc-play: hands against the solved strategy
exploit_server.pyleduc-exploit-server: the endpoint behind /rung2's exploit panel
scripts/leduc/generate_referee.pyInterrogates OpenSpiel once and writes tests/leduc/referee.json

The rules: three actions, never three choices

  • Action is FOLD, CALL or RAISE, but legal_actions_for makes folding illegal with nothing to call and raising illegal once the round's two-raise cap is spent. A node is therefore 2 or 3 wide, and legal_actions() is the authority on which.
  • LeducState is one node: deal, board, both rounds' betting. kind reads which of the three things it is (NodeKind: decision, chance or terminal). At a chance node chance_outcomes lists each possible board card at probability ¼ and apply_chance turns it. contributions and returns do the chip accounting.
  • InfoSet is the actor's own rank, the board's rank and both betting lines. Suits are dropped because Leduc has no flushes, so a seat's two jacks are the same hand. all_infosets yields 18 in round 1 and 270 in round 2.
  • hand_strength ranks a private card against a board. A pair with the board beats everything, so a jack is trash until a jack lands.

The memory: width is per key

new_infoset_table sizes each ledger by len(infoset.legal_actions()): 192 are two wide and 96 three wide. Entry k of a ledger belongs to legal_actions()[k], not to Action value k. Column 0 means CALL at the open and FOLD facing a raise. Every loop over a ledger's columns pairs enumerate(legal_actions()) with the ledger, and nothing else keeps a column's meaning in step with the rules. format_strategy_tables prints the 288 spots as four tables: round 1, then round 2 per board rank.

The walk gains a chance branch

cfr.py's walk now has three branches, and the chance branch does two different things:

  • For values, it averages. Nobody chooses the board, so a chance node's value is the plain probability-weighted average of its children. Every round-1 regret above it depends on this average.
  • For weights, it multiplies a receipt. Counterfactual reach includes chance, so the walk threads π_c down: 1/30 once the private cards are out, and 1/120 once the board lands. It rides in the regret weight only. The strategy weight is still the player's own reach.

One iteration walks 9,450 nodes in about 50 ms, so leduc-analysis defaults to 2,000 iterations, roughly two minutes. As with Kuhn there is no seed, and figures/rung2/ is a regression fingerprint.

The meter: sum over hidden worlds first, then take one max

A Leduc seat has about 6 × 10⁵¹ pure strategies, so Kuhn's enumeration is dead. best_response_value runs the recursion that was hiding inside the enumeration:

BRᵢ(I) = maxₐ Σ_{h ∈ I} π_c(h) · π₋ᵢ(h) · v(h · a)

Each piece is a part of the problem. I is the responder's infoset. h ranges over the hidden worlds it could be, which are the opponent's possible cards. π_c(h) · π₋ᵢ(h) is how often the deck and the frozen opponent produce that world. v(h · a) is what action a then earns there. The order is the point. The sum over worlds happens inside and the max outside, because one action has to be chosen for the whole infoset. Put the max inside and it picks a different action per world: that is an adversary who peeks at face-down cards, and it over-reports exploitability. best_response returns the counter-strategy, and expected_value and exploitability are as in Kuhn. See Exploitability.

The referee: OpenSpiel, interrogated once

generate_referee.py is the only file in the repo that imports OpenSpiel, and nothing imports it. It is run by hand:

uv run --with open_spiel --with cvxpy --with ecos --no-project \
    python3 scripts/leduc/generate_referee.py

It writes tests/leduc/referee.json: the node census, the collapse of OpenSpiel's 936 suited infosets onto our 288 ranked ones, the payoffs, the path probabilities, the sequence-form LP's game value and OpenSpiel's own CFR curve. tests/leduc/test_referee.py re-derives every fact from our code with no OpenSpiel present. OpenSpiel's wheels are large and the Vercel build must not pull them, so it is deliberately not a dependency.

Entry points

CommandWhat it does
uv run leduc-analysisrun solves and samples Trajectory checkpoints; three figures go to figures/rung2/: exploitability, game value against the LP's −0.0856, and three poker facts no equilibrium can miss (the nuts check-raise, a beaten hand folds to a raise, the king opens for a raise)
uv run leduc-playhands against the solved strategy; turn_board draws the board at the deck's own odds, seats alternate (the first seat is worth about −0.086)
uv run leduc-exploit-serveras Kuhn's, with two changes for scale: runs capped at 2,000 iterations, and the Nash baseline is read from the committed web/rung2-viz/src/data/solve.json (nash_profile) instead of re-solved at boot; row_of shapes one spot's distribution for the wire

The answer: 0.011 chips a hand, game value −0.0874 against the LP's −0.0856.

Exploitability of the average falls; the current strategy does not
Exploitability of the average falls; the current strategy does not

Implementation · Mini-drawmaha

Mini-drawmaha rules

The 15-card bomb pot with one draw and a split pot — the deck and its suit relabelling, both hand rankings, the state machine, the 6,220,050-infoset enumeration and the 136-float features the nets read.

Mini-drawmaha is a shrunken Drawmaha that keeps the two things the full game is made of, a draw and a split pot, on a deck small enough that the exact answer can still be computed. It is the last game where "how far is this strategy from optimal?" has an exact answer, which is why Deep CFR is graded here before it meets the real game. No engine implements Drawmaha, so every rule is a decision, and rules/ has no outside referee. It holds no solver code either: the exact grader imports this package and nothing else.

The hand, start to finish

  1. Both players ante 1 from a 26-chip stack. It is a bomb pot: no blinds, no position asymmetry and no preflop betting.
  2. Three private cards each, then board card 1, face up.
  3. Betting round 1. Pot-limit, so the actions are fold, check/call and pot, with the bet capped at the stack.
  4. The draw. P0 throws at most one card, then P1 does, knowing whether P0 drew but never what. The replacement is dealt face down; the full game's face-up draw-one rule is not played here.
  5. Board card 2, then betting round 2, then the showdown.
  6. The pot splits in half. The inner half goes to the best 3-card hand made of the three private cards alone. The outer half goes to the best 4-card hand made of exactly two private cards plus both board cards. Winning both halves is a scoop.

Files

FileWhat it is for
cards.pyThe 15-card deck and the one collapse allowed: relabelling suits jointly
hands.pyBoth halves of the pot as orderable scores
game.pyThe hand as a state machine, the InfoSet, the chips
enumeration.pyEvery infoset, all 6,220,050, reached through 141 public points
features.pyEvery infoset as 136 floats, the view Deep CFR's nets read

cards.py: five ranks, three suits, one relabelling

Card is a rank 0–4 (printed 2–6) and a suit 0–2. Three suits is forced: with two, a rank holds only two cards and the inner half loses trips. The fifth rank gives the outer half a complete ranking. hand_symbol and parse_cards convert to and from text such as "6h 2c".

Suits carry information here, because both halves of the pot have flushes. The table may forget which physical suit is which, but never how the suits relate. canonical relabels the three suits to the lexicographically smallest image of all the visible groups jointly, and canonical_relabelling returns the relabelling it used. relabel applies one. Doing the hole and the board separately would forget whether they share a suit, which is exactly what a flush is. The joint relabelling takes the 455 possible hands down to 95 classes at no cost.

hands.py: rankings by measured rarity in this deck

inner_score scores the best 3-card hand of the three held cards. outer_score takes the best of the three ways to choose exactly two hole cards and scores each with four_card_score against both board cards. Each returns a Score: the category (InnerCategory, six of them; OuterCategory, seven), then what it is made of, so plain tuple comparison decides who wins.

Both orders are measured rarity in this 15-card deck, not 52-card convention, which makes two rows look wrong when they are correct. A four-card straight (25.11%) ranks below two pair and trips, because a run of four from five ranks is easy to make. Outer high card is the rarest outer result (0.74%) and still ranks last, because high card is the absence of a made hand.

game.py: the state machine

  • Action holds every decision in one enum: FOLD, CHECK_CALL and POT for betting, and four throws (none, low, mid, top) for the draw. Values are not ledger indices. throw_count is the public half of a throw. action_label and line_symbol are for display.
  • MiniState is one node: both holes, the discards, the board and the betting. Its kind (NodeKind) is decision, chance or terminal. The chance node depends on the action: a player who stands pat gets no replacement deal. chance_outcomes and apply_chance turn board cards and deal replacements; legal_actions is betting or the four throws; returns pays half the pot on each hand, using pot_shares.
  • ChipState and chip_state replay the betting to report the pot, each player's contribution and what is owed. betting_legal_actions applies pot-limit.
  • InfoSet is what the actor can see, suits relabelled: canonical_picture of (hole, discarded, board in the order it came). The board order is kept on purpose, because round 1 was played against the first card alone. DrawSignal is what the table learns of an opponent's draw: the count and nothing else. draw_order lays the actor's cards out as low/mid/top so a throw names one card.
  • random_deal deals uniformly over all 100,100 deals.

enumeration.py: 6,220,050 infosets from two small halves

The concrete tree has 85,129,644,600 decision nodes, so it cannot be walked. A position does factorise into two small, independent halves:

  • A public point (PublicPoint): the betting so far, both draw counts, how many board cards are out and whose turn it is. Its width is the ledger width at that point. public_tree builds the whole public game, 141 decisions, 36 chance nodes and 178 terminals, from one representative deal, because no public point depends on a card. merged combines two routes into the same public spot by adding their weights, and refuses if the arrivals disagree about who acts. public_decision_points lists the 141 in walk order.
  • A private key (private_keys): the canonical (hole, discarded, board), sorted. A decision stands on one of three shapes: 970 keys with one board card out, 10,170 with two out before a throw, and 100,400 with two out after one.

all_infosets is two nested loops over those halves and yields each infoset exactly once, 6,220,050 in all. Streaming them takes about fifteen seconds.

features.py: what a net sees instead of a key id

A net handed a key id would just be a table with extra steps, so encode turns an infoset into the 136 floats the player could see:

PartFloatsBuilt by
hole, discard, board card 1, board card 24 × 23 = 92card_group: a multi-hot over the 15 cards, plus 5 rank counts and 3 suit counts (a pair reads as a 2)
both players' draw signals6in public_row
the betting lines, 2 rounds × 5 slots × 3 actions30
chips3
seat and stage5

The rank and suit counts repeat what the multi-hot already says, on purpose. They are a hand-built stand-in for learned rank and suit embeddings, and they hand a single unit "any pair" directly. Because the vector splits where the key splits, private_features and public_features build each half once. layout returns the whole-game Layout, which places each table row: locate_rows gives its public point and position, and features gathers its 136 floats. legal_gather maps each ledger column to the net's 7-wide head.

The census: a stand-in for a referee

With no engine to ask, scripts/minidrawmaha/generate_census.py counts what the enumerator yields, one InfoSet at a time, and writes tests/minidrawmaha/census.json. tests/minidrawmaha/test_census.py rebuilds every number a second, different way. A change to the deck, the throw cap or the betting tree then fails loudly instead of shipping a solver for a different game. The rules were decided in a plan that lives outside the repo, at ~/Desktop/Claude/poker/rung3-delta/rung3-delta-plan.md. Check it before changing a ranking, a count or a rule.

Implementation · Mini-drawmaha

Mini-drawmaha solver

The packed table, external-sampling MCCFR, the four regret rules, lockstep's determinism contract, W processes over shared memory, and the run driver that survives being killed.

Mini-drawmaha has 6.2 million infosets, about 21,000 times Leduc's 288, and a tree of 85 billion decision nodes, so a full pass per iteration does not run at all. solver/ keeps the shape of the earlier walks and changes three things. It stores the ledgers in four flat arrays, samples the deck and the opponent instead of enumerating them, and walks W sampled hands per seat against one frozen strategy so that W processes can share one table. The race this machinery ran, four regret rules for 20M hands per seat on Modal, is what picked LCFR as the tabular answer.

Files

FileWhat it is for
regret_rules.pyWhat one visit banks: vanilla, LCFR, CFR+, DCFR, and the extra averaging columns
packed_table.pyThe only table: every ledger as a window into four flat arrays
infoset_table.pyThe ledger-per-infoset interface over the packed table, and the readouts
mccfr.pyExternal sampling, one hand per seat per iteration
lockstep.pyW hands per seat against one frozen σₜ: the reference
parallel.pyThe same in W processes over shared memory, equal bit for bit
lockstep_run.pyDrive a run to its target: resume, grid checkpoints, progress log
exploitability.py (one level up)The exact meter; imports rules/ only

The packed table: 0.3 GB instead of 5.4 GB

The first version held one RegretMatcher object per key: 3.4 GB and half a minute to build. Ten of those cannot share a machine. PackedTable keeps the numbers and drops the objects, in four public arrays laid out in all_infosets() order:

ArrayShapeWhat it holds
cumulative_regret, strategy_sumfloat64, total widthevery ledger's columns end to end
extra_sumsfloat64, k × total widththe extra averaging columns
stampint64, one per ledgerDCFR's discount stamp

table[key] returns a ledger window (RegretMatcher.over) onto that key's slice. whole_game builds the full table, listed builds one over chosen keys (the tests use it), and over lays the index over buffers someone else owns: shared memory, or a loaded checkpoint. row_of and slot_of locate a key's row and columns. widths, one per row, is the table's fingerprint.

infoset_table.py keeps the familiar interface on top: new_infoset_table, plus ledger_widths, which counts ledgers by width without allocating. A ledger is 2 wide at an open or a shove, 3 facing a bet and 4 at the draw. Column 0 is CHECK_CALL at the open, FOLD facing a bet and THROW_NONE at the draw. A uniform 4-wide table would bank regret for throwing a card at a betting spot, and that phantom column could win the normalisation. StrategyView, current_strategy and average_strategy compute strategies lazily, and spots and format_strategy_tables print one hand's strategy for eyeballing.

External sampling: sample the deck and the opponent, enumerate yourself

mccfr.py keeps Leduc's recursion and changes two branches. A chance node follows one outcome drawn by its probability, and an opponent's spot follows one action drawn from the opponent's current mix. The traverser's own spots are still fully enumerated, because regret is a comparison between actions and a comparison needs all of them. Since the walk arrives at a spot exactly as often as π₋ᵢ · π_c, the sample is the weight:

at my spot: R(I, ·) += 1 · (u − ⟨σ, u⟩)
at their spot: S(I, ·) += t · σ(I, ·)

traverse is that walk. run_iteration deals a fresh hand for P0 to traverse and another for P1, alternating. Solve is everything needed to continue a run (table, t, RNG, rule, columns), with new_solve and train around it, and save_solve / load_solve / read_run for flat .npz checkpoints. State is the nine-member interface the walk reads, which LeducState and MiniState both satisfy, so the same walk was calibrated on Leduc first. See MCCFR.

The four regret rules

regret_rules.py is the only thing that differs between the rules the race compared. Each bank_* function takes one sampled regret r = u − ⟨σ, u⟩:

RuleBankIn plain words
vanilla (bank_vanilla)R += revery iteration counts the same
LCFR (bank_linear)R += t · riteration t counts t times iteration 1, so noisy early iterations fade
CFR+ (bank_cfr_plus)R ← max(R + r, 0)negative regret is floored at zero, so a late-good action is played at once
DCFR (bank_discounted)pay the discounts owed since the stamp, then addpositive and negative regret are discounted at different rates

Under sampling a row can go 37,000 iterations between visits, so DCFR's per-iteration discount cannot be applied eagerly. The ledger's stamp records how far the discount has been paid, settled_regret reads a row's regret as of iteration t without touching it, and cumulative_log_discount keeps the product in log space. bank_rows banks many rows at once for the packed path. RegretRule and Average name the rules and averaging weights (1, t or t²) a checkpoint records. validate_averages, extra_weights and column_average handle the extra columns, which cost nothing and cannot change play. The mechanism behind why CFR+ and DCFR lost is in Regret rules.

Lockstep: W walks against one frozen σₜ

lockstep.py makes one iteration W sampled hands per seat, one per worker, all reading the same σₜ and writing nothing while they walk. walk returns a Recorded, the writes it would have made. apply then banks each touched row once, in worker order, with the sum of its regrets. The W walks of one seat are a phase, and seats alternate. Two consequences matter:

  • It is exact MCCFR with W samples per iteration. CFR+ floors once per iteration, on the W-sample total.
  • It is a determinism contract. stream derives each walk's random stream from (seed, t, seat, k), never from W or scheduling. A run's floats are therefore the same however the walks were run.

LockstepSolve, new_lockstep and train_lockstep wrap a run. Flat, flatten and apply_arrays are the array form a worker hands back, banked vectorised. save_lockstep writes atomically, load_lockstep resumes, and is_lockstep / read_lockstep inspect a checkpoint without loading it.

parallel.py: W processes, no locks

ParallelLockstep runs each walk in its own process over one packed table in shared memory (packed_table_for builds it). One phase:

  1. The parent writes (t, seat) into a control block and posts each worker's go semaphore.
  2. Each worker walks its hand, reading σₜ and writing nothing, copies its record into its own scratch block and posts done.
  3. Once done has been taken W times, the parent banks the W records in worker order while every worker waits.

The table is only written while nobody reads it, so no lock is needed. Semaphores are used rather than multiprocessing.Barrier, whose crossing cost about 2 ms on macOS, more than a walk. test_parallel.py holds train and step equal to lockstep bit for bit at the same W.

lockstep_run.py: a run that survives being killed

run_to(out, rule=..., target=..., workers=...) keeps everything in out/: resume.npz, rewritten every few minutes; iter-<t>.npz at each grid mark (kept_name), never overwritten; and progress.jsonl, one line per save. Each save is a temp file renamed into place, then its progress line, then one commit (a Modal Volume's commit, or nothing locally). SIGTERM and SIGINT finish the iteration and save. Calling run_to again resumes, and because the streams are counter-based a resumed run equals one that never stopped. A resume refuses a different W or rule, by name.

The meter: vectors over the public tree

exploitability.py asks the same question as Leduc's meter, sum over hidden worlds first and then one max, but it cannot tag 8.5 × 10¹⁰ concrete worlds. It walks the 141-point public tree instead, carrying a vector over the opponent's private states. At each responder point it carries a vector of values over the responder's states:

v[b, r] = Σₕ w[b, h] · M_b[r, h] (a terminal, board b)
v[b, r] = maxₐ vₐ[b, r] (a responder spot)

r is the responder's private state and h the opponent's. w is the reach vector, how often the deck and the frozen opponent bring h here. M_b is the showdown matrix on board b, zero where the hands share a card. minidrawmaha_game compiles the rules into a VectorGame of arrays once: Node, Edge, DrawMap, Level, KeyMap and Showdown are its parts. exploitability, best_response_value and expected_value answer as before. gains_by_infoset splits a seat's exploitability over infosets, together with how often each is reached; this is what minidraw-exploit-by-reach bins. A grade holds about 1.5 GB of showdown tables and takes about two minutes.

Running it

WhereCommand
this machinecaffeinate -i uv run python scripts/minidrawmaha/run_lockstep.py lcfr --out runs/lockstep-lcfr (run_lockstep.py)
Modal, the racemodal_rung3.py: ::smoke, deploy, ::main --target 2000000 --rules ..., ::status; one 11-core container per rule (10 workers + parent) on the drawmaha-rung3 Volume, self-continuing past Modal's 24-hour cap
grade one checkpointuv run python scripts/minidrawmaha/grade_checkpoint.py <ckpt.npz> [--column quadratic] (grade_checkpoint.py), about 2 GB
grade the raceuv run minidraw-analysis --grade-runs runs/modal adds new checkpoints to figures/rung3/grades.json
Keeping negative regret wins
Keeping negative regret wins

Implementation · Mini-drawmaha

Mini-drawmaha answer

The frozen strategy and its release, the race's analysis and figures, the terminal table, and the two exports — the deal pack and the range export — that let a browser play a 6.2-million-row strategy.

answer/ is everything built on the solved game. A training checkpoint is the wrong thing to hand anyone: 555 MB, almost all of it the learner's working state. What the game needs is one probability per legal action. This package freezes that, publishes it as a release asset pinned by SHA-256, grades it, and cuts it into the two shapes the site reads, because the browser cannot look the strategy up itself.

Files

FileCommandWhat it is for
strategy.py—The frozen strategy: two flat arrays, a pinned digest, download-and-verify
releases.py—The catalogue of every released strategy and where its grade is recorded
analysis.pyminidraw-analysis, minidraw-exploit-by-reach, deep-analysisThe race's figures and answer sheet, the reach profile, Deep CFR's price figure
play.pyminidraw-playThe terminal table against the frozen strategy
deal_pack.pyminidraw-dealsHands dealt in advance, with the strategy at every node, for /play
range_export.pyminidraw-rangesThe strategy cut by public decision point, for /solver and Learn

strategy.py: 57 MB of probabilities, refused if the layout differs

FrozenStrategy holds two arrays in the packed table's order: probabilities, every infoset's average strategy laid end to end (14,253,840 float32s, about 18 MB compressed), and widths, each row's number of legal actions. widths is the layout's fingerprint. A file written against a different enumeration, such as the old unordered-board key's 3,142,290 rows, is refused on load instead of being read as somebody else's strategy.

  • from_checkpoint freezes one averaging column of a lockstep checkpoint, using average_rows to normalise each row. A row that sums to zero plays uniformly, exactly as RegretMatcher.average_strategy reads it.
  • uniform_strategy is the race's floor line, every action played equally.
  • save_strategy / load_strategy write and read the compressed .npz.
  • fetch_strategy downloads a release asset once into default_cache (~/.cache/drawmaha-solver) and checks sha256_of against the pinned digest. StrategyInfo records where a strategy came from.

scripts/minidrawmaha/freeze_strategy.py is the hand-run freeze. It prints the size and digest to pin, and --grade checks the freeze against the checkpoint's own exploitability to the sixth decimal.

Of the LCFR run's 6,220,050 rows, 2,710,428 were never banked and 3,537,196 read exactly uniform, nearly all in round 2. A human who deviates walks the bot into them. The remedy is more iterations or Deep CFR's nets, not a patch to the lookup.

releases.py: one catalogue both exporters read

Release names one shipped strategy: its tag and asset on GitHub (url), its SHA-256, and the committed file and key that hold its exact grade. exported says whether /solver carries it behind its switch. RELEASES holds every one:

NameTagWhat it isGrade from
tabularrung3-lcfr-2mLCFR's linear average at 2M iterations, the tabular answerfigures/rung3/answer_sheet.json
deep-pi, deep-exactrung4-deep-t500the 4 × 256 Deep CFR arm at t = 500: its policy net Π and its exact averagefigures/rung4/sweep_256x4-s16000-lr5e-4.json
deep-pot-exact, deep-pot-pirung4-deep-pot0.5-t200the ÷ √pot arm at t = 200; its exact average is what the Play tab deals againstfigures/rung4/pot_0.5-256x4-s16000-lr5e-4.json

read_grade copies a release's grade from its committed file into an export's manifest at export time, and entry_mismatch says why an entry does not match. A page therefore quotes a number with a committed source and never one typed into TypeScript. provenance_json writes the strategy block both manifests open with, byte for byte the same shape. release_of recognises a local file by its digest. See Releases.

analysis.py: grade rarely, draw often

Grading a checkpoint takes about two minutes and needs the 555 MB file from the Modal Volume, so the work is split.

  • --grade-runs DIR (grade_runs) grades every checkpoint and averaging column not yet in figures/rung3/grades.json, saving after each. grade computes both best responses, their mean and P0's self-play value. grade_key builds keys rule/iteration/column, with the iteration zero-padded so keys sort in training order.
  • Without it, main reads the committed grades and the released strategy and draws the four figures: fig_race (exploitability against hands per rule, built from series), fig_columns (every averaging column at the last iteration), fig_round_one and fig_draw (what the answer does by inner category, from round_one_frequencies and draw_frequencies). answer_sheet writes every number the figures and README quote; print_summary prints them.
  • reach_main (minidraw-exploit-by-reach) splits a strategy's exploitability over spots with reach_profile. It bins the spots by how often play reaches them, computes per_visit, what a spot leaks each time it is reached, and draws fig_reach. stage_of classes a node as round 1, the draw or round 2.
  • deep_main (deep-analysis) reads the two committed grade files. price_table pairs each Deep CFR object with LCFR at equal hands, fig_price draws the result, and deep_answer_sheet writes figures/rung4/answer_sheet.json. deep_key and deep_series read the deep/<t>/<column> keys.

The tabular answer graded: 0.061346 chips a hand (BR₀ −0.034421, BR₁ +0.157114). In self-play P0 is worth −0.0906 a hand, because P1 acts last in both rounds and in the draw.

Round 1 betting by inner category
Round 1 betting by inner category

play.py: the terminal table

main loads the frozen strategy, introduce says what is at the table, and play_session runs hands until you quit, seats alternating. play_hand deals, plays both rounds and the draw, and scores. deal_chance turns cards at the deck's own odds, and bot_action samples the strategy at the bot's own infoset. narrate reports a bot decision only as the table would see it: whether it drew, never what. thrown_card maps a throw's canonical position to the physical card, so "throw low" is never a guess. showdown_line and category name both halves' winners. ask_action, prompt_for and parse_action handle input. report prints the session's net beside each seat's self-play value.

The two exports: the lookup done ahead of time

The table's key is the joint canonical suit relabelling, the one routine the repo most wants to exist exactly once, and the site runs no Python. So the lookup happens in Python, ahead of time, and the browser only follows paths.

deal_pack.py: every node of 600 pre-dealt hands

A deal is a fixed deck order: P0's three cards, P1's three, board card 1, then the stub in the order it will be consumed. For one deck order the hand's decision tree is finite (8 nodes in round 1, 21 at the draw, 28 per throw combination in round 2). export_deal walks every node for both players and records the strategy's mix there, quantised by per_mille to integers summing to exactly 1000. Beside the vectors it records each player's cards in canonical low/mid/top order and the showdown of all 16 post-draw worlds, so the browser transcribes neither the key nor the hand ranking. random_decks draws the deck orders, deal_from_deck sets up the hand, and write_pack writes chunks of 100 plus a manifest under web/rung3-viz/public/deals/. card_index is the shared card numbering, rank * 3 + suit.

range_export.py: every hand at every spot

/solver asks the opposite question: what does the strategy do with every hand at this spot? The export keeps the table's factorisation on disk. Each of the 141 public points is one chunk, point-<index>.bin.gz (point_meta, PointMeta). Each shape's sorted key list is written once (shape_name gives b1d0, b2d0, b2d1; ShapeMeta; encode_keys). Every point is a plain array in its shape's key order (point_rows), quantised to bytes by quantise with each row summing exactly to the scale. export_ranges writes it all with an index.json Manifest.

The browser must still relabel suits to find a key. write_fixtures therefore writes seeded random Pictures (random_picture, fixture_case) with Python's four answers beside each, and settle_fixtures refuses to overwrite fixtures that differ. The TypeScript side is tested against them.

Implementation · Deep CFR

Deep CFR package

Eight modules that replace the regret half of mini-drawmaha's 6.2 million ledgers with a network — without changing the walk, the ledger or the grader.

Mini-drawmaha's tabular solve keeps one ledger per infoset. That works for 6.2 million infosets and is impossible for full Drawmaha, whose table could never exist. deepcfr/ replaces the ledgers' regret half with a network that reads what the player can see, the 136 features from rules/features.py, and outputs one number per action. Everything else stays the same: the walk is mini-drawmaha's lockstep.walk, the ledger's strategy() is unchanged, and the exact grader is untouched. Mini-drawmaha's answer is the yardstick. At equal hands per seat, how exploitable is Deep CFR compared with LCFR?

The trick: write the net into the table, don't teach the walk about nets

The walk already plays regret matching over cumulative_regret. So instead of changing the walk, materialise writes each net's outputs into the regret slots before every phase, and the walk plays the net through the ledger it always read. The table becomes a cache of the net's opinion, rewritten twice per iteration.

Eight modules, one concern each, and one study beside them

ModuleWhat it is for
net.pyThe MLP, chunked prediction into NumPy, and its save file
materialise.pyTwo seats' nets into the table's regret slots: Deep CFR's eq. (3) obtained by what is written
fit.pyThe from-scratch refit on (row, t, target) arrays
reservoir.pyThe memories the refit reads: Algorithm R, with an exact resume
collect.pyK lockstep walks for one seat on W processes, returned as memory rows
train.pyThe loop that calls them in order, its resume state, marks and stop, behind a Learner seam
average.pyThe exact average strategy: own reach through a predecessor table, summed every phase
policy.pyΠ, the average learned by regression on the policy memory: the only average a game without a table can keep
distill.pyBeside the loop, not in it: the net fitted straight to a known strategy by reach-weighted cross-entropy, so a copy's grade says what the net can hold

net.py: 136 → 64 → 64 → 64 → 7, linear head

Net has depth hidden layers of width ReLU units and a linear head of 7, one output per Action in enum order, so an output column means the same thing at every spot. A spot reads its 2, 3 or 4 legal columns through legal_gather; the rest are trained on nothing and read by nobody. The head is linear because the outputs stand in for cumulative regrets, which are signed, and materialise needs the sign. Width and depth are parameters so one trainer can run the size sweep. The default 3 × 64 has 17,543 parameters, a number a test pins. The best arms ran 4 × 256. predict runs chunks of 262,144 rows under inference_mode and returns float32 NumPy. save_net / load_net store the size and weights, and loading uses weights_only=True so a file cannot execute code.

materialise.py: eq. (3) by the write

Deep CFR plays regret matching over the net's outputs when any is positive, and the single best action when none is. The ledger cannot express the second clause, since it reads all-negative as uniform. So regret_slots_from_outputs writes:

Net outputs at a spotWritten into the slotsThe ledger then plays
some positivethe outputs as they areregret matching (negatives clipped)
all ≤ 0, not all equala one-hot at the argmax (shared by ties)the best action
all equal (untrained net, exact tie, no net)zerosuniform

materialise does this for every row of the whole game, one pass per public point, because each point's rows are a contiguous slice beside one public feature row.

fit.py: throw the net away, fit a new one

Deep CFR does not update a net between iterations. It fits a fresh one to everything in memory. fit_from_scratch takes three equal-length arrays: rows (which table row; turned into the 136 features, never seen as a number), t (the iteration that offered the sample) and targets (4 wide, padded). batch_loss is per-row squared error over the row's legal columns, with padding masked before the subtraction so a padded NaN never reaches a gradient. Each row is weighted by t over the batch's mean t, borrowed from Linear CFR, so later and better samples outvote early ones. FitConfig is the net's size, Adam's steps, batch, rate, clip, threads and device (cpu or mps). Nothing here knows what a target is, so the same fit serves the regret nets and Π.

reservoir.py: a uniform sample of everything ever offered

Over 2,000 iterations the walks offer about 280 million advantage rows per seat. A memory keeps C = 50 million. Reservoir uses Algorithm R so that every offer is resident with probability C/n and no iteration is favoured by when it happened. The first C offers fill the slots. Offer g ≥ C draws j uniformly from [0, g] and overwrites slot j if j < C. merge offers one phase's rows at once. contents returns read-only (rows, t, targets) for the refit, and size, offered and capacity report the counts. reservoir_stream seeds each merge from (seed, t, seat, Memory), and save / load resume exactly. A row is 24 bytes (int32 row, int32 t, float32 × 4 target), so a memory is 1.2 GB.

collect.py: walks in, memory rows out

Collector owns the run's table and W spawned workers for the whole run. collect runs walks 0..K−1 of one seat at iteration t against the table as it stands. The walk records, per traverser visit, (key, slot, r) with r = u − ⟨σ, u⟩, and per opponent visit the strategy it sampled from. rows_of reshapes one walk's record into a Collected: advantage rows (traverser rows and their r, padded to 4) and policy rows (opponent rows and their σ). joined lays K of them end to end. σ is read from the walk's uniform averaging column, whose weight is t⁰ = 1, so it is σ bit for bit; dividing t · σ by t would not be. Walk k's stream is lockstep.stream(seed, t, seat, k), so the rows never depend on W. A Mac run and a Modal run of one config see identical hands.

average.py: CFR's actual answer, rebuilt exactly

The strategy played at iteration t never settles; only the average converges. The tabular solver got the average for free in strategy_sum, and Deep CFR throws the table away every iteration. ExactAverage rebuilds it (Single Deep CFR). While each iteration's whole strategy sits materialised in the table, accumulate adds

sums(I, a) += t · own_reachₜ(I) · σₜ(I, a)

and frozen normalises each row for the grader. Own reach is the weight because the average strategy must reproduce what the seat actually played, so a spot it rarely steers into counts for little. own_reach pushes reach through the 141 public points using predecessors (a Pred per point) and current_strategy. A post-draw key has three pre-draw predecessors, since the key forgets which card was drawn, and all of them count, with multiplicity. A brute-force walk in tests/deepcfr/test_average.py is the gate. save / load make it part of the resume set, because the sum cannot be rebuilt later except by replaying every net.

policy.py: Π, the average a net can carry

At a game whose table cannot exist, ExactAverage is impossible. The original Deep CFR keeps the average anyway. Every opponent visit copies (row, t, σ) into a policy reservoir, and a fresh net is fit to it with the same t-weighted squared error. The output minimising Σ tᵢ(y − σᵢ)² at a spot is Σ tᵢσᵢ / Σ tᵢ, the t-weighted mean, and the reach weight rides in how often each spot was sampled. fit_policy fits Π at a mark under policy_seed. policy_strategy freezes it over every key: clipped, normalised, uniform if all zero. Π carries two errors the exact average does not, the reservoir's sampling and the fit's. Measuring both on mini-drawmaha is the reason it is graded.

Three objects graded at every mark

ObjectBuilt byAnswers
exact averageaverage.ExactAveragewhat CFR's guarantee is about
Πpolicy.fit_policy on the mark's policy memorywhat full Drawmaha will have to use
current iteratetrain.current_iterate, regret matching over the materialised tablewhat CFR never promises

Implementation · Deep CFR

Deep CFR training

One iteration of the loop, the oracle pin that proves it is CFR, the resume state, dividing regret targets by the pot, and how runs are launched on a Mac or on Modal and graded at eight grid marks.

deepcfr/train.py is the order the other seven modules are called in, plus everything a 100-hour run needs around them: a resume state that survives a crash, grid-mark saves, a progress line per iteration and a graceful stop. It adds no new math. This page follows one iteration, then the run around it, then how a run is launched and graded.

One iteration: two phases, each a write, a walk and a refit

Seats alternate, as in lockstep. For each seat p, in order:

for p in (0, 1):
    learner.write(table, nets)                    # both seats' current nets into the regret slots
    for observe in observers: observe(table, t, 1 - p)   # the exact average sees σₜ
    rows = collector.collect(seat=p, t=t, walks=K)        # K walks for seat p
    advantage[p].merge(rows.advantage_rows, rows.advantage_targets, t=t, rng=...)
    policy.merge(rows.policy_rows, rows.policy_targets, t=t, rng=...)
    nets[p] = learner.fit(*advantage[p].contents(), seed=(seed, t, p)).model
StepModuleWhat it does in the run
writematerialiseturns both nets into regret slots, so the unchanged walk plays them
observeaverage.ExactAverageadds t · own reach · σₜ while the whole strategy sits in the table
collectcollect.CollectorK = 10,000 walks for the seat on W spawned processes
mergereservoir.Reservoirthe regret samples into seat p's advantage memory, the opponent's σ into the policy memory
refitfit.fit_from_scratcha fresh net on seat p's memory: 4,000 to 16,000 Adam steps of 10,000 rows

Materialise runs before every phase, twice per iteration, because seat 1's walks at iteration t must sample seat 0's freshly refit net. step runs one iteration in that order and returns StepStats: seconds per stage per seat, the refits' losses and the rows offered. The refit is about 95% of the time. Moving it to Apple's GPU (--device mps) cut a 4 × 256, 8,000-step refit from 127 s to 34 s per seat.

The learner seam, and the oracle that proves the loop is CFR

The loop never names torch. It talks to a Learner with four methods, fit, write, save and load. NetLearner is the real one; it imports fit, materialise and net inside those methods, and fit_config builds its FitConfig once torch is first needed. Fitted is what a refit returns, the model and its loss on a held batch.

OracleLearner is a "perfect net". Its model (OracleModel) is a copy of the memory, and its write puts at every row the t-weighted sum of every sample there, in LCFR's own float order. With it, Deep CFR collapses into the tabular solver's LCFR. tests/deepcfr/test_train.py pins the loop bit for bit over three iterations against LCFR built from lockstep's own walk, flatten and apply_arrays, with each sample rounded through float32 first. Every possible join error breaks that pin: a stale table, the wrong t, the wrong seat, a dropped or reordered row. The test runs in the default suite with no torch.

Dividing each regret target by the pot

pot_scaled exists because one net fits every spot's mean regret with a single accuracy budget, and the loss decides where that budget goes. Raw chips spend it badly.

  • What each piece is. A regret is in chips, so a spot's regrets are about the size of its pot. A relative error ε in the net's output there costs (ε · pot)² per sample. Raw squared error therefore weighs a spot by how often it is reached × pot². A 52-chip spot counts 676 times as much as a 2-chip spot reached equally often, and the ante spots where every hand begins are fitted last.
  • Why dividing is free. The divisor is the pot at the decision, which every sample at that spot shares. Each spot's mean regret is scaled by one constant across its actions, and regret matching reads only a row's sign pattern and ratios, so it plays the same σ from scaled regrets as from raw ones. What changes is what the fitted net gets wrong, and so the σ it actually plays. A divisor drawn from the sampled hand's outcome would reweight samples within a spot and move its mean, which is a different algorithm.
  • What the power k does. Dividing by potᵏ leaves a weight of how often × pot^(2 − 2k). k = 0 is raw chips (pot²). k = ½ leaves how often × pot, the chips a σ mistake there actually costs. k = 1 weighs every spot alike, whatever its pot.

spot_pots gives the pot at each of the 141 public points. The default pot_power is ½ since 2026-10-27. Measured on one seed at 2M hands per seat, the exact average was 0.115 raw, 0.086 with ÷ √pot and 0.084 with ÷ pot. Run on to 20M hands, the ÷ pot arm flattens at 0.068. docs/notes/rung4-run-log.md (findings 7 and 8) has the caveats, including a gradient-clipping confound that no arm has separated yet.

rule_targets is the screen behind the DCFR and CFR+ arms. It rescales each sample by what the rule would weigh it at iteration now. Because Deep CFR stores no running total, the sign it tests is each sample's own rather than the cumulative regret's. That is why per-sample DCFR and CFR+ lost to linear by about 17×. See Regret rules.

Config and resume

DeepConfig is everything a run is:

FieldDefaultIs
seed0every stream derives from it
walks (K)10,000walks per seat per iteration
workers (W)8processes; changes nothing in the sample
capacity (C)50,000,000rows per reservoir
width, depth64, 3the net (the best arms ran 256, 4)
steps, batch, lr, clip4,000, 10,000, 1e-3, 1.0the refit
threads, device8, cputhe refit's arithmetic; device may be mps
rulelinearthe regret rule; others are the screen
pot_power0.5the k above
grid10, 20, 50, 100, 200, 500, 1,000, 2,000iterations at which a full mark is kept
save_every_s1,800seconds between resume saves

new_run builds a fresh DeepRun: iteration 0, no nets, three empty memories. save_run writes the three memories, the two latest nets (net_name: p{seat}-t{t:05d}.pt), every StatefulObserver's file (the exact average's) and, last and atomically, state.json. That file is the commit point: a crash anywhere in a save leaves the previous state.json naming the previous, complete files. load_run restores a run, saved_config reads a save's config from state.json alone, and config_from_saved fills any field in ADDED_LATER with the value old runs had. pot_power is the field that made the rule: runs saved before it existed fitted raw chips, and they must not resume under ½.

require_same_run refuses a resume that changes the seed, K, C, the net's size, any refit field or the thread count, naming each one. All of these change the sample or the fit. W may change because rows depend on (seed, t, seat, k) and never on W. The device may change on purpose so a run can move to the GPU, at the cost of no longer being reproducible from its seed.

run_deep trains out up to a target, a deadline or a signal: SIGTERM or Ctrl-C finishes the iteration and saves. At each grid mark it keeps a full save under marks/ (mark_name: t{t:05d}). current_iterate freezes regret matching over the materialised table for the grader.

Launching a run

WhereHow
this Macscripts/deepcfr/run_deep_cfr.py: argparse over DeepConfig (parse, config_from_args); re-running the same --out resumes; --device mps is checked (mps_available) before any walk is spent
Modalscripts/deepcfr/modal_rung4.py: deploy once, then ::smoke, ::sweep, ::train, ::status, ::grade; the trainer (one 33-core container) claims its directory on the drawmaha-rung4 Volume, saves resume/ every 30 minutes and a mark at each grid point, and spawns its own continuation at 23 hours

The smoke command, from the script's docstring:

caffeinate -i uv run --extra deep python scripts/deepcfr/run_deep_cfr.py \
    --out runs/deep-smoke --target 3 --walks 1000

On Modal, train spawns train_run. smoke runs benchmark, which trains three iterations at K = 1,000, resumes one more, grades the mark and prints seconds, hours and dollars for a full run. sweep launches 3 × 64, 4 × 64 and 3 × 128 to 100 iterations. With --grade, sweep_winner applies the pre-registered rule: stay at 3 × 64 unless another size grades lower by more than 10%. It picked 3 × 128. status / read_status print each run's last line.

Grading a mark: three objects, one untouched meter

grade (via grade_mark on Modal, or ::grade --local DIR for a Mac run) grades every kept mark (read_marks). grade_mark_dir freezes three objects from a mark directory and grades each with the exact meter in minidrawmaha/exploitability.py: the exact average's sums, Π fit fresh on the mark's policy memory (sharing the run's FitConfig), and the current iterate over the mark's nets. Results go to figures/rung4/grades.json or an arm's own file, keyed deep/<t>/<column>. uv run deep-analysis then draws the price figure and the answer sheet (see Mini-drawmaha answer).

At K = 10,000, iteration t has seen 10,000 · t hands per seat, the same as LCFR at lockstep iteration 1,000 · t. The eight grid marks therefore sit exactly on the race's 100k to 20M checkpoints, and the price is read at equal hands:

price = exploitability(Deep CFR's average) ÷ exploitability(LCFR's linear average)
Deep CFR ÷ LCFR at equal hands
Deep CFR ÷ LCFR at equal hands

Side studies

ScriptQuestion it answered
pi_loss_sweep.pyΠ stalled while the exact average kept falling: memory or fit? It refits Π on one stored memory under other losses, heads and schedules (loss_fn, fit, freeze) and grades each. The answer was the fit; cross-entropy beat squared error
pi_reach_weighted.pyWhich refit error tracks exploitability? Per refit, the reach-weighted mean absolute difference from the exact average (own_reach). That measure tracks it; a count of zeroed bluffs does not
rule_sign_flips.pyHow far does a per-sample rule bend the memory? seat_counts measures mass kept and bad actions turned good under DCFR and CFR+
distill_teacher.pyHow much of the 0.068 plateau is the net's capacity? It fits the net straight to the LCFR table's average (distill.distill, reach-weighted cross-entropy over every infoset) at several sizes and grades each copy; see Distilling the table into the net

Every arm and its grades is in docs/notes/rung4-run-log.md; Run log summarises it.

Site & Ops · The site

The site

Six static pages and five Vite apps, served from one public/ folder by Vercel, with every number read from a committed file.

The site at drawmaha-solver.vercel.app is how the project shows its work. It has two kinds of surface. Static pages are plain HTML written by a Python generator. Apps are Vite + React + TypeScript bundles that draw solves exported from Python. One shell script builds both into public/, and Vercel serves that folder with a handful of rewrites from vercel.json.

The rule that shapes the whole site is that the browser never solves anything. Every strategy, grade and curve on a page comes from a file the Python side wrote and committed, so a number on the site always has a source in the repository and a test that holds it to that source.

Every route and what serves it

RouteServesKind
/web/cover/index.html: the homepage, with the current result, the results table, the opening spot's range matrix and the research indexstatic, generated
/rulesweb/cover/rules.html: the rules of Drawmahastatic, generated
/benchmarksweb/cover/benchmarks.html: the four graded games on one pagestatic, generated
/releasesweb/cover/releases.html: every released strategy, with its tag and gradestatic, generated
/wikiweb/cover/wiki.html: this wiki, rendered from docs/wiki/static, generated
/research, /learn, /playweb/rung3-viz, opening on its Research, Learn or Play tabapp
/rung3the same app, choosing the tab by hashapp
/solverweb/solver-viz, the range explorerapp
/rung0, /rung1, /rung2the rock-paper-scissors, Kuhn and Leduc appsapp
/coverweb/cover/cover.html, the old front page, kept as an archive and unlinkedstatic, hand-kept
/heroweb/hero-studies/, a share page with video clipsstatic, hand-kept

web/AGENTS.md keeps the same table. When the two disagree, vercel.json is the truth: scripts/site/serve_site.py reads its rewrites rather than mirroring them, and tests/test_vercel_config.py checks that every app the build emits has a rewrite.

How a number gets from Python to a page

solver run → committed file (figures/, an export, a release asset) → generator or app → page
  • Solver run. An analysis command (kuhn-analysis --json, minidraw-ranges, minidraw-deals, deep-analysis) or a trainer writes its output.
  • Committed file. Grades live under figures/. The apps' exports live under web/*/public/. Copies the apps import live under web/*/src/data/. Large frozen strategies are GitHub release assets, pinned by SHA-256. None of these is edited by hand.
  • Generator or app. The static pages are rewritten by make_static_pages.py. An app imports its data copy or fetches its export at runtime.
  • Tests at each join. tests/test_cover_page.py fails when a committed page differs from what the generator writes now. tests/test_rung3_page_data.py holds each src/data/ copy byte for byte to its figures/ original. The Python exporter tests hold each manifest's grade to its grades file.

So a re-run that moves a number has to show up as a diff in the committed file, the page, or a red test. It cannot drift silently.

One shared look

Everything the reader sees is one visual system, defined in a single theme file:

  • The theme is one file in five copies. web/rung0-viz/src/theme.css holds the tokens (paper, ink, one oxblood accent), the top bar, the footer, figures, hairline tables and the contents rail. Each of the other four apps carries a byte-identical copy, and the static pages link the build's copy at /theme.css. See the design standard.
  • The chrome is written twice. site.tsx renders the top bar and footer in the apps, and the generator writes the same markup out as HTML for the static pages. test_the_pages_chrome_says_what_site_tsx_says checks that the two copies say the same words.
  • The fonts are self-hosted. Newsreader, Geist and IBM Plex Mono are copied out of rung 0's @fontsource packages into the site's fonts folder and declared once in /fonts.css. Nothing is fetched from another host.

Site & Ops · The site

The static pages and their generator

Five pages written by make_static_pages.py from committed data, one archived page kept by hand, and the tests that keep the generated ones regenerated.

The homepage, Rules, Benchmarks, Releases and this wiki are plain HTML files under web/cover/. They are not written by hand. Each one is the output of scripts/site/make_static_pages.py, a standard-library-only Python script that reads committed files and writes the pages. Each page names the generator in a comment above its <head>.

Generating them, rather than editing HTML, solves one problem. The pages quote numbers (exploitabilities, game values, release grades), and a hand-typed number goes stale the moment a run is regraded. The generator reads every number from the file that holds it, so regenerating the pages is how they stay true.

Changing a page means changing the generator

uv run python scripts/site/make_static_pages.py

Edit the generator (or the data it reads), run it, and commit the regenerated pages in the same change. tests/test_cover_page.py renders every page in memory and fails if a committed page differs (test_the_pages_are_what_the_generator_writes). A hand edit to index.html therefore shows up as a test failure, not as a quiet fork.

Data sources

PageReads
index.html (/)the README's results table; figures/rung3/ and figures/rung4/ grades for the headline chart; the frozen table's export under web/solver-viz/public/mini/ (the opening matrix is rebuilt from it and checked against the answer sheet); the first chunk of the Play tab's deal pack, for a dealt hand
rules.html (/rules)the rules, written in the generator
benchmarks.html (/benchmarks)the README's results and Kuhn tables; the Kuhn and Leduc solves the live demos draw; the rung 3 and rung 4 grade files
releases.html (/releases)the release catalogue in src/drawmaha_solver/minidrawmaha/answer/releases.py, read with ast rather than imported, plus each release's grade file
wiki.html (/wiki)every page under docs/wiki/, in the order docs/wiki/nav.json gives

Two kinds of fact have no committed home: the publication dates of the release tags, and the dates in the homepage's research index. They are set in the generator beside a comment naming their source (RESEARCH_DATES, PUBLISHED).

The wiki is a generated page too

This wiki is written as markdown under docs/wiki/. scripts/site/wiki.py renders that markdown, and the generator wraps the result in the site's chrome as web/cover/wiki.html. The source layout, the markdown dialect (callouts, cards, eq blocks, wiki: and repo: links) and the writing rules are in docs/wiki/README.md. An edit to a wiki page follows the same loop as any other page: edit the markdown, regenerate, commit both.

Shared assets

  • /theme.css is rung 0's theme, copied by the build. The chrome, tokens and type come from there, and each page adds one inline <style> block of its own layout and chart classes (the generator's BASE_CSS plus a per-page sheet).
  • /fonts.css declares the @font-face rules for the files in the site's fonts folder. A page never names a font file itself.
  • /chip.js is the plain-JS twin of the apps' src/ui/mark.ts. Each page mounts the turning chip twice, once in the top bar and once in the footer.
  • /favicon.svg is the chip, flat.

What test_cover_page.py holds

CheckWhy it exists
every href reaches a route vercel.json serves, or the repositorya link to a page that was never deployed is the commonest silent break
every #anchor into a page lands on an id that page hasanchors break invisibly when a heading is renamed
nothing loads from another hostno CDN; the site works from its own origin
fonts.css declares exactly the files the build copies, and the faces the apps bundlea declared font that is never served falls back silently
each page links the theme, the fonts and the chip, and mounts the chip twicethe chrome comes from one place
chip.js draws the same chip as mark.tstwo hand-kept copies of one drawing
every inline script parsesa syntax error kills the page's scripts with no build step to catch it
the product pages never say "rung", "ladder" or "climb"the method's words belong in the README and this wiki; the product pages speak the product's words
the pages are byte for byte what the generator writesno hand edits
the top bar and footer say what site.tsx saysthe chrome is written twice

The design rules (motion, tokens) for the pages' inline styles are checked separately by tests/test_ui_standard.py. See the design standard.

Looking at the pages locally

The real check is the full build (see Deploying), but that runs five npm ci builds first. To look at the static pages alone, copy them into a temp folder shaped like public/ and serve it with python3 -m http.server. web/cover/README.md has the exact commands under "Checking them locally". Extensionless routes (/rules) are Vercel rewrites and will not resolve there, so open each page by file name. Look for an empty console, a 200 for every font request, and no request leaving the host.

Site & Ops · The site

The five apps

Vite + React + TypeScript visualizers that draw solves exported from Python, transcribe only rules arithmetic, and share one copied chrome.

Five apps live under web/, one per interactive surface. Each is a Vite + React + TypeScript project with the same four commands:

cd web/<app>
npm install
npm test        # vitest
npm run dev     # local dev server
npm run build   # tsc -b && vite build, into dist/

The apps never run CFR

No app reimplements CFR or anything else that touches the solver. Each one draws a solve that Python exported (kuhn-analysis --json, leduc-analysis --json, minidraw-deals, minidraw-ranges), or it POSTs a locked strategy to a local Python exploit server and draws the trace that comes back. The reason is the same one that keeps graders apart from solvers (Exploitability). A second copy of the reach-weighted tree walk is exactly the routine where a swapped weight converges, silently, to a wrong answer.

What an app may transcribe is rules arithmetic: payoffs, the betting grammar, the hand-ranking key. Each transcription has a *.test.ts that pins it against the same tables the Python suite pins, so a port that disagrees with Python fails a test.

The five apps

AppRouteDrawsData
rung0-viz/rung0regret matching learning rock-paper-scissors livenone needed; the 63-line ledger is small enough to port safely
rung1-viz/rung1Kuhn's twelve information sets converging, plus the Exploit tabkuhn-analysis --json, copied into src/data/; Exploit posts to kuhn-exploit-server
rung2-viz/rung2Leduc's solve on a timeline, a playable table, an exploit tableduc-analysis --json; leduc-exploit-server
rung3-viz/research, /learn, /play, /rung3the research publication, the Learn range interface and the trainercopies of figures/rung3/ and figures/rung4/ under src/data/; the deal pack under public/deals/
solver-viz/solverthe range explorer: mini-drawmaha exactly, full Drawmaha with an illustrative policythree exports under public/mini*/ written by minidraw-ranges

rung3-viz: one app, three tabs

The tab comes from the path (/research, /learn, /play) or, at /rung3, from the hash. parseLocation in src/learn.ts reads it.

  • Research is a publication, not a dashboard: a masthead whose results strip computes its three numbers from the data, a contents rail with scroll-spy, nine chapters and Methods. Its numbers come from exactly three modules, src/research.ts, src/deep.ts and src/reach.ts, each with a vitest file that pins what the prose quotes.
  • Learn is /solver's mini-drawmaha range interface, imported from ../solver-viz/src/ui/MiniRange.tsx, not copied. A change to the range interface is made in solver-viz and shows up here.
  • Play deals from the pack written by uv run minidraw-deals: 600 deals in six chunks, each carrying the shipped strategy's mix at every node either player can reach. The bot is the release deal_pack.PACK_RELEASE names, which is the Deep CFR exact average from the ÷ √pot arm at iteration 200.

Learn fetches its exports from /solver/mini*/, the one copy on the site, so npm run dev inside rung3-viz cannot load them. To drive Learn, build the whole site and serve it (Deploying).

solver-viz: two games, not equally real

  • Mini-drawmaha is exact. Three strategies sit behind one switch: the rung 3 tabular LCFR, and rung 4's Deep CFR at iteration 500 as Π (the policy net) and as the exact average. The three exports share their key tables and point walk byte for byte and differ only in the chunks, so switching changes the strategy under a spot while the line and the filters stay. The browser rebuilds the actor's range from the export, weighted by the actor's own action probabilities along the line.
  • Full Drawmaha is the interface built ahead of the network. The range's composition (src/engine/sample.ts) is sampled and real. The action mix is illustrative, a smooth function of each hand's two equities in src/engine/policy.ts, and every view drawing it says so in its status line. When the network exists, that is the file that changes.

The ports of the Python key, hand ranking, betting grammar and public tree (src/mini/canonical.ts, ranking.ts, grammar.ts, points.ts) are pinned to src/mini/fixtures.json. The Python exporter writes that file only when told (--write-fixtures). When the Python key or grammar changes, re-pin the fixtures on purpose and re-export every strategy. Never patch a port to agree with stale data.

Five files are one file

Five files are copied, byte for byte, into every app: src/theme.css, src/main.tsx, src/ui/site.tsx (the top bar, page head, figures and footer), src/ui/site.test.tsx and src/ui/mark.ts (the turning chip). They are copied rather than extracted into a shared package on purpose. The apps deploy independently, and a workspace would buy about 300 lines at the cost of a build step.

The price is drift, and tests/test_shared_chrome.py is what pays it. Edit rung 0's copy, cp it to the other four, and cmp them. Per-app styling goes in that app's own src/index.css, on top of the theme.

Adding a surface

A new app or page touches, in the same change:

  1. vercel.json: a rewrite for its route.
  2. scripts/site/vercel_build.sh: the rungs list for an app, or the page loop for a static page.
  3. The APPS tuples in tests/test_shared_chrome.py and tests/test_ui_standard.py, or PAGES there and in tests/test_cover_page.py for a page.
  4. If the nav names it: NAV and the Surface type in site.tsx (copied to all five apps), and NAV in the generator, then regenerate the pages.

test_every_rung_the_build_script_emits_has_a_rewrite exists because a surface added to the build but not to the rewrites builds fine and is then never served.

Site & Ops · The site

The design standard

Paper, ink and one oxblood accent; two type voices; motion read from three tokens; and a test that enforces the mechanical half on every stylesheet.

web/DESIGN.md is the UI standard, and it is enforced. Every surface is held to it: the static pages, the archived Cover, and the five apps. The standard treats taste the way Emil Kowalski describes design engineering, as a set of decisions made the same way every time. Most of them nobody consciously notices, and together they make software feel right.

The standard has two halves. The mechanical half (which curve, how long, what a press does, what reduced motion keeps) is checked by tests/test_ui_standard.py on every stylesheet. The judgement half is a review table at the end of DESIGN.md. Before you touch a stylesheet, a <style> block or a component's classes, read DESIGN.md. Afterwards, run:

uv run pytest tests/test_ui_standard.py tests/test_shared_chrome.py tests/test_cover_page.py

The visual system in six decisions

DecisionWhat the theme declares
Paper, ink, one accent--paper #f7f5f0 is the page and --ink #16140f the text. --accent #8e2038 is oxblood, the deeper shade of the chip's red, and the only accent: the winning series in a chart, bets and raises, links, the focus ring
Semantic chart and action tokens--series-1/2/3 are ink, accent and muted ink, with a dash pattern for a fourth series and a direct label at the end of every line. The action triplet is --act-bet (accent), --act-call #2f7d4a green and --act-check #2f66a3 blue (check and fold)
Two voices and a monoNewsreader (--serif) for display, headings and research prose; Geist (--sans) for UI, tables, charts and numbers, with tabular numerals; IBM Plex Mono (--mono) only for the chip's glyph grid and for code
One scaledisplay 64, h1 52, h2 32, h3 22, lede 21, research body 18 serif, UI 15, captions 13 / 12 / 11
One grid1160 px of content (--max) inside a 40 px gutter, 12 columns on a 24 px gap
Hairlines, not boxesevery container is a top or bottom hairline, never a shadowed box; the only radii are 6 px on buttons and controls

Playing cards follow one drawing too: the theme's .cardface, the "bleed pip". The rank sits top-left in Geist 600, and one oversized suit glyph is cropped by the card's lower-right edge. Spades wear ink, hearts the oxblood, diamonds the check-blue and clubs the call-green. A surface sets only the card's width, --w.

Motion reads three tokens

--ease-out: cubic-bezier(0.23, 1, 0.32, 1);     /* anything entering or settling */
--ease-in-out: cubic-bezier(0.77, 0, 0.175, 1); /* a thing that moves while on screen */
--t-press: 160ms;                               /* the press */

The built-in keywords are too weak to read as intended. ease is kept for exactly one job: a colour, background or border changing under the pointer, at 150 ms. ease-in is never used. It delays the first frame of movement, which is the frame the user is watching, so 300 ms of ease-in feels slower than 300 ms of ease-out.

Before animating anything, decide in this order:

  1. How often will the user see it? A keyboard action, or a bar that updates a thousand times a second, never animates; the number printed beside it would disagree with the bar. A hover gets 150 ms of colour. A sheet opening is occasional and gets the full treatment.
  2. What is it for? Press feedback, state, spatial consistency, or preventing a jarring swap. "It looks cool" is not a purpose for anything seen more than once.
  3. Which curve? Entering or settling uses --ease-out. Moving while on screen uses --ease-in-out. Colour under the pointer uses ease. Constant motion uses linear.
  4. How long? A press is 160 ms, hover colour 150 ms, a popover or sheet 150–250 ms. Nothing in the chrome runs over 300 ms per declaration, delays included. Exits are faster than entrances.

What the test enforces

RuleExample on the site
everything pressable shrinks to scale(0.97) for 160 ms on :active.btn, .navbtn, .view, .seg button, .radios label, .rail a, and the solver's cells, keys and deck cards (PRESSABLES in the test)
a disabled control does not answer a press.btn:active:not(:disabled)
no transition: all, no ease-in, no hard-coded curve in the chrome.sweep uses 0.4s var(--ease-out); it used to be 0.55s ease-in
only transform and opacity animate in the chromea bar that moves uses transform: scaleX(), never width
nothing moves on loadthe turning chip is the one thing that moves without being asked
a hover that moves something is gated on a real pointerinside @media (hover: hover) and (pointer: fine), because touch fires hover on tap and never un-fires it
reduced motion means fewer and gentler, not none, named by selectorthe theme quiets .btn:active:not(:disabled), not bare .btn:active, because :not() adds specificity
a one-shot that code waits on still ends under reduced motionthe swept chips switch to a fade-only keyframe, so animationend still fires and unmounts them

A motion that explains rather than decorates can be exempted. EXPLANATORY in the test lists each such selector with its reason: today .sweep (chips crossing the felt to the pot, 400 ms) and the archived Cover's three. test_every_allowlisted_selector_names_a_rule_that_moves fails on an entry that exempts nothing, so a stale exemption cannot hide the next real one.

Reviewing UI

A UI review in this repo is a single markdown table with Before | After | Why columns, one row per finding:

BeforeAfterWhy
transition: all 300mstransition: transform var(--t-press) var(--ease-out)name the property; all animates things you did not mean
.foo:hover { } and no :activeadd .foo:active { transform: scale(0.97) }a press has to be heard

Review a change the next day, at 4× slow motion in the DevTools Animations panel, and on a phone over the LAN.

Where the CSS lives
  • The theme (tokens, top bar, page head, buttons, figures, hairline tables, .views, .seg, .radios, .stats, .rail, footer): web/rung0-viz/src/theme.css and its four byte-identical copies. Edit rung 0's and cp it across.
  • Each app's own figures and layout: web/<app>/src/index.css, on top of the theme.
  • The generated pages: one inline <style> each, written by scripts/site/make_static_pages.py. Change the generator, never the HTML.
  • The archived Cover: its old chrome inlined, minified, one rule per line.

Site & Ops · Running things

Testing: the suite is the CI

No workflow runs tests; uv run pytest before every PR is the gate, one directory per game, with fixtures re-derived a second way and heavy work gated by environment variables.

CI runs no tests. The only GitHub workflow, .github/workflows/claude-code-review.yml, is an automated code review. The gate is the suite you run yourself:

uv run pytest                      # the whole default suite, about seven minutes on a laptop
uv run pytest -x tests/kuhn        # one game's directory: the fast loop
cd web/<app> && npm test           # vitest, in any app you touched

Run both before you open a PR. The default suite needs no torch, no Modal, no OpenSpiel and no network download, and it has to stay that way. Anyone with uv sync can check the whole project.

How the suite is laid out

DirectoryHolds
tests/rps/the rules table, players, convergence to the uniform Nash
tests/kuhn/the payoff table, tree shape, infoset inventory, one hand-worked traversal, convergence, the 64-strategy grader, the exploit server's table
tests/leduc/the same, plus referee.json (frozen from OpenSpiel) and test_referee.py, which re-derives every fact in it from our code
tests/minidrawmaha/cards, hand ranking, the game, the census, the table, MCCFR and lockstep, regret rules, the parallel collector, exploitability, strategy files, releases, exports and the deal pack
tests/deepcfr/the reservoir, net, fit, collect, materialise, average and policy modules; training (torch-gated); the Modal launcher's logic; the run log's quoted numbers
tests/test_regret_matching.pythe shared ledger, including that the two reach weights stay distinguishable
tests/test_cover_page.py, test_ui_standard.py, test_shared_chrome.py, test_vercel_config.pythe site: links, fonts, generated pages, design rules, copied chrome, the deploy contract
tests/test_rung3_page_data.pyeach /rung3 data copy equals its figures/ original, byte for byte
tests/test_agent_docs.pyevery path an AGENTS.md names exists, and every AGENTS.md has its CLAUDE.md shim

Fixtures are re-derived, not trusted

Some ground truth is expensive or comes from outside the repo, so it is computed once and committed:

  • tests/leduc/referee.json comes from OpenSpiel, through scripts/leduc/generate_referee.py.
  • tests/minidrawmaha/census.json comes from enumerating the game's own rules, through scripts/minidrawmaha/generate_census.py. It takes about fifteen seconds and counts all 6.2 million infosets one at a time.

The tests then rebuild every number in the fixture a second, different way and demand that the two agree. A fixture that only asserted itself would keep passing while the solver rotted. For the same reason, fixtures are regenerated by their script and never edited.

The same idea covers numbers quoted elsewhere. minidrawmaha/test_play.py reads figures/rung3/grades.json. minidrawmaha/test_range_export.py reads the committed /solver export. kuhn/test_exploit_server.py recomputes the README's exploit table from the committed solve. deepcfr/test_rung4_quotes.py checks the run log's scoreboard and the README's rung-4 table against figures/rung4/. A re-export changes a test's inputs, never its assertions.

Heavy work is behind a named switch

Tests that would take minutes or gigabytes are skipped by default. Each skip reason names the environment variable that turns it on:

VariableTurns on
MINIDRAWMAHA_FULL_TABLEbuilding the full 6.2M-row table
MINIDRAWMAHA_FULL_GRADEthe exact best-response walk on a full strategy
MINIDRAWMAHA_FULL_CALIBRATIONthe calibration runs
MINIDRAWMAHA_FULL_DEEPDeep CFR end to end

Tests that need torch call pytest.importorskip("torch"). Install it with uv sync --extra deep to run them.

Three gotchas

  • --import-mode=importlib is set in pyproject.toml, because every game has its own test_game.py and identical basenames would collide at collection. So test modules never import one another (shared helpers are restated, and say so), and tests/ holds no __init__.py.
  • Spawned workers unpickle by module name. A test that hands a job to the parallel collector defines that job at module level and puts the repo root on sys.path first, as minidrawmaha/test_parallel.py and deepcfr/test_collect.py do. A lambda or a closure cannot reach a worker process.
  • Convergence thresholds are seeded and justified in a comment beside them. Loosening one to make a test pass is a finding to report, not a fix.

Site & Ops · Running things

Training runs on a Mac and on Modal

Every trainer is resumable from its output folder; the Modal launchers deploy first, claim each run with a heartbeat, and continue themselves across the 24-hour cap. Only a person launches them.

Two pieces of training need more than a laptop afternoon. One is the mini-drawmaha race: four regret rules, each run to 2M lockstep iterations. The other is Deep CFR's runs: 4 × 256 nets, up to 2,000 iterations of 10,000 walks per seat. Each has a local script for a Mac and a Modal launcher for the same work in the cloud.

The local trainers resume from their output folder

caffeinate -i uv run python scripts/minidrawmaha/run_lockstep.py lcfr --out runs/lockstep-lcfr
caffeinate -i uv run --extra deep python scripts/deepcfr/run_deep_cfr.py --out runs/deep-smoke --target 3 --walks 1000
  • --out is the run. Re-running the same command resumes from what is in the folder: resume.npz for lockstep, <out>/resume/ for Deep CFR, the exact average included.
  • Ctrl-C or SIGTERM is safe. The trainer finishes the current iteration and saves.
  • caffeinate -i keeps the Mac awake. Mac sleep still counts toward the training clock, so a run's wall-clock figures include any sleep it suffered.
  • What a resume may change. For Deep CFR, --workers and --device may change on resume. The seed, walks, capacity, net size and --pot-power may not. --device mps is checked against the machine before the run loads, so a missing GPU costs no walks.

The rows a Deep CFR run collects do not depend on the worker count, because collect.py draws every walk from (seed, t, seat, walk). A Mac run and a Modal run of the same config see identical data. Only the refit's arithmetic order can differ.

The Modal launchers share one shape

uv run --with modal modal deploy scripts/deepcfr/modal_rung4.py          # once, and after any code change
uv run --with modal modal run scripts/deepcfr/modal_rung4.py::smoke      # seconds per step, priced
uv run --with modal modal run scripts/deepcfr/modal_rung4.py::train      # the run, resumable
uv run --with modal modal run scripts/deepcfr/modal_rung4.py::status     # where every run stands
uv run --with modal modal run scripts/deepcfr/modal_rung4.py::grade      # every kept mark → figures/rung4/grades.json

scripts/minidrawmaha/modal_rung3.py has the same verbs for the race (smoke, main --target … --rules …, status). Every launcher's docstring opens with its exact commands. The design behind both:

  1. Deploy before you run. modal deploy first, then modal run …. The run spawns on the deployed app, so it outlives your terminal and can spawn its own continuation.
  2. One container trains one run. The race gives each rule a container of 11 cores (10 workers and the parent) and writes to the Volume drawmaha-rung3 under <rule>/ (cfr+ is stored as cfrplus/). Deep CFR gives a run one container of W + 1 cores (32 walk workers and the parent) and writes to drawmaha-rung4 under the run's name, in exactly the layout a Mac run writes: nets/, marks/t00010/ …, resume/ and progress.jsonl.
  3. A heartbeat claims the run. Each trainer writes owner.json at every save. A newcomer refuses to start while another heartbeat is under 15 minutes old, unless that trainer's last progress line says it stopped and saved (a preemption), in which case the newcomer takes over. Two newcomers in the same second are separated by a 30-second settle and a re-read.
  4. Runs cross Modal's 24-hour cap by themselves. A container stops after 23 hours, saves, and spawns its own continuation. Saves are committed at once, so a container that dies loses at most the time since its last save.

Deep CFR's claim, heartbeat and continuation are lifted from the race's launcher rather than imported, because that script builds its own app and Volume at import time.

Grading fans out one container per mark

modal_rung4.py::grade starts one container per kept mark. Each loads the mark's memories, nets and sums and grades three objects with the untouched exact meter:

ObjectWhat it is
exact averagethe true average strategy over every iteration's nets, carried along by an observer from iteration 1. It cannot be rebuilt later without replaying every kept net
Πa fresh policy net fit on the policy memory as it stands at the mark: what a deployed Deep CFR would actually play
current iteratethe strategy this iteration's advantage nets imply, which cycles and is graded to show it

Beside each grade go hands_per_seat (t · K, which lines a mark up with the tabular race), wall_clock_s and the run's config. The local entrypoint writes everything into figures/rung4/grades.json, keyed deep/<t>/<exact|pi|current>. A grade is a pure function of the mark, so regrading is idempotent. Then uv run deep-analysis draws the figures from the grades file. The three graded strategies are also kept on the Volume under frozen/, which is where a release comes from.

Run records

Every Deep CFR arm trained so far, with where it ran, its net and refit settings, and its graded marks, is in docs/notes/rung4-run-log.md. Add a row when an arm is graded. Every number there is read from a file under figures/rung4/, and tests/deepcfr/test_rung4_quotes.py checks the scoreboard against those files. See the run log page for what the arms showed.

Grading a checkpoint by hand
uv run python scripts/minidrawmaha/grade_checkpoint.py runs/mccfr-seed0/solve.npz
uv run python scripts/minidrawmaha/grade_checkpoint.py runs/dcfr/solve.npz --column quadratic

This grades one of a lockstep checkpoint's averages (linear by default, or any extra column the run banked) and prints both seats' best-response values, the exploitability and the expected value, with timings. Budget about 2 GB of memory: the packed table is about 0.3 GB, and the grader holds 1.5 GB of showdown tables.

Site & Ops · Running things

Releases: frozen strategies the site ships

Each shipped strategy is a GitHub release asset pinned by SHA-256 in one catalogue, with its grade read from a committed file, so every export is reproducible from a clean checkout.

A trained strategy is far too big to commit. The frozen LCFR table alone is an 18 MB file. Yet the site has to say exactly which strategy a page plays and how exploitable it is. Releases are how both are done. The strategy file is published as a GitHub release asset, and the repository commits only its name and SHA-256 digest, plus a pointer to the committed file that holds its grade.

The one place this is all written down is the catalogue, RELEASES in src/drawmaha_solver/minidrawmaha/answer/releases.py. The Releases page is generated from it.

The catalogue, entry by entry

--release nameTagWhat it isShipped on
tabularrung3-lcfr-2mthe tabular LCFR average at 2M lockstep iterations, the race's winner/solver (public/mini/), /learn
deep-pirung4-deep-t500Deep CFR 4 × 256 at iteration 500, as Π (the policy net)/solver (public/mini-deep-pi/)
deep-exactrung4-deep-t500the same run's exact average/solver (public/mini-deep-exact/)
deep-pot-exactrung4-deep-pot0.5-t200the ÷ √pot arm's exact average at iteration 200/play (the deal pack)
deep-pot-pirung4-deep-pot0.5-t200that arm's Πcatalogued, not exported

Each entry is a Release with seven fields: name, directory (where /solver's export lands, or None for a strategy shipped another way), tag, asset, sha256, grade_file and grade_key (the object in the grades file holding its exploitability and value_p0).

Why each field exists

  • Published by tag and asset, refused unless the digest matches. fetch_strategy downloads the asset once into ~/.cache/drawmaha-solver (or $XDG_CACHE_HOME/drawmaha-solver) and re-hashes the cached copy on every call. A clean checkout can therefore reproduce every export, and a local copy is identified by its digest (release_of) rather than by its file name.
  • The grade is read, never typed. At export time, read_grade copies the strategy's verdict from its committed grades file into the export's manifest. The tests hold the manifest to the file, so a page quotes a number that has a committed source and never one typed into TypeScript.
  • One provenance block for both exporters. provenance_json writes the strategy block that both /solver's index.json and the deal pack's manifest open with: digest, tag, asset, rule, column, iteration, workers and seed. The two pages can therefore share the words they use about a strategy (describeStrategy in solver-viz).
  • A strategy without a directory cannot reach /solver by accident. minidraw-ranges refuses to export it without an explicit --out.

Cutting a release

  1. Freeze the strategy into a file. For a tabular checkpoint: ``bash uv run python scripts/minidrawmaha/freeze_strategy.py \ runs/modal/lcfr/iter-002000000.npz runs/strategy/strategy-lcfr-2m.npz --grade ` This prints the file's size and SHA-256. --grade re-grades the written file itself (about two minutes, 2 GB), and a faithful freeze matches the checkpoint's own exploitability to the sixth decimal. For Deep CFR, the grader has already saved the three graded objects under frozen/` on the run's Volume (Training runs).
  2. Make sure the grade is committed under figures/. The release will point at it.
  3. Publish the GitHub release with the file as its asset. The tags so far are rung3-lcfr-2m (2026-10-02), rung4-deep-t500 (2026-10-26) and rung4-deep-pot0.5-t200 (2026-10-28).
  4. Add the Release entry with its digest, grade file and key.
  5. Export it to wherever it ships: ``bash uv run minidraw-ranges --release <name> # /solver: web/solver-viz/public/<directory>/ uv run minidraw-deals # /play: the pack for deal_pack.PACK_RELEASE ``
  6. Regenerate the static pages (uv run python scripts/site/make_static_pages.py), so /releases lists it, and commit everything together.
What an export holds
  • /solver's export cuts the strategy by public decision point. Each of the 141 points is one column of the table, and every point of a shape (board cards × discards) stands on one sorted key list written once: 970 keys at b1d0, 10,170 at b2d0 and 100,400 at b2d1. Probabilities are stored as bytes that sum to a scale of 250, so each entry is within 0.4% of the strategy's own number. The whole strategy is 14 MB raw and about 4 MB gzipped, against 57 MB of float32.
  • The deal pack is 600 deals dealt in advance, in six chunks. Each deal carries the strategy's per-mille mix at every node either player can reach, plus all 16 post-draw showdowns. The browser follows the path it is on and reads the vector, so it never has to transcribe the infoset key. The pack holds both holes, so the bot's cards can be read from the network tab. That is accepted for a trainer.

Site & Ops · Running things

Deploying to Vercel

One build script lays every app, font and static page into public/, framework detection is pinned off, and serve_site.py reproduces the rewrites locally.

The site deploys on Vercel from vercel.json. There is no server and no framework: Vercel runs one build script, serves the folder it writes, and applies a list of rewrites. Every PR gets a preview deployment through Vercel's GitHub integration, which shows up as a status on the PR.

What vercel.json says

{
  "framework": null,
  "buildCommand": "bash scripts/site/vercel_build.sh",
  "outputDirectory": "public",
  "rewrites": [{ "source": "/rules", "destination": "/rules.html" }, "…"]
}

The build command lives in a script because Vercel caps buildCommand at 256 characters.

The build script

scripts/site/vercel_build.sh, in order:

  1. Builds each app. For rung0 rung1 rung2 rung3 solver, it runs npm ci && npm run build in web/<app>-viz and copies dist/ to public/<app>. rung3 builds before solver (test_rung3_builds_before_solver_in_the_build_script).
  2. Copies the hero studies to public/hero.
  3. Copies the fonts. Every app bundles the same @fontsource packages, so the script copies rung 0's woff2 files into the build's fonts folder: Newsreader 400, 500 and 400 italic; Geist 400, 500 and 600; IBM Plex Mono 400 and 600; and Instrument Serif, kept only for /hero.
  4. Copies the shared assets: rung 0's theme.css to /theme.css, plus fonts.css, chip.js and favicon.svg from web/cover/.
  5. Copies the static pages from web/cover/ to the root of public/. Vercel serves index.html at / without a rewrite, and the rewrites map /rules, /benchmarks, /releases, /wiki and /cover to their files.

The static pages are not built here. They are generated ahead of time and committed (The static pages). So a deploy can never produce a page that differs from the one reviewed in the PR.

The deploy contract, as tests

Test in tests/test_vercel_config.pyHolds
test_framework_detection_is_pinned_off"framework": null
test_static_output_is_served_from_publicthe output folder is public/
test_the_cover_page_is_served_from_the_filesystem/ needs no rewrite
test_the_product_paths_reach_their_pages/research, /learn, /play and the other product routes land on their files
test_every_rung_the_build_script_emits_has_a_rewritethe rungs=(…) list and the rewrites move together
test_what_rung3_imports_from_solver_viz_imports_only_reacta sibling-app import that would resolve on a laptop but not on Vercel

Checking the whole site locally

bash scripts/site/vercel_build.sh          # build public/ (five npm ci builds first)
uv run python scripts/site/serve_site.py   # http://localhost:4321

serve_site.py serves public/ filesystem-first and applies the rewrites it reads from vercel.json. It does not mirror them, because a hand copy silently missed each new route's rewrite. It is standard library only, so it can never break on the solver's numeric stack. This is the only local way to drive /learn, which fetches its exports from /solver/mini*/.

Debugging a deploy

SymptomUsual cause
a route 404s on the preview but works locally in npm run deva missing rewrite in vercel.json, or the app missing from rungs=(…)
every path returns a function errorframework detection is back on
an app's build fails on Vercel onlyan import from a sibling app that only resolves when both are installed
text renders in a fallback facea font that fonts.css declares but the build does not copy
a static page shows an old numberthe generator was not re-run after a data change; test_the_pages_are_what_the_generator_writes fails locally

Research · Where it stands

Where the project stands

The tabular answer to mini-drawmaha, the Deep CFR net beside it at equal hands, and what comes next.

Mini-drawmaha is solved twice: once by a table (LCFR, rung 3) and once by a network (Deep CFR, rung 4), both graded by the same exact best-response walk. Full Drawmaha, the game the project exists for, is not started. This page is the scoreboard. The arm-by-arm record is the run log, and the next unknowns are under open questions.

Every number below is exploitability: what a perfect adversary wins against the strategy, averaged over the two seats, in chips per hand with an ante of 1. Zero means equilibrium. For scale, playing uniformly at random on mini-drawmaha is exploitable for 4.842 (figures/rung3/grades.json, key uniform_random).

Every graded game, one row each

GameMethodGraded againstResult
Rock-paper-scissorsregret matchingthe uniform Nash0.0009 chips/round after 100k rounds
Kuhn pokervanilla CFRthe closed-form equilibrium; all 64 pure strategies0.00063 chips/hand; game value −0.05555 against −1/18
Leduc pokervanilla CFRan OpenSpiel solve, frozen as a fixture0.011 chips/hand; game value −0.0874 against −0.0856
Mini-drawmaha, 6.2M infosetslockstep MCCFR, four regret rules racedthe exact best-response walkLCFR wins: 0.061 at 20M hands per seat
Mini-drawmahaDeep CFRthe same walk, and LCFR's answer0.068 at 10M hands per seat (LCFR 0.126); still 0.068 at 20M, where LCFR is 0.061
Full DrawmahaDeep CFR at scalea head-to-head checkpoint ladder and a trained exploiternot started

The table is the README's results table; Benchmarks draws each row's chart.

LCFR wins the four-rule race on mini-drawmaha

Rung 3 trained four regret rules for 2,000,000 lockstep iterations each — ten hands per seat per iteration, so 20M hands per seat — and graded every checkpoint exactly (figures/rung3/grades.json, linear average, seed 0):

hands per seatvanillaLCFRCFR+DCFR
100k3.0213.0483.4183.442
1M1.1451.1311.5641.560
5M0.3130.2640.5530.553
10M0.1670.1260.3690.370
20M0.0860.0610.2480.250

The two rules that keep negative regret (vanilla, LCFR) beat the two that throw it away (CFR+, DCFR) by 3–4×. Under sampling a row is revisited so rarely that DCFR's halving of negative regret drives it to exactly zero between visits, which is CFR+'s floor — why that happens, and why linear weighting is immune, is the subject of regret rules.

The winner is frozen as the project's tabular answer: LCFR's linear average at 2M iterations, 14.25M probabilities, released as rung3-lcfr-2m and graded at 0.061346 (BR₀ −0.034421, BR₁ +0.157114). In self-play the first seat is worth −0.0906 a hand: the second seat acts last in both rounds and at the draw, and that is worth about a tenth of an ante.

Deep CFR vs the table

Rung 4 replaces the 6.2M ledgers with a network and asks one question at every point of the race's grid:

price = exploitability(Deep CFR's average) ÷ exploitability(LCFR's average), at equal hands per seat

Read it as how many times more exploitable the net is than the table after both have seen the same hands. Below 1 the net is ahead; above 1 the table is. The hands are made equal on purpose — Deep CFR walks K = 10,000 hands per seat per iteration, so its iteration t has seen what LCFR saw at lockstep iteration 1,000·t, and the eight marks t = 10 … 2,000 land exactly on the race's checkpoints. Equal hands is not equal compute: a Deep CFR iteration spends most of its time refitting two nets (see finding 6 of the run log).

The best arm — a 4 × 256 net, a 16,000-step refit at learning rate 5e-4, each regret target divided by its spot's pot — was run to the race's endpoint (figures/rung4/pot_1-256x4-s16000-lr5e-4.json):

hands per seat1M2M5M10M20M
Deep CFR, exact average0.1140.0840.0710.0680.068
LCFR, linear average1.1310.6300.2640.1260.061
price0.100.130.270.541.12
training clock, hours6.210.622.551.7100.7

Three readings, all on one seed and one net:

  • Early, the net is about a tenth as exploitable as the table. At 1M hands the table has visited only a sliver of its 6.2M rows; one learned function covers spots the table has not banked yet. That is consistent with generalisation, but the share of rows LCFR had visited at 1M hands is not measured.
  • The net has a floor and the table does not. The second 10M hands, 49 hours of training, moved the net from 0.0679 to 0.0685. The table is still falling on its ~1/√N line, so it passes the net somewhere between 10M and 20M hands. That is a measurement, not an extrapolation.
  • Dividing regret targets by the pot moved the floor, not removed it. The same net with raw-chip targets flattened at 0.108; ÷ pot flattens at 0.068 (−37%). Why the divisor is free and what it reweights is the pot-normalised targets.
Deep CFR ÷ LCFR at equal hands
Deep CFR ÷ LCFR at equal hands

Three objects are graded at each mark, and they usually order exact < Π < current

ObjectWhat it isWhy it is graded
exact averageCFR's actual answer: every iteration's strategy, weighted by t and by the player's own reach, summed in a table as training runsthe strategy CFR's theory promises converges
Π, the policy neta fresh net fit at the mark on the stored strategy samplesthe only average a game without a table can keep — full Drawmaha will have to use it
current iterateregret matching over the net's latest regretsCFR promises nothing about it; graded to show how far it sits from the average

The exact average beats Π at every graded mark of every linear arm, so Π's fit costs exploitability (at 20M hands, 0.103 against 0.068). The current iterate is usually worst but not always; none of those exceptions is a bug, because CFR's guarantee is about the average only.

Next

  • Release the 0.068 object for /solver, Learn and Play — the best graded Deep CFR strategy is sitting frozen under runs/ unpublished.
  • Move the floor. A bigger net, a cross-entropy Π, ÷ √pot at the full learning rate, a second seed — the untried arms are listed in open questions.
  • Full Drawmaha. The engine, the encoder and an evaluation that works without an exact grader: a head-to-head checkpoint ladder plus a trained exploiter. /solver already runs on the full game with an illustrative action mix, built ahead of the network. See full Drawmaha.
  • A PPO chapter: train a PPO agent by self-play and measure what the CFR solution wins against it.
  • A test job in CI. The only workflow is an automated code review, so the ~1,600-test suite is run by hand.

Research · Where it stands

The Deep CFR run log, digested

Every Deep CFR arm trained on mini-drawmaha, its settings, its grades, and the nine findings they add up to.

The lab notebook for Deep CFR is docs/notes/rung4-run-log.md: every arm, where it ran, with which net and refit, what the exact grader said. This page is a reader's digest of it. When the two disagree, the notebook wins — and the notebook itself loses to the files under figures/rung4/, which tests/deepcfr/test_rung4_quotes.py checks it against.

The settings every arm shares

All arms are seed 0, K = 10,000 walks per seat per iteration, 50M-row reservoirs and 10,000-row minibatches. So iteration t has seen 10,000·t hands per seat, exactly what LCFR saw at lockstep iteration 1,000·t, and the two can be compared at equal hands.

The rows a run collects never depend on the worker count W: every walk is drawn from (seed, t, seat, walk index). A Mac run and a Modal run of the same config therefore see identical data, and only the refit's floating-point order can differ. That is what lets arms trained on different machines sit in one table.

The scoreboard: the exact average at each mark

Net is depth × width; "÷ pot^k" is how the regret targets were scaled before the refit (0 = raw chips). Blank means not graded.

armnetrefit stepslr÷ pot^k100k200k500k1M2M5M10M20M
LCFR (the table)————3.0482.6971.8761.1310.6300.2640.1260.061
3 × 64, Mac3 × 644,0001e-300.5810.4440.396
sweep 3 × 64, Modal3 × 644,0001e-300.382
sweep 4 × 64, Modal4 × 644,0001e-300.375
sweep 3 × 128, Modal3 × 1284,0001e-300.243
3 × 128 to t = 2003 × 1284,0001e-300.234
3 × 128, double fit3 × 1288,0001e-300.2130.184
4 × 1284 × 1284,0001e-300.2200.221
4 × 2564 × 2564,0001e-300.2940.1730.152
4 × 256, double fit4 × 2568,0001e-300.3300.1670.134
4 × 256, 4× fit, half rate4 × 25616,0005e-400.6430.3300.1690.1330.1150.108
4 × 256, 4× fit, full rate4 × 25616,0001e-300.8500.1270.104
… half rate, ÷ √pot4 × 25616,0005e-4½0.8790.1080.086
… half rate, ÷ pot, to t = 2,0004 × 25616,0005e-410.1140.0840.0710.0680.068
DCFR screen4 × 2564,0001e-301.5152.2272.870
CFR+ screen4 × 2564,0001e-302.5392.8583.092

Each arm's file under figures/rung4/ also holds Π, the current iterate, each seat's best response, the training clock and the full config. The JSON names spell nets width × depth: sweep_256x4 is four hidden layers of 256.

The nine findings

1. Width was the lever that mattered

At 1M hands, going from 3 × 64 to 3 × 128 cut the exact average from 0.382 to 0.243 (−36%). An extra layer did far less: −2% at 64 wide, −9% at 128 wide. The sweep's rule was fixed before its numbers existed — stay at 3 × 64 unless another size is lower by more than 10% — and it picked 3 × 128. Later arms on the Mac went wider still, to 4 × 256 (0.152).

2. A longer refit helps, by less than a wider net

Doubling the refit from 4,000 to 8,000 Adam steps took 3 × 128 from 0.243 to 0.184 (−24%) and 4 × 256 from 0.152 to 0.134 (−12%). The gain shrinks as the net grows — what you would expect if the small net was under-fit and the large one is near its fit's floor. Early on a longer fit can be worse (4 × 256 at 200k hands: 0.330 against 0.294), so arms are compared at 500k hands or later.

3. The net is far ahead early, then flattens

At 1M hands the best raw arm is 0.133 against the table's 1.131 — an eighth. But from 1M to 5M hands the 16,000-step arm moved 0.133 → 0.115 → 0.108 (−19% over five times the hands) while LCFR fell 1.131 → 0.630 → 0.264 (−77%). That arm was stopped at t = 506; its two averages at t = 500 are the release rung4-deep-t500 that /solver shows. Finding 8 measures where the curves cross.

4. The three graded objects usually order exact < Π < current

The exact average beats Π at every graded mark of every linear arm. The current iterate is usually worst but not always — the 16,000-step arm's current iterate beats both averages at 100k hands. CFR promises convergence of the average only, so none of these orderings is a bug.

5. Per-sample DCFR and CFR+ lose to linear by 17×, and get worse with training

At 500k hands the two screens grade 2.870 and 3.092 against linear's 0.173, and both rise with t. A sampled trainer has no running regret total whose sign the rules could test, so they test each sample's sign instead, and a bad action's positive samples outvote its negative regret. The mechanism, at both rungs, is written up in docs/notes/regret-rules-dcfr-cfrplus.md and on regret rules. Linear is the only rule worth running further.

6. The refit is the iteration

The two refits are about 95% of an iteration's time. Moving them to Apple's GPU (--device mps, PR #61) cut a 4 × 256, 8,000-step refit from 127 s to 34 s per seat. The 16,000-step arms cost about 70 s per seat per refit, about 152 s an iteration.

7. ÷ pot targets: −25% at 2M hands

Three 16,000-step arms differ only in the divisor k:

targets1M2M
raw chips (k = 0)0.1330.115
÷ √pot (k = ½)0.1080.086
÷ pot (k = 1)0.1140.084

What problem it solves. One net fits every spot's regrets with one accuracy budget, and the squared-error loss decides where that budget goes. A regret is in chips, so its size scales with the pot; squared error therefore weighs a spot by how often it is reached × pot². The 52-chip spots count 676 times as much as the 2-chip ante spots reached equally often, and the ante spots — where every hand begins — are fitted last.

Why dividing is free. The divisor is the pot at the decision, shared by every sample at that spot, so it scales the spot's whole regret row by one constant. Regret matching reads only the row's signs and ratios, so it plays the same strategy from the scaled regrets as from the raw ones. What changes is only where the net's errors land.

What k moves. Dividing by pot^k leaves a weight of how often × pot^(2 − 2k). k = ½ weighs a spot by the chips a mistake there costs; k = 1 weighs every spot alike. ÷ √pot is the default for new runs since 2026-10-27.

The caveats. One seed each, and the only noise estimate is two raw runs 0.005 apart; the pot arms' gain is five to six times that. The two weightings are tied at 2M hands. The targets are not otherwise normalised, so dividing also shrinks the gradients the clip of 1.0 sees — part of the gain may be a changed clipping regime rather than the reweighting.

8. ÷ pot to 20M: the table passes the net

hands per seat1M2M5M10M20M
exact average0.1140.0840.0710.0680.068
Π0.1520.1150.0980.1140.103
current iterate0.1970.2140.1750.1630.191
LCFR1.1310.6300.2640.1260.061
training clock, hours6.210.622.551.7100.7

To four figures the last two exact grades are 0.0679 and 0.0685: the second 10M hands bought nothing. The plateau of finding 3 is real for this weighting too — 37% lower than the raw arm's, not removed. Π does not track the exact average at these marks.

The arms that did not count

  • The Modal main run. A 3 × 64 run with target 500 was started on Modal on 2026-10-23 and stopped at t = 64, six minutes after the sweep picked 3 × 128. Its marks are kept on the volume drawmaha-rung4 and ungraded; they are only worth grading as a Modal-vs-Mac reproducibility check.
  • The 16,000-step arm's stop rule. Before its marks were graded, the rule said: go past 1M hands only if the exact average there is below 0.121 (10% under the double-fit arm's 0.134). It came in at 0.133, so by the rule the longer fit bought nothing; it was run on anyway to see the curve's shape, which is what finding 3 reads.

How an arm is run and graded

A Mac arm, as the ÷ √pot one would be launched today (÷ √pot is now the default):

uv run --extra deep python scripts/deepcfr/run_deep_cfr.py --out runs/pot0.5-256x4-s16000-lr5e-4 \
    --target 200 --walks 10000 --workers 16 --width 256 --depth 4 --steps 16000 \
    --lr 5e-4 --device mps --save-every-s 600

Each mark is graded as it lands, locally and without Modal spend, through the Modal app's grade entry point:

uv run --extra deep --with modal modal run scripts/deepcfr/modal_rung4.py::grade \
    --local runs/<arm> --marks 200 --grades figures/rung4/<file>.json

A grade takes several minutes and competes with a live trainer for cores, so grade one mark at a time. Add the arm's row to the notebook when it is graded. Running on Modal itself is covered in Modal.

9. Distilling the table into the net splits the plateau into capacity and training

The LCFR table's average at iteration 10,000,000 (graded 0.0157) fitted straight into the rung-4 net by reach-weighted cross-entropy grades 0.0389 at 4 × 256 and 0.0175 at 4 × 512, against Deep CFR's 0.0685. So at Deep CFR's size at most about a third of the plateau is what the net cannot hold (a 400,000-step copy grades 0.0331, 33%; the 100,000-step one 44%) and the rest is what its training did not find, and one doubling of the width removes almost all of the first part. Distilling the table into the net has the method, the ablations and the caveats.

Research · Where it stands

Distilling the table into the net

The LCFR table copied straight into the rung-4 net by reach-weighted cross-entropy, so the 0.068 plateau splits into what the net cannot hold — at most a third at 4 × 256 — and what Deep CFR's training did not find.

Deep CFR's ÷ pot arm, a 4 × 256 net, flattens at an exact-average exploitability of 0.0685 from 10M hands per seat while the tabular LCFR run keeps falling, to 0.0157 at iteration 10,000,000 (100M hands per seat). Two very different things could hold the net there: it cannot represent a better strategy, or Deep CFR's training (sampled regrets, reservoir memories, from-scratch refits, a learned average) never finds one it could. No Deep CFR arm can tell them apart, because every arm trains the net the same way. This page is the experiment that does, run on 2026-11-05; the numbers are figures/rung4/distill/results.json and the full write-up is finding 9 of the run log.

The method: fit the net to the table, then grade the copy

Take Deep CFR out of the picture. The teacher is the table's linear average strategy at iteration 10,000,000, one distribution over the legal actions at every one of the 6,220,050 infosets. The student is the rung-4 net itself (deepcfr/net.py: 136 features in, seven outputs, ReLU layers of a given width and depth). deepcfr/distill.py fits the student to the teacher by supervised learning over every infoset and freezes the result as a strategy the exact grader reads. If the copy grades near the teacher, the net can hold such a strategy and the plateau is a training effect; if it grades near the plateau, capacity is the story.

Three choices make the fit mean something:

  • The target is the average strategy, not the regrets. The average is what CFR's guarantee is about and what the Deep CFR run was graded on. Copying regrets would copy the current iterate (0.1911 for the same run), a different and worse object.
  • The loss is cross-entropy on a softmax over the legal columns. Illegal columns are held at −10⁹ before the softmax, so they carry no probability and get no gradient. The loss is smallest where the softmax is the teacher's row, which a net reaches only where it can represent that row (an exact zero in the teacher is approached, never reached). The Π loss sweep had already shown cross-entropy copies a distribution better than squared error.
  • Each row weighs what arrives at it. A best response exploits a spot in proportion to how often the player's own choices and the deck bring the player there, and a spot the player's own play never reaches cannot be exploited at all. So a row's weight is its seat's own reach under the teacher (the same π^q(I) the exact average accumulates with) times the number of physical deals its canonical key stands for (6 over the size of the key's suit stabiliser). Rows are drawn in proportion to that weight, which minimises the weighted objective at far lower variance than weighting uniform draws.

There is no held-out split for the headline: the exact grader scores every infoset, so the whole-game grade is the test and memorising is the goal.

The result: at most a third is capacity at 4 × 256, the rest training, and the capacity part goes away one doubling later

Every arm in the table is 100,000 Adam steps of 16,384 rows under a cosine schedule from 1e-3, one seed, on this Mac's GPU; the longer fits are under the ablations. err is the reach-weighted mean absolute error per column against the teacher, measured on the frozen file.

armnetdiffers byerrexploitability× teacher
teacherthe LCFR table—00.01571.0×
3x643 × 64size0.11470.184211.7×
3x1283 × 128size0.09040.10156.5×
4x2564 × 256the Deep CFR net0.04500.03892.5×
4x5124 × 512size0.00610.01751.1×
4x10244 × 1024size0.00100.01581.0×
Deep CFR4 × 256trained by Deep CFR, 20M hands—0.06854.4×
Each copy of the table against its parameter count
Each copy of the table against its parameter count

The three gaps at 4 × 256. After 100,000 steps the copy grades 0.0389 against the teacher's 0.0157 and Deep CFR's 0.0685, which puts what the net cannot hold at +0.0233 (44% of the plateau) and what its training did not find at +0.0295 (56%). The copy's grade only bounds what the net can hold from above, and the 400,000-step copy grades 0.0331, so the first part is at most 33% (+0.0174) and the second at least 67%; the figure draws that tighter split. Neither part is negligible: a wider net trained the same way would still carry the training gap, and a better training procedure with this net could get no lower than what the net can hold, which these fits place at or below 0.0331. The two shares are a reading of grades along one axis, not a decomposition of causes: exploitability is not additive in a strategy's errors.

At most a third of the 4 × 256 plateau is what the net cannot hold
At most a third of the 4 × 256 plateau is what the net cannot hold

One doubling of the width removes almost all of the representation part. The 4 × 512 copy grades 0.0175, within 0.0019 of the teacher, and its error against the teacher falls from 0.0450 to 0.0061. At 4 × 1024 the copy grades 0.0158, the teacher to four places. Every Deep CFR size that was ever run on this game (3 × 64 to 4 × 256) sits on the steep part of this curve.

What the ablations add

Loss and reach-weighted error by step
Loss and reach-weighted error by step
  • The 4 × 256 floor is partly the fit's. 25,000 steps grade 0.0560 against 100,000's 0.0389, the headline's error curve was still easing down when the schedule ran out, and 400,000 steps grade 0.0331: the representation share of the plateau moves from 44% to 33% with a four-times longer fit, while the 4 × 512 copy got further in a quarter of the steps. The copy's grade is an upper bound on the net's floor for this reason and one more: cross-entropy copies the teacher's probabilities, it does not minimise exploitability, and another near-equilibrium may be easier to represent.
  • The reach weight is worth 2.6× in the grade. Rows drawn uniformly over the 6.2M infosets, everything else equal, grade 0.0995 against 0.0389. The uniform draw gives every infoset the same share of the steps whatever its weight, so the rows the strategy's own play reaches most are fitted no better than rows it never reaches, and the best response finds them; which rows absorbed the lost steps was not measured.
  • Trained on a random half of the game, this net does not transfer to the other half. A copy trained on a random half of the rows grades 0.1046: its error on the rows it saw is 0.0228, better than the full copy's 0.0450, so it is not short of steps there, but on the rows it never saw 0.1185, and the best response goes there. A tenth grades 0.3457. What this does not separate is why: the 136 features, the architecture, or the teacher itself — an average strategy that is irregular across neighbouring infosets, as an average built from few samples at rarely visited rows may be, would defeat any model that infers. For full Drawmaha, which cannot be enumerated and where every spot is in effect unseen, transfer is the number that matters, and on one seed and one split this net on these features does not have it.
A copy trained on a share of the infosets
A copy trained on a share of the infosets

What it means for the next arm

A wider net trained by Deep CFR the same way is capped near its own copy, so the larger and more durable gain is in the training: the sampling, the memories and the average. A 4 × 512 copy at 0.0175 is the floor that training would have to approach. Caveats: one seed of everything; the teacher's own error (0.0157) is in every copy; the weight is own reach under the teacher, not under a best response.

Research · Where it stands

Open questions

The Deep CFR arms nobody has run, the decisions full Drawmaha forces, and the gaps the literature audit found.

Two lists feed this page. The first is the run log's "Not yet run" — experiments on mini-drawmaha whose answer would change a default. The second is docs/research/09-gaps-audit.md, written when the literature survey was assembled, which lists nineteen things the survey had to cover or scope out. Many of the audit's items are now engineering decisions for full Drawmaha rather than gaps in a paper; they are sorted that way below.

On mini-drawmaha: what would move the 0.068 floor

The best Deep CFR net flattens at 0.068 chips/hand from 10M hands, and the table passes it by 20M (where the project stands). Every comparison so far is one seed, so a difference under about 10% is suggestive, not settled. The untried arms, each with the question it answers:

ArmThe question it answers
A second seed of any armHow big run-to-run noise really is. The only estimate today is two raw runs 0.005 apart that also differ in learning rate.
A bigger net, or a cross-entropy Π, to t = 2,000 under ÷ potIs 0.068 a capacity floor (a bigger net moves it) or a fitting floor (a better objective moves it)?
÷ √pot at learning rate 1e-3The two single-seed gains (√pot targets, the full rate) have not been tried together — and it is the pairing the default config now runs, so the default is an ungraded config.
A raw-target arm with a looser gradient clipSeparates the pot weighting from the clipping regime it changes: dividing targets shrinks the gradients, so the clip of 1.0 bites less.
The √pot arm past t = 200Does it flatten where ÷ pot did (0.068) or elsewhere?
The tᵞ sample-weight arm (γ = 0 vanilla, γ = 2 quadratic)Linear is γ = 1; is it the best weighting for a sampled, refit-from-scratch learner?
A Modal run of the ÷ pot configA reproducibility check of the Mac run, not a new arm — rows do not depend on the machine, only the refit's arithmetic order does.

On full Drawmaha: decisions the game forces before training

These come from the audit and are open because no code for the full game exists yet.

  • How the net outputs a discard. A five-card hand has 32 keep/discard subsets. A flat 32-way head, per-card keep/discard logits, or something factorised — and how regret matching behaves over a combinatorial action space, with masking for illegal actions — is an encoding decision with no citation anywhere in the survey (audit item 8).
  • How the non-card state is encoded. Betting history, pot and stack normalisation, position, and both players' public draw counts all have to enter the net's input. Mini-drawmaha's 136-feature encoder is the starting point, not the answer (item 9).
  • How hands are indexed. Suit isomorphism collapses 2,598,960 five-card holdings to 134,459 classes before the deal, but the factor shrinks as board and kept cards fix suits. The standard perfect-indexing algorithm (Waugh 2013) is not yet used (item 7).
  • The exact rule variant. Drawmaha has sub-variants (draw timing, number of betting rounds, odd-chip rules). The project pins one — see full Drawmaha — and every node count and street-boundary claim depends on that choice (item 3).
  • Units and diagnostics. Hold'em calibration anchors (mbb/g, the always-fold baseline) do not transfer to a split-pot bomb pot. The pre-registered protocol reports total chips across both halves, per-half EV and scoop rate, and the terminal-utility table for odd chips and double ties still has to be written (item 16).
  • What is served. Between "trained average-policy net" and "strategy shown or played" sits purification and thresholding — rounding near-zero probabilities before serving — which the project does not do today (item 12).

Techniques the project has not used, and would consider

TechniqueWhy it could matter hereAudit item
Regret-based pruning (Brown & Sandholm 2015, 2017)skips negative-regret subtrees; a major speedup in the Libratus and Pluribus blueprints4
Predictive / optimistic CFR+ (Farina, Kroer & Sandholm 2021)the tabular state of the art past DCFR, with faster rates5
Public chance sampling / vector-form CFR (Johanson et al. 2012)walk the public tree vectorised over private hands — the trick behind commercial-solver speed, and the shape the exact grader already has6
Warm-starting CFR from a strategy (Brown & Sandholm 2016)start full-game training from a shrunken variant's solution13
Value nets and depth-limited resolving (DeepStack, Libratus)the search phase after a blueprint; needs a way to generate counterfactual-value data and to update ranges through a hidden discard10, 11
NeuRD, ARMAC, QPG/RPGthe bridges between policy gradients and CFR; relevant to the planned PPO chapter17

Scope statements, decided or to be decided

  • Heads-up only. Every guarantee the project relies on — any Nash strategy is safe, the game has one value — is a two-player zero-sum fact. Drawmaha is usually dealt multiway in mixed games, where none of that holds (audit item 18). The project solves heads-up.
  • Equilibrium, not exploitation. The solver plays Nash, which is unbeatable but declines to punish mistakes; Kuhn's exploit tab measures exactly how much that leaves on the table. Opponent modelling and safe exploitation (Restricted Nash Response) are out of scope (item 14).
  • Benchmarking a first-of-its-kind agent. No outside Drawmaha bot exists, so head-to-head baselines must be homegrown: a duplicate-dealt ladder between independently seeded checkpoints plus a trained exploiter, reported as a lower bound on exploitability (item 19). Real-money play against people on poker sites is a terms-of-service violation and not a goal.
  • Rake. Whether the solver targets raked or rake-free play is not stated anywhere; today's games are rake-free.

Research · Literature

Literature

The nine research notes the ML survey was written from — the question each answers, its key papers, and what the solver took from it.

Before any solver code existed, a literature sweep (2026-08-28) split the field into eight lanes, each a question a Drawmaha solver has to answer, and a ninth note audited what the eight left out. The notes live under docs/research/ and are finished reference: they are not updated as the code moves. The 26-page survey written from them is paper/drawmaha-ml-survey.pdf.

Read a lane when the code you are about to touch rests on its idea. Each note opens with a lane narrative — the story, the builder's options as a decision tree, the pitfalls — then one section per paper.

The nine lanes

NoteThe question it answersKey papersWhat the project took from it
01 · CFR foundationsWhy does iterating a local update rule produce a Nash equilibrium in an adversarial game?Hart & Mas-Colell 2000 (regret matching); Zinkevich et al. 2007 (CFR); Lanctot et al. 2009 (MCCFR); CFR+ 2014; Cepheus 2015The whole tabular stack: the shared regret-matching ledger, vanilla CFR on Kuhn and Leduc, external-sampling MCCFR on mini-drawmaha, and the four-rule race. The note recommended DCFR; the race found LCFR beats it under sampling.
02 · Neural CFRHow do you replace CFR's regret table with a regression when the game is too big to store?Deep CFR 2019; Single Deep CFR 2019; DREAM 2020; ESCHER 2022; Double Neural CFR; ReCFRRung 4's design: external sampling (the project owns its simulator), reservoir memories, a from-scratch refit each iteration, t-weighted strategy samples, and grading the exact (SD-CFR-style) average beside the distilled Π so distillation error is not blamed on CFR.
03 · Search and value netsCan test-time search be made sound when you do not know which state you are in?Burch et al. 2014 (decomposition); DeepStack 2017; safe and nested subgame solving 2017; Libratus 2018; depth-limited solving 2018; Pluribus 2019Not yet used. It frames the phase after a blueprint — a depth-limited resolve at the draw boundary — and its warning: Drawmaha ranges are 2,598,960-dimensional, so range-vector value nets do not port without bucketing.
04 · RL and equilibriumWhy does naive self-play RL fail at mixed strategies, and what fixes it?NFSP 2016; PSRO 2017; Pipeline PSRO 2020; Perolat et al. 2020 (recurrence); DeepNash / R-NaD 2022; MMD 2022That the last iterate orbits and the time average converges — the same reason the site queries averages, never the final iterate. It is the background for the planned PPO chapter.
05 · Abstraction and bucketingHow do you build a smaller game whose solution transfers back?Gilpin & Sandholm 2007 (lossless abstraction); Waugh et al. 2009 (pathologies); Ganzfried & Sandholm 2013–14 (action translation, EMD imperfect recall)Mini-drawmaha is a hand-designed abstraction that keeps the draw and the split pot; its infoset key relabels suits jointly, the lossless-abstraction idea this lane opens with. See abstraction.
06 · EvaluationHow do you measure a strategy when training loss says nothing?Johanson et al. 2011 (fast best response); local best response 2017; AIVAT 2018; Cepheus; Deep CFR's evaluationThe project's first rule: grade by exploitability, computed by a best-response walk that shares no code with the solver, and at full scale a head-to-head ladder plus a trained exploiter as a lower bound. See exploitability.
07 · Variant gamesWhat do solved draw and Omaha variants teach about Drawmaha's two blow-ups — hand count and the draw round?Ostroumov's Draw Solver; Omaha hi-lo jam/fold 2015; Poker-CNN 2015; GTO Wizard PLO 2026; MonkerSolver docs; Zadeh 1977That the split pot is the cheap part (it changes only terminal payoffs), that discard identity matters for blockers, that infoset keys must hold both players' draw counts — and the load-bearing absence finding: no Drawmaha solver, bot or paper existed as of 2026-08.
08 · Engineering practiceWhat do you actually build with, and where do projects break?Neller & Lanctot 2013 (CFR tutorial); OpenSpiel 2019; PokerRL and the Deep CFR reference code 2019; RLCard 2019OpenSpiel as the Leduc referee (frozen in tests/leduc/referee.json), the Deep CFR hyperparameter anchors (Adam 1e-3, clip 1, batch 10K, 4K steps, 10K traversals), and the validation-ladder habit of checking every piece on a game with a known answer.
09 · Gaps auditWhat did the eight lanes leave out that the survey had to cover or scope out?nineteen items, from Kuhn's theorem and sequence-form LP to discard encoding and multiway playThe open engineering decisions for full Drawmaha, sorted on open questions.

Three findings from the sweep that shaped the plan

  • Average, never last. Lanes 01, 02 and 04 arrive at the same fact from three directions: in a two-player zero-sum game the time-averaged strategy converges to equilibrium while the current one cycles. Kuhn's exploitability table shows it on the smallest game (Kuhn), and it is why every Deep CFR mark grades the average.
  • The grader is the product. Lane 06's one-sentence summary — self-play produces no labels, so the only principled question is what a perfect adversary wins — is why each game in the project was solved against a known answer before the next was attempted.
  • Nobody had done this. Lane 07 searched papers, code, the commercial solver market and forums for any Drawmaha equilibrium work and found none. The game is documented and dealt online; no solver plays it.

Caveats

Each lane ends with its own verification caveats: which papers were fetched in full, which hyperparameters were read from summaries, which venue labels come from background knowledge. One correction is worth knowing before quoting the notes: lane 01 gives the two-player deal space as C(52,5)² ≈ 6.7 × 10¹², but the hands are disjoint, so the right count is C(52,5) × C(47,5) ≈ 4.0 × 10¹² (the audit's own note).

Research · Literature

Full Drawmaha, the target game

The game the project exists to solve, what changes from mini-drawmaha, and how a solve will be graded once no exact grader can run.

Everything so far — four solved games and two solves of mini-drawmaha — exists so that a method can meet full Drawmaha already checked. This page states the target, measures how much bigger it is than anything solved, and lays out what the solve needs that mini-drawmaha did not. Nothing here is built yet except the range interface at /solver, which runs on the full game with an illustrative action mix.

The game: a bomb pot with one draw and a pot that pays twice

Standard Drawmaha, heads-up, as the project pins it (README, "The target game"):

  • Each player is dealt 5 hole cards.
  • It is a bomb pot: both players ante and the flop comes out with no preflop betting. The order is ante → flop (3 cards) → betting → one draw (discard 0–5, replace) → turn → betting → river → betting. Three betting rounds, not four.
  • The draw-1 rule. A player who draws exactly one card gets it dealt face up; they may keep it or reject it and take the next card face down.
  • The split pot. Half goes to the best Omaha hand (exactly 2 hole cards + 3 board cards), half to the best five-card poker hand made of the hole cards alone.

The solve's v1 defaults, all configurable: pot-limit with the action set {fold, check/call, pot}; 100 big blinds effective; both players ante, no blinds. The bet-size menu is a config, so a multi-size solve is a re-solve, not a rewrite. The Rules page walks a hand through both readings.

What changes from mini-drawmaha, and by how much

Mini-drawmaha was designed to keep the two things that make Drawmaha Drawmaha — a draw, and one pot read two ways — on a deck small enough to solve exactly.

Mini-drawmahaFull Drawmaha
Deck15 cards: ranks 2–6, three suits52 cards
Hole cards35
Board2 cards, one per round5 cards: flop, turn, river
Betting rounds23
Drawthrow at most one card, replaced face downthrow 0–5 (32 discard subsets); a one-card draw is dealt face up
The two halvesbest 3 held (inner); best 2 held + both board cards (outer)best 5 held (draw half); best 2 held + 3 board (Omaha half)
Stacks26 chips, ante 1100 big blinds
Holdings a player can be dealt455 (three of fifteen)2,598,960, against 1,326 in hold'em and 270,725 in Omaha
Infosets6,220,050 — storable as a tablefar beyond any table
Exact graderyes: a best-response walk over the whole gameno

Two of those rows end the methods that solved mini-drawmaha. The table cannot be stored, which is why rung 4 existed: Deep CFR replaces the 6.2M ledgers with a network, and the full game has no table to replace. The exact grader cannot run, and that one is harder, because the grader is what every result so far has been trusted on.

Suit isomorphism helps, but less than it seems: the 2,598,960 five-card holdings collapse to 134,459 suit-isomorphic classes before the deal (docs/research/05-abstraction-bucketing.md), and the factor shrinks as the board and the kept cards fix suits.

What carries over

  • The algorithm. External-sampling Deep CFR with linear weighting, ÷ √pot regret targets, reservoir memories and a from-scratch refit — the configuration the run log converged on. The rule that came out of rungs 3 and 4 holds: anything sampled uses linear weighting, and DCFR or CFR+ need a reason to be tried again (regret rules).
  • Π as the answer. Mini-drawmaha could keep an exact average in a table; the full game cannot. The policy net Π — fit at each mark on the stored strategy samples — is the only average it can keep, and on mini-drawmaha Π was always more exploitable than the exact average (0.103 against 0.068 at 20M hands). How good Π can be made is a full-game question that mini-drawmaha can still answer cheaply.
  • The rule that a grader never shares code with what it grades. At full scale the grader is an exploiter rather than a walk, but it must still be written against the rules alone.
  • The range interface. /solver already samples 50,000 five-card holdings for a flop, classifies each twice (the five cards alone, and the best two with the board), and draws the range as a grid of inner rows × outer columns. The composition is real to sampling error; the action mix is a smooth function of each hand's two equities and is labelled illustrative. web/solver-viz/src/engine/policy.ts is the one file that changes when the network exists.

Full Drawmaha pipeline

  1. An engine for the full game: deck, streets, the draw with its face-up rule, both scorers, terminal payoffs including odd chips and double ties.
  2. An encoder. Mini-drawmaha's net reads 136 features; the full game needs a representation of five hole cards, the discard, a five-card board, three rounds of betting, and both players' public draw counts. Which cards were kept matters, not only how many — draw-game solvers found discard blockers change strategies.
  3. A discard head. Thirty-two keep/discard subsets as a flat output, or per-card logits, and how regret matching behaves over them, is an open decision (open questions).
  4. Training at scale. The refit is already about 95% of an iteration on mini-drawmaha, so the full game's cost is mostly fitting a bigger net on more rows.
  5. An evaluation that needs no exact grader — below.

How a full-game solve will be graded: pre-registered before training

Training loss is not a progress meter in self-play; the only honest question is what an adversary wins. The README's protocol, fixed before any full-game code exists:

  • A duplicate-dealt head-to-head checkpoint ladder. Each checkpoint plays the next with the cards dealt twice, seats swapped, so card luck cancels and what remains is skill. It shows progress, not distance from equilibrium.
  • A trained RL exploiter against the frozen net, reported as a lower bound on exploitability. A best response found by training is never better than the true one, so whatever it wins, the true exploitability is at least that.
  • Units: total chips across both pot halves, per hand, in big blinds, with per-half EV and scoop rate as split-pot diagnostics.

On mini-drawmaha both ends of this can be checked against the exact grader first — how close a trained exploiter comes to the true best response there is a measurement that can be made today.

Plans

Full Drawmaha has no design document yet. The rung-4 spec, docs/superpowers/specs/2026-10-07-deep-cfr-m1-design.md, lists the full game's engine and encoder as out of scope for its milestone, and its PR table is the live record for current work. The range interface was planned in docs/superpowers/plans/2026-10-07-solver-range-interface.md; the ten design studies it was placed from are under docs/research/range-ui-studies/. Per the docs convention, a change of direction gets a new dated spec — the full-game solve will start with one.

Changelog · Updates

Changelog

What landed on main, day by day, newest first — read from the merge history of the repository.

Nearly every change reaches main through a pull request, and its subject states the finding. This page groups them by the day they merged and says in plain words what each day changed. Numbers in brackets are the pull requests on GitHub; the full history is git log origin/main.

November 3, 2026

Deep CFR — the ÷ pot net, run to the race's endpoint, flattens at 0.068

  • The 2,000-iteration run was made, on a Mac. The 4 × 256 net with regret targets ÷ pot was run on from 2M to 20M hands per seat — 100.7 hours of training — and graded at 5M, 10M and 20M hands (#86).
  • The net has a floor and the table passes it. The exact average went 0.071 → 0.068 → 0.068; the second 10M hands bought nothing (0.0679 → 0.0685). LCFR is 0.126 at 10M and 0.061 at 20M, so the table passes the net between the two.
  • The homepage, Benchmarks and /research's Figures 8, 9 and 13 now draw this arm against LCFR. Its best object is graded but not released.

November 2, 2026

Site — the card's rank and pip, settled

  • The rank is 0.8 of the card's width on a 15 px floor, the oversized suit pip tucked into the corner so the rank clears it, and a ten is T on the static pages as in the apps (#85).

October 30, 2026

Site — one playing card everywhere: the bleed pip

  • The Play tab's card first (#82): a small rank and one suit glyph too big for the card, cropped by its own edge, in four inks.
  • Then every card on the site (#83): one drawing in the theme, each surface setting only its width.
  • The rank grew from 0.36 to 0.44 of the card's width with a 12 px floor, because it read small beside the pip (#84).

October 29, 2026

Site — the site on paper

  • One theme for every surface (#81): warm paper, near-black ink and one oxblood accent; Newsreader for display and research prose, Geist for the UI and every number. A product nav — Research, Solver, Benchmarks, Play, Rules — and declarative copy. The turning ASCII chip stays as the mark. Argued in docs/superpowers/specs/2026-10-29-paper-lab-site-design.md and enforced by web/DESIGN.md.

October 28, 2026

Site — /rung3 becomes a research publication

  • A masthead and nine chapters (#67), each heading stating its finding and each figure captioned with the file it reads; Deep CFR sits beside the tabular race, with the reach decomposition and a chapter on the √pot regret targets.
  • The contents rail boxes the chapter you are reading and follows the scroll (#72), and names what each chapter is about in one line (#76).
  • Figures re-drawn to read along one axis: the reach figures as columns with the leak per visit as a rising line (#74), chapter 05's equal-hands grid as a figure of two clocks on one ruler of hands (#79), and Figure 8's axes as a named domain a test holds every mark inside (#80).

Play — the trainer deals against the best released Deep CFR

  • The Play tab's opponent is now rung 4's ÷ √pot exact average at 2M hands (#69), released as rung4-deep-pot0.5-t200, its grade (0.086) quoted from the deal pack's manifest.
  • The trainer reads in the Learn tab's register: lanes in a well, Learn's cards, the coach's mix as bands (#77).

Solver — the hand so far is a path

  • The steps of the hand are a path with the next decision forking off its tip, and the range grid's largest region fills its cell (#70).

Repo — the README opens on the live site

  • The front page opens on the live site and one results table; the top-level research/ and writeups/ folders fold into docs/ (#73).
  • read_grade refuses a malformed grade entry by name (#75); the chip mark's docstring and its tests were tightened (#71).

October 26, 2026

Solver — Deep CFR beside the table

  • /solver shows three strategies behind one switch (#65): rung 3's tabular LCFR and rung 4's Π and exact average at 5M hands (rung4-deep-t500), each read from a release asset with its grade in the manifest.
  • /rung3 gets a Learn tab (#66): the mini-drawmaha range interface, imported from solver-viz rather than copied.

October 25, 2026

Deep CFR — the regret-rule screens and the sweep arms

  • DCFR and CFR+ lose to linear by 17× under per-sample regrets, and get worse with training (#62).
  • The sweep arms are graded (#63), the trainer gains --lr and --save-every-s, and two living notes start: the run log and why DCFR and CFR+ lost twice.
  • Every grade quoted in the docs is now checked against its file by a test; 3 × 128's sweep grade is 0.243, not 0.244 (#64).

October 24, 2026

Deep CFR — the refit on Apple's GPU

  • --device mps makes the refit 3.7× faster per seat, and the device can change on resume (#61).

October 23, 2026

Deep CFR — the Modal launcher and the price figure

  • One resumable trainer, a grader per grid mark for the exact average, Π and the current iterate, plus the smoke run and the net-size sweep (#60).
  • The price figure: Deep CFR's exploitability ÷ LCFR's, at equal hands per seat. The sweep picks 3 × 128 over 3 × 64.

October 21, 2026

Deep CFR — the average strategy

  • The exact running average: own reach pushed through a predecessor table into a t-weighted sum, and Π fit on the policy memory, both frozen for the grader (#59).

Repo — instructions at every subtree

  • An AGENTS.md in every subtree, scripts grouped by what they serve, and mini-drawmaha split into rules/, solver/ and answer/ (#58).

October 20, 2026

Deep CFR — the loop

  • One iteration in the spec's order, an exact resume, grid marks, and the oracle pin: with a perfect "net" the loop is LCFR bit for bit (#57).
  • The reservoir (Algorithm R, exact resume) and the collector: K walks on W spawned workers, the same rows for any W (#56).

October 19, 2026

Deep CFR — the net, materialise and the refit

  • The 3 × 64 net, materialising its outputs into the table's regret slots (Deep CFR's eq. 3 obtained by what is written), and the from-scratch refit (#55).

October 9, 2026

Deep CFR — the encoder

  • 136 features per mini-drawmaha infoset, and torch as an optional extra so the default suite still needs none (#52).
  • The /rung3 deal pack is checked against the grammar as it loads (#49).

October 8, 2026

Solver — /solver plays mini-drawmaha

  • Reach-weighted ranges at every decision over the frozen LCFR strategy, exact to the export's 1/250 rounding (#54).

October 7, 2026

Mini-drawmaha — the race and the answer

  • Lockstep MCCFR on W workers, the Modal launcher, and the four-rule race to 2M iterations (#45): LCFR wins at 0.061 chips/hand.
  • The frozen LCFR strategy as a release asset, with minidraw-play and minidraw-analysis (#46).

Site — /rung3 and the UI standard

  • /rung3: the research page and a browser trainer for mini-drawmaha over a pre-dealt pack (#48).
  • Every pressable answers a press, chrome motion settles under 300 ms, and the UI standard becomes a document with a test (#47). The chip mark is redrawn to read as a chip (#51), and the headline's red squiggle grows with it (#50).

October 1, 2026

  • The rung-0 page's end label says whose numbers they are, and a re-export cannot silently revert hand edits (#44).

September 30, 2026

Mini-drawmaha — the exact grader and the packed table

  • The exact grader (#36): a vector best response along the public tree, and the first checkpoint's number.
  • The infoset key keeps the board in the order it came (#39), because the first round was played against the first card alone.
  • The packed table (#40): four flat arrays with a ledger window per key, extra averaging rows and a discount stamp; about a microsecond per window.
  • The four regret rules and the averaging columns (#41), each gated on Leduc against the LP value.

Site — the pages share one theme

  • The rung pages and the cover share the homepage's theme (#38); on the rung-0 page the settling dot rolls down a bowl and reaches Nash sooner (#42, #43).

September 29, 2026

Mini-drawmaha — the sampled learner

  • External-sampling MCCFR, graded on Leduc first (#33).
  • The ledger table: 3.1M spots, 1.77 GB, and a readout for one hand (#32). (The ordered-board key of #39 later doubled the rows to 6.2M.)

Site

  • The cover becomes two static pages with a rules tab at /rules (#37), and a share page for the hero studies at /hero (#34).

September 24, 2026

  • The deck gate for mini-drawmaha: 1.57M infosets per player, inside the memory budget (#31).

September 18, 2026

  • The mini-drawmaha game environment: the deck, both scorers, the rules (#30).

September 15–17, 2026

Leduc — the /rung2 dashboard

  • The action-timeline dashboard over a committed solve (#24), and the likely-holding bar (#25).
  • Play the solver heads-up on an oval table (#27); unreachable lines fall back to the deal (#26).
  • Edit any frequency and read the response over its Nash baseline (#28).
  • The decision coach (#29): a roll per decision, graded against the solver's mix.

September 7–10, 2026

Leduc — the second poker game, refereed by OpenSpiel

  • The Leduc game environment (#18), then the solver: ledger table, CFR walk, exploitability (#22).
  • The referee's numbers are pinned — OpenSpiel's node census, payoffs and LP game value, frozen in tests/leduc/referee.json — then the analysis and play CLIs (#23).

September 4–5, 2026

Deploy — the visualizers actually served

  • "framework": null pinned in vercel.json (#15): without it Vercel detects Python and sends every path to a serverless function, so /rung0 and /rung1 never served. A test now fails the suite if the pin ever leaves (#16).
  • The site is redesigned as a duotone with the games as its nav (#17).

August 30 – September 2, 2026

Kuhn — the first poker game

  • The game environment (#8): frozen states, infosets that never carry the opponent's card.
  • The infoset table, and the ledger gains its two CFR weights, keyword-only on purpose (#9).
  • Vanilla CFR (#10), the exact exploitability meter over all 64 pure strategies (#11), the analysis pipeline, three figures and a play CLI (#12), and the /rung1 visualizer (#13).
  • The exploit tab (#14): lock a strategy and let CFR find the best response to it, served by a local endpoint.

August 30, 2026

Rock-paper-scissors — the first live demo

  • The interactive regret-matching visualizer at /rung0 (Vite + React + TypeScript), its layout stabilised so nothing shifts while the simulation runs (#3, #6).
  • Vercel's build moves into a script to fit the 256-character buildCommand limit; fonts are self-hosted instead of loaded from a CDN.
  • An automated Claude code review on every pull request — the repository's only CI workflow (#4).
  • Figure scripts and chart UI cleaned up to the house style (#7).

August 29, 2026

Rock-paper-scissors — regret matching

  • The shared regret-matching ledger, the RPS framework, the analysis and a play CLI (#1). The ledger measures regret against the strategy's expected utility — the same counterfactual form CFR uses at every infoset.
  • The research corpus: an eight-lane literature sweep, a gaps audit, and the 26-page ML survey.

August 28, 2026

The repository

  • Scaffolded: a README with the game's rules and the plan to build up to it through games with known answers.