The study / The minds study: the plan

Can a small model learn your next move? The plan, written before the data

In Beat the AI, the village game's duels, the other side shows its guess of your next move before you tap. If players choose to share their duel rounds, we will retrain a small model on them every night and check, on days it has not seen, whether it predicts their next move better than the same model frozen after a burn-in of at least a week, and better than a plain count of each player's habits. This page is the plan, fixed before anyone's rounds were recorded.

Pre-registered on 26 September 2026. Nothing on this page comes from people: no player's duel had been recorded when it was written, and the only numbers here are from bots, labelled so. The plan of record is docs/MINDS-STUDY.md in the project's repository, which is private for now and published with the study's code; this page says the same in plain language. After today the plan is never edited to fit results: any change is a dated amendment, listed at the end of this page.

The question

Does a small next-move predictor, retrained every night on the duels of players who chose to share them, predict those players' next move better than the same recipe frozen after a burn-in period, and better than a strong per-player habit model? "Better" means a lower average log-loss (how surprised the model is by the move actually played) on the very same rounds, on days the models had not seen.

• The model: a small multinomial logistic regression over 63 features of the round (both sides' hearts and power, the round, the moves so far in the duel, the opponent, what that opponent has counted about the player in earlier duels, the player's gear and cards, and what the screen showed). It runs on the game's server; its recipe is fixed in the plan.
• The habit model it must also beat: for each player, how often each move followed their last one or two moves and the opponent's, backing off to everyone's averages when a player has few rounds.
• The frozen model: the same recipe, trained once when the nightly retraining is switched on, never again.

What you would see

Sharing is off unless you say yes. After a duel with a rival, the result screen asks once, with "Not now" as large as "Yes"; the Rival trail has a switch for later. The words you would agree to:

Players who do not share play exactly the game they always had. Your tutorial duels (the tutorial Slime and your initial duel with Pip) and the daily boss are never part of the comparison: they always show the guess.

The three groups

Each shared duel (apart from those above) is put in one of three groups by a keyed hash of the duel's id, so nobody chooses. Rewards are the same in all three, and the opponent plays the same way in all three: its own move reads you with the game's built-in Reader, which counts your moves in this duel and what this opponent remembers of you, for the whole study.

During the first week, and until the nightly retraining is switched on, both open groups show the built-in Reader's guess, as every other duel does. The poker group is the clean measurement: what you see before moving there does not depend on any model being tested, so your move cannot be a reaction to it.

What is measured

• Main measure: log-loss on poker-group rounds in the scored window, paired: every model predicts every round. Also reported: the Brier score, how often the top guess is right, and calibration (do 60% guesses come true about 60% of the time).
• Which rounds count: poker-group rounds with no card played, nothing extra revealed or shown, not in a tutorial duel, not a Tengu's "silent" round, against one of the 14 opponents the plan fixes, from an eligible player, in a duel that gave rounds at all (not, for example, one where the opponent's memory of you included rounds that were not shared: see Left out).
• A player is a random id, not a person: the game cannot tell people apart. Someone with two heroes, or who turns sharing off and on again (a new id), counts as more than one player; how many did is reported.
• Players are split for good: by a hash of their random id, 20% are test players the model never trains on, 10% tune the habit model, 70% train. Each night's model is scored only on the next day.

What counts as a yes

Each interval is a 95% interval from resampling whole players 2,000 times. G1, G2 and G5 decide the answer together (the claim needs all three); G3 and G4 are fixed in advance and reported with it; everything else reported is exploratory. Nobody looks at the gates before the window ends: only counts of rounds and players are watched.

• G1, G2 and G5 pass: we may say that, retrained nightly on the duels players shared, the model predicted their next move better than the same model frozen, with the size of the effect and its interval.
• G1 passes, G2 fails: "the model learns, but not better than a count of each player's habits", said as prominently as a pass. We expect this may happen: on bots with fixed habits, the habit model beat ours clearly in every run.
• G5 fails, whatever G1 and G2 say: the learning claim is not made. We report that the effect in the poker group did not carry to the groups that show the model, and give G1 and G2 for the poker group only.
• G1 fails: not supported. The game may say that it retrains, never that the model got better.
• G3 or G4 fails with G1 passing: said in the same sentence as the pass.
• Too few players (the minimum not met, the window unable to start by day 60, or an early stop): the gates are still computed on what was collected and reported as exploratory, with their intervals. The rules above do not apply and no pass may be claimed.

How many players

People are the limit, not rounds: rounds from one person resemble each other. The plan sets the minimum by a formula (enough to detect a 0.02 difference in log-loss, at 5% and 80% power, with each player counted for 20 poker rounds and discounted for how alike one person's rounds are) and never below 100 players, each with at least 20 counted rounds in the scored window.

• From bots only: no test with people was run before the plan. With the largest spread and within-player likeness the bot pilot measured, the formula alone asks for 33 players; the floor makes it 100. Bots are more regular than people are likely to be, so this is a floor, not a forecast.
• Re-checked on people, by a rule fixed now: after day 7, and each later day until it can be computed, a script on the game server measures the same two quantities on the shared rounds (never the result itself, so it is not a peek at the answer). The minimum becomes the larger of the two answers and is then fixed. This check uses all groups' rounds and only the first week's nights, so it probably underestimates the spread; the floor of 100 is there to cover that.
• If the minimum is not met, the gates are reported as exploratory and no pass may be claimed (above). No sample size is planned for G2.

When

1. Day 1 is the first game day (13:00 to 13:00 UTC) that starts with recording on. Its date is written in the plan when it happens.
2. Burn-in, days 1 to 7 at least: recording only; no retraining.
3. The switch-on: the nightly retraining is turned on, by a dated entry in the plan, once at least the minimum number of players have shared 20 or more rounds (outside tutorials) and every check of the records passes. The model trained that night is the frozen one.
4. The scored window: the 28 days after that night. If the minimum is not met by then, it continues day by day until it is, and never past day 60.
5. Day 60 is a hard stop. If the minimum is not met by then, the study ends and is reported as underpowered, with no extension. Recording stops by itself after day 60. Days the study is paused count toward the 28 days and day 60.

What is left out

Everything left out is counted and reported: players who turned sharing off (all their rounds), testers and the owner, players whose tap times look automated (over 80% under a quarter second, over at least 50 timed moves), unfinished duels, duels whose records do not verify, duels where the opponent's memory of you included rounds that were not shared (they cannot be replayed from the records), and from the counted rounds: tutorials, daily boss duels, card rounds, rounds with extra information, and silent rounds.

When it stops early

• Any failed check of the records (the sealed moves, the replay, the order of records, the consent) pauses the study until a dated amendment explains it. Rounds recorded during a pause count only if that amendment finds the failure did not affect them.
• No change to the duel rules, the served content or any of the 14 opponents during the window, including the release that shows the trained models on screen.
• The owner can stop it at any time; a stop is dated, with its reason, and what was collected is reported.
• An offline side check of other models on the published dataset (our fine-tune of Laya, Convai Innovations' open model, and TypeSafe's Jev) stops at $5.00 of Jev spend. It is reported, never a gate.

What happens to the rounds

• They stay on the game server during the study; the training runs there.
• After the study ends they become a dataset, in which each player's random id is replaced by a new one, with no key, name, address or time of day; rounds of players who turned sharing off or deleted their hero before it is built are not in it. It is built when the raw records are deleted, and published after the study ends.
• It is pseudonymous, not anonymous: each duel keeps its own id and secret so anyone can check it, and the game server's secret with a hero's save, or the duel details a player's own browser saw, could still match a duel to that player. We will not attempt that matching.
• The raw records, and every working copy made from them, are deleted at the latest at the end of study day 60 (counted from day 1, the first full game day with recording on); from then on only the dataset is kept. Once it is built, turning sharing off or deleting your hero can no longer reach it.
• Details: Privacy, Beat the AI duels.

What we will not claim

• that people "become predictable", or that any model "understands" anyone;
• anything about players in general: the people in it chose to share, after playing;
• that our fine-tune of Laya, or any general model, reads duels, unless measured and reported as measured;
• that anything here is new: a review of earlier work on rock-paper-scissors-like games comes before any write-up.

The bot pilot (bots, not people)

Before the plan was fixed, the whole data path ran end to end with bots on a copy of the game server: bots that share, play, withdraw and tap too fast, and the nightly training and scoring on them. Every number in this section is from bots; none is a result about people.

• Seven full runs of 120 bots over 4 game days (25 and 26 September 2026): every record verified, the players who withdrew left out, the fast tappers caught by the automation check and no one else.
• The habit model learned bots with fixed habits in every run. Our model learned them only weakly: it beat the uniform guess on them clearly in 3 of the 7 runs. On bots that react to the shown guess, its log-loss was lower than the habit model's in every run.
• The spread and within-player likeness used for the bot-based minimum: a per-round spread of 0.131 to 0.155 and a likeness of 0.010 to 0.018 across the six runs that kept a summary.

Frozen before the start

The plan records, by SHA-256 hash, the duel rules' code, the game content served, the configuration of each of the 14 opponents, the 63 features and the study's scoring code. A test in the repository recomputes every hash; changing any of them needs a dated amendment. New opponents added during the study are counted apart, never mixed in.

Decisions behind the plan

The project's owner decided, on 25 September 2026: the groups at 20% / 40% / 40%, a 28-day window with a hard stop at day 60 and "underpowered" as a publishable outcome; the nightly retraining built but left off until enough players share; our fine-tune of Laya only in the offline side check first. On 26 September 2026: the rounds published as a dataset after the study; under-16s asked to leave sharing off; raw records kept only until publication, at most 60 days, then only the dataset; and this plan published here before recording starts.

Amendments

Amendment 6, 26 September 2026, before the study starts (no one's rounds recorded, no one had agreed to share): after a review of the plan against the game's code. The consent now says the dataset can still be linked to a player by the game's own records (it is pseudonymous, not anonymous); the dataset is built when the raw records are deleted, so a withdrawal counts until then; working copies are deleted with the raw records; recording stops by itself after day 60, counted from day 1; the privacy page names the banded count of earlier duels; the rules for G5 failing and for too few players, which analyses are confirmatory, pauses, the probable underestimate in the sample-size check, and "a player is a random id" are written down; the release that shows the trained models may not change the frozen duel rules. No gate, threshold or frozen hash changed.

Five notes written before the pre-registration (the Reader the game serves, what the opponent remembers, older content versions, the daily boss, and why a memory built from unshared rounds is not recorded) are part of the plan above.

The files

The repository is private for now; these names let each rule be traced when the study's code is published.

• The plan of record: docs/MINDS-STUDY.md
• The data card: training/duel/README.md
• The bot pilot's summaries: bench/results/minds-study/
• The sample-size rule and its re-check: src/duel/study/score.ts, training/duel/power.ts
• The frozen hashes: test/minds-prereg.test.ts