The study / Can a System One model become its own teacher? (pre-registered plan, round 1)

Can a System One model become its own teacher? (pre-registered plan, round 1)

Rendered from layatown/docs/SELF-IMPROVE.md in the project repository, 22 September 2026, word for word. Laya is an open model by Convai Innovations (Apache-2.0); Jev is TypeSafe's model. Not affiliated with either. Cited result files are listed in the study.

Written and committed on 23 September 2026, before any round-1 label is generated or any student is trained. Changes after this commit are dated amendments at the end; the text above them stays as first written.

Question

Layatown v1 was taught by Jev (TypeSafe's paid model). Can it improve without Jev, from signals that do not come from Jev, and without getting worse at what it already does? Round 1 tests the cheapest such signal: self-consistency distillation. The teacher is v1 itself asked several equivalent ways; the student is v1 fine-tuned to give, from one view, the answer the views agree on.

Why it could work

Level D of the Layatown study measured that v1 is far more consistent than Jev under option reordering (top choice changes 1% against 14%) but not perfectly (mean TVD 0.015 across option orders, 0.053 across state-key orders). Averaging equivalent views cancels view-specific quirks; distilling the average into one view should make the model more consistent and possibly more rule-following, at no loss of fidelity. It could also fail: averaging can blur sharp, correct answers, and training on its own outputs can amplify its own biases (a model can become confidently wrong).

Systems

Data (no Jev labels anywhere in round 1)

• States: villager decision states from Layatown's own runs driven by v1 (the live city since 21 September and the Lab/Survival runs), plus decision states from the train split of the Jevtown dataset with their Jev targets discarded. Never val or test_production states.
• Views (K = 4): the original option order plus 3 seeded permutations of the options; for a random half of states the state's keys are also reordered. Probabilities are mapped back to the canonical option order.
• Teacher target: the mean of the K view distributions, then renormalised (a geometric-mean variant is recorded but not the primary).
• Size: up to 150,000 states (the size v1 was trained on). The exact count and hashes are recorded in an amendment before training.

Training

Fine-tune from v1 (not from base Laya), KL to the teacher target on a single random view per example, the same recipe as v1 (layatown/train.py), one epoch. To fit beside the live city's model server on the 12 GB GPU without stopping it, the lower encoder layers may be frozen; the frozen depth is recorded in an amendment before training. Temperatures are refitted on the validation split, as for v1.

Evaluation (identical suites to the Layatown study, same held-out data)

1. Fidelity to Jev, held-out day (test_production, 238,779 decisions; sheets; jurors), tick-bootstrap 95% CIs.
2. Level D suites (bench/beyond): D1 written-rule compliance (400 pairs), D2 option-order and state-key invariance (100 items x 4 orders), D3 human-labelled sets. Laya-only; no new Jev calls.
3. Village runs: lab-siege-v2 baselines and one Survival v2 run, compared descriptively with v1.
4. Cost: GPU time to label and to train; inference cost unchanged (one view).

Pre-registered criteria (self-1 against v1, paired where possible)

self-1 counts as an improvement only if all hold:

• S1 (the point): D2 option-order invariance improves: mean TVD across orders lower than v1's, paired 95% interval of the difference excludes 0; and top-choice change rate not higher.
• S2 (no regression in fidelity): held-out decision top-1 agreement with Jev not lower than v1 by more than 1.0 point, and TVD not higher by more than 0.01; sheet and juror MAE not higher by more than 0.01.
• S3 (no regression in rules): D1 compliance not lower than v1 by more than 2 points.
• S4 (reported, not required): D3 human-labelled sets and village-run metrics, with the direction stated plainly.

If S1 fails, round 1 is reported as a negative result. If S1 passes but S2 or S3 fails, it is reported as a trade-off, not an improvement, and self-1 is not deployed. Deployment to the live city needs all of S1-S3 and the owner's yes.

Threats to validity

• The teacher is the student's own past self: gains in consistency may come with confident errors; S2/S3 guard this.
• States from v1-driven runs follow v1's own policy (a closed loop): mixed with Jev-era train states to reduce it.
• Views are only reorderings; paraphrase invariance is not tested in round 1.
• Temperature refitting can change calibration independently of the weights; both are reported.

Later rounds (not pre-registered here)

Round 2: rewards from lived outcomes in the simulation, weighted by each villager's personality. Round 3: players' choices, with consent. Each gets its own plan before it runs.

Amendments

A1, 2026-09-22 04:00 UTC: the state pool and the sample (written before any label was generated)

• Date note: the header says 23 September; the plan was committed at 2026-09-21 23:49 -04:00 (commit 4f795f2). Nothing else about it changes.
• Pool (layatown/selfimprove/pool.ts, output in layatown/selfimprove/cache/, gitignored). v1-run states: every decision in runs/lt1-* and runs/lab/** answered by layatown-v1 with status 200, rebuilt exactly as training/build.ts rebuilds a decision; runs/lab/tune-* (transient work directories of a concurrent tuning job) and unfinished lab runs are left out so the pool is a fixed snapshot. 71 runs scanned, 68 used; 999,025 logged decisions, 501,986 exact duplicates dropped, 497,039 distinct v1-run states (runs/lt1-*: 443,156; lab: 53,883). Train-split states with targets dropped on read: 537,076 decisions, 4,265 character sheets, 251 juror votes. States identical to any val or test_production state (compared by state and question only) are excluded: none were found. SHA-256: v1-run ed3847fa…b337b599, train decisions 36817b6e…8655d8, sheets 0dd3ab97…0a60e, jurors adefb536…e14d5e (full hashes in pool-manifest.json).
• Live-city states: the live city's own log was not in the run roots this round (the instructions named runs/lt1-* and runs/lab/**); runs/lt1-*jevcity* are v1 runs of the city world.
• Sample (smaller than the 150,000 cap, for the 5-hour budget): measured v1 throughput on this shared GPU is about 150 sequences/s, so K = 4 views of 150,000 decisions would take about 86 GPU minutes to label alone. Round 1 uses 100,000 decision states: 50,000 v1-run + 50,000 train, drawn with random.Random(20260923).
• Anchors (a detail the plan did not fix): v1's recipe (train.py) also trains on character-sheet and juror questions. With their Jev targets discarded, all 4,265 train sheet states (8 questions each) and 251 juror states are labelled by v1 the same way and included (jurors repeated 20x, as in v1), so sheet and juror answers are anchored to v1's own rather than left to drift. Their options cannot be reordered (a yes/no question is always shown to the model as [false, true]), so their views 1-3 are three state-key orders for every state.
• Views, as the plan says, with these details fixed: view 0 is the logged order; views 1-3 are 3 option orders that differ from the logged one and from each other (seeded per state from SHA-256 of 20260923:<state id>); for a seeded half of the decision states, views 1-3 also reorder every JSON object with 2+ keys (the D2 "keys" transformation of bench/beyond/generate.ts). Teacher probabilities are v1's as served (its fitted temperatures applied), mapped back to the logged order and averaged (arithmetic mean, renormalised; the geometric mean is stored too). The single training view per question is also seeded per state.

A2, 2026-09-22 05:25 UTC: labels, frozen depth and hyperparameters (written before training and before any evaluation)

• Labels (layatown/selfimprove/label.py, selfimprove/cache/labels.pt, SHA-256 ebc7e1ab65bce2cf…b24f6e67): 104,516 states, 134,371 questions (100,000 decisions: 50,000 v1-run + 50,000 train; 34,120 sheet answers; 251 juror votes), 0 dropped, 537,484 forward passes of v1. 50,090 of the decisions had their state keys reordered in views 1-3. GPU time: 57.8 min of forward passes (58.8 min wall). Mean TVD between the 4 views: v1-run decisions 0.044 (top choice differs across views in 10.5%), train decisions 0.039 (8.5%), sheets 0.016, jurors 0.045; mean TVD of the teacher from the logged view: 0.035 / 0.031 / 0.010 / 0.047. The geometric mean differs from the arithmetic one by a TVD of 0.002 on decisions (so the primary choice hardly matters).
• Frozen depth: embeddings + encoder layers 0-19 frozen; layers 20-27, the final norm and the 2-layer head train (124.6M of 421.3M parameters). Freezing layers 0-15 ran out of the capped memory share (0.45 of the card, 5.23 GiB) in a 1-step smoke test at 8,192 tokens per step; with 20 frozen and 6,144 tokens the peak was 4.05 GiB. The smoke tests used the labels only (no evaluation of any kind).
• Hyperparameters (fixed now, not tuned): one epoch over 100,000 + 34,120 + 251x20 = 139,140 items in their one seeded view; length-bucketed batches of about 6,144 padded tokens (v1: 12,288; the memory cap); AdamW (weight decay 0.01), learning rate 1e-5 for the trainable encoder layers and 3e-5 for the head (a third of v1's 3e-5 / 1e-4, because this starts from v1 and the targets are close to v1's own answers); 2% linear warmup then linear decay to 5%; gradient clipping 1.0; bf16 autocast; gradient checkpointing; seed 7; loss KL(teacher || model) on the arithmetic mean target. Temperatures refitted on val with layatown/train.py:fit_temperature (same val slices as v1).
• Evaluation details fixed now: (a) held-out day: selfimprove/heldout.py runs layatown/evaluate.py's own main() with --split test_production --ci --floor for self-1 (output results/self-1-test_production.json), keeping its per-item predictions, then predicts the same items with v1 and computes paired tick-bootstrap intervals (10,000, seed 0) of self-1 minus v1. S2's margins are applied to the point estimates of the differences; intervals are reported. (b) Level D: selfimprove/beyond.ts asks self-1 exactly the probes of bench/results/beyond/ through serve.py on 127.0.0.1:8768, scored with the same rules and pairedBootstrap (B = 10,000, seed 20260921). The v1 side of S1/S3 is v1's committed answers (answers-layatown-v1.jsonl, runs/beyond/d3-layatown-v1.json); v1 is also re-served today and its fresh answers reported as a check only. S1 as written: the interval of the mean-TVD difference must lie below 0 and self-1's top-choice change rate must not exceed v1's. S3: point estimate of the pooled D1 compliance difference >= -0.02. (c) Village runs: v1 and self-1 both run today on the same engine working tree under runs/self-1-lab (lab-siege-v2 baselines seeds 1-3; Survival lab-survival-v2 seed 2, empty draft, the same inputs as v1's logged Survival v2 run), descriptive only. (d) v1-views (the teacher at inference) is not evaluated in round 1: it costs 4x on 238,779 held-out items and no criterion uses it; its view agreement on the training pool is reported above.
• Safety of the live service: this process's allocator is capped at 0.45 of the card and runs at nice 10; a guard (selfimprove/guard.sh) pauses the job while a 1-question probe of the live server takes over 1 s. During labelling the probe went from about 12 ms (job paused) to about 37 ms (job running); during a burst of city traffic it reached 1-2 s with or without the job.

A3, 2026-09-22 06:25 UTC: the trained checkpoint (written before any evaluation)

• checkpoints/self-1/model.safetensors SHA-256 dfdbeb523803547122a4b3860300533aa86be538e77fcb20c2755cb9af97c4bd (gitignored). 6,225 steps, one epoch, as fixed in A2. Refitted temperatures (choice, score, noul): 1.0218, 1.0, 1.0171 (v1: 1.0172, 1.0, 1.0014).
• Training time: 75.1 min of wall time between the first and last step, of which about 29.5 min the job was stopped (a bug in the safety guard: its process match also matched itself, so after pausing the job for a slow live probe it paused itself and never resumed; found and fixed, the job resumed from where it stopped, nothing was recomputed). GPU training time: about 45.6 min, plus 0.8 min of temperature refit. The guard now watches a PID.
• The in-training validation checks (every 1,500 steps, 2,162 val decisions against Jev, as in train.py) stayed at TVD 0.107 and top-1 0.862-0.865 (v1 at the end of its own training: 0.107 / 0.866). They were logged, not used to choose anything.

A4, 2026-09-22 08:35 UTC: outcome record (written after evaluation; changes nothing above)

• Criteria applied exactly as fixed in A2: S1 pass (option-order mean TVD 0.0155 -> 0.0147, paired difference -0.0008 [-0.0013, -0.0004]; top-choice changes 1% -> 0%), S2 fail (juror-vote MAE +0.0122 [+0.0094, +0.0148] against a margin of 0.01; decision top-1 -0.01 pt, decision TVD +0.0002, sheet MAE +0.0001 all within margin), S3 pass (D1 compliance 0.968 for both). Round 1 is therefore reported as a trade-off, not an improvement, and self-1 is not deployed. Full numbers: bench/results/self-1/results.md, layatown/results/self-1-*.json.