The study / Layatown v1

Layatown v1

Rendered from layatown/docs/MODEL_CARD.md in the project repository, 21 September 2026, word for word. Laya is an open model by Convai Innovations (Apache-2.0); Jev is TypeSafe's model. Not affiliated with either. Cited result files are listed in the study.

A local System One model for the citizens of Jevtown: given a situation and a typed question with fixed answers, it returns a probability for each answer in one forward pass, without writing text. It is Laya (ModernBERT-large, 421M parameters) fine-tuned by distillation on the answers TypeSafe's Jev (jev-1.13.0) gave while running Jevtown, so that the city can run on one local GPU instead of a paid API.

Status: frozen and evaluated. The results below come from the pre-registered evaluation (layatown/docs/EVALUATION.md) and the committed result files it names; every number is quoted from those files. In short: it matches Jev closely on a day of the city it never saw and passes four of the five level A criteria, fails the fifth (one character effect overshoots), follows the city's written rules more often than Jev, and is clearly worse than Jev at screening text written by people.

Model details

Intended use

• Primary: answering Jevtown's own questions in the live city and the simulator (bun serve --live --policy laya): what a citizen does next, a citizen's character sheet, a juror's vote. Same wire shape as TypeSafe's POST /v1/systemone, served locally by layatown/serve.py.
• Users: the Jevtown project; researchers studying distillation of System One judgment models.

Out-of-scope use

• Anything outside Jevtown's worlds and question formats. The model has seen one simulator, one question style and one teacher; it is not a general judgment model.
• Screening text written by real people (chat moderation, usernames, reading event proposals). The model was not trained on these tasks and is clearly worse than Jev at all of them (see "Human-labelled sets" below). Do not use it for them. The live city under --policy laya runs in no-free-text mode instead: preset reactions instead of chat, a menu of events from the world data instead of typed proposals, and unscreened handles hidden, so no model reads player text.
• Decisions about real people, of any kind.
• Treating its probabilities as correct. They are an imitation of Jev's.

Training data

The Jevtown System One dataset (training/, data card training/README.md), train split only:

• 150,000 villager_decision questions sampled at random (seed 7) from 537,076 train decisions;
• all train character_sheet answers (4,265 sheets, 8 yes/no questions each: 34,120 answers);
• all train juror_vote answers (251), repeated 20 times.

Targets are Jev's full probability distributions (soft labels), not the top choice. The train split is 38 runs of the village (80 citizens) and the city (1,000 citizens), 18 to 20 September 2026. Excluded by the dataset itself: runs that addressed citizens by list position (wrong labels), runs not driven by Jev, and anything a real person typed. The held-out production day (test_production) and the validation runs were never trained on. All citizens are fictional.

Training procedure

• Loss: KL(Jev || model) over the options of each question (cross-entropy against soft targets minus their entropy).
• One epoch, 4,345 steps of about 12,288 padded tokens, length-bucketed batches.
• AdamW, weight decay 0.01; learning rate 3e-5 for the encoder and 1e-4 for the head; 2% linear warm-up then linear decay to 5%; gradient clipping at 1.0; bf16 autocast; gradient checkpointing.
• Fixed schedule: no early stopping, no checkpoint selection.
• After training, one temperature per question type (choice, score, noul) is fitted on the validation split by minimising cross-entropy against Jev, and stored in rl_agent_config.json.
• Hardware: one NVIDIA RTX 3080 Ti; about 77 minutes (EVALUATION.md, amendment recording the frozen checkpoint).

Evaluation

Pre-registered in layatown/docs/EVALUATION.md (three levels: fidelity to Jev on the held-out production day, human-labelled sets, city-level behaviour in the simulator; plus serving cost and latency).

Validation partials (not test results)

Seen during training at step 2,000 of 4,345, on every 20th validation decision and every 10th validation sheet answer, raw temperatures. These are monitoring numbers on the validation split, not the evaluation result:

Last validation look before freezing: decision TVD 0.107, top-1 0.866; sheet MAE 0.034.

Level A: fidelity to Jev on the held-out production day

test_production is the city's first public day (live-2026-09-19T23-51-37, 1,000 citizens, 435 ticks), never used for training, temperature fitting or selection. Every item, fitted temperatures, 95% cluster-bootstrap intervals over ticks (10,000 resamples). Source: layatown/results/v1-test_production.json; untouched Laya: layatown/results/base-raw-every10.json.

* Untouched Laya (revision 1c5edc17, raw temperatures) was measured before the plan was written, on every 10th item (23,878 decisions, 4,103 sheet answers, 17 juror votes), not on the full set; it is a disclosed pre-plan look. ** A4's paired interval of (model MAE - floor MAE): -0.051 [-0.058, -0.043] over 14 trials, 10,000 resamples (layatown/results/v1-juror-paired.json): it passes.

Offline speed in the evaluator: about 7.1 ms per decision question, 2.2 ms per sheet answer.

Character effects (A5), effect = mean p(action | in-group) - mean p(action | others) on the same test items. Tolerance: max(25% of Jev's effect, 0.05). Source: layatown/results/v1-character.json.

Level A result: A1 to A4 pass; A5 fails as written (brave fight). Level A therefore fails.

Level D: against references that do not come from Jev

Pre-registered by amendment before any system was queried; probes built from test-day states only. Estimates with paired 95% bootstrap intervals (10,000 resamples); "better" only where the interval of the difference excludes 0. Source: bench/results/beyond/results.md.

Reading notes from the results file: Jev is not bit-deterministic, so its order-invariance numbers include rerun noise that no baseline separates; the rule check counts direction only, and where Jev's effect is larger (cowardly vs brave, child flee) Layatown complies more often but moves less.

Human-labelled sets (level B)

The level B gates (B1 to B3) were written against the same five sets; level D's D3 scores them against the human labels. Layatown is worse than Jev on every one: chat moderation, username screening and event reading, and it blocked all 20 harmless lines screened beside abusive ones. Consequence: it must not serve chat, handle or proposal screening. Under --policy laya the city runs in no-free-text mode, so no model reads player text.

City-level behaviour (level C)

World jevtown (the village of 80, hash a635a002c79cd306), scenarios hard-week, siege and strange-week, 240 ticks. Primary comparison: seeds 6 to 10, out of sample for every system (fresh Jev runs eval8-*-jev, $2.70 in total; Layatown lt1-*-laya; rules eval8-*-rules). Source: bench/results/levelC/results.md, levelC-primary.txt, levelC-character.txt.

Seeds 1 to 5 (in-sample for Jev) are secondary and are not summarised in the committed results.

Speed and cost (RTX 3080 Ti, bf16)

Source: bench/results/serving/results.md and bench/results/levelC/results.md.

Serving gate D1 (p95 below 4,000 ms for a 1,000-decision tick) is not met as written: the first scale test with Jev's 4 s request limit skipped 3 of 60 ticks (request p95 3.83 s), and the limit was then raised to 7 s (the tick minus 1 s); at real speed p95 is 5.67 s. These measurements are on the same machine as the model; a live deployment adds the network between the GPU and the game server.

Verdict under the plan's rules

Not a "faithful replacement" (section 8 of EVALUATION.md requires every criterion to pass): A5 fails, C2 fails in strange-week, and D1 is not met as written. C1 passes everywhere, so it is not "not faithful" by section 8 either; the plan's three verdicts do not name this combination. Level B/D3 rules out every request kind that reads text written by people.

Bias, risks and limitations

• It copies its teacher. Trained to match Jev, it inherits Jev's biases and known failure modes (for example freezing when shown its own last answer, joining the crowd nearby, scoring a harmless chat line as more abusive next to abusive ones, missing two-step inferences; docs/JEV.md). Agreement with Jev is not correctness.
• One citizen at a time. Jev answered up to 100 citizens in one shared request; this model sees one citizen's slice. Where context moved Jev's answers, the model may differ.
• Narrow domain. One simulator, one question style, two worlds of the same design. Expect degradation on new worlds, new actions or reworded questions.
• Brave citizens fight more than Jev's. On the held-out day the brave-fight effect is 0.268 against Jev's 0.180 (A5 fails). Other character effects are within tolerance.
• Small juror data. 251 juror votes in training; juror answers are the least supported. MAE to Jev on the test day is 0.062, but on the jury factorial only 0.677 of matched pairs move in the direction the jury design predicted, against Jev's 0.910.
• Bad at text written by people. Chat moderation 0.737 (Jev 0.974), username screening 0.778 (Jev 1.000), event reading 0.426 (Jev 0.987), and every harmless line screened beside abusive ones was blocked. Never use it to screen or interpret player text.
• Thin serving margin on one GPU. At the city's full load the 95th-percentile request takes 5.67 s of a 7 s limit; a slower machine, a longer prompt format or more citizens would skip ticks.
• Not a text model. It cannot explain, write or converse.

Licence and terms

• The weights are a derivative of Laya and are released under Apache-2.0, the same licence. Redistribution must keep the licence and the attribution to Convai Innovations.
• The training labels are outputs of TypeSafe's Jev. The project owner confirmed on 21 September 2026 that TypeSafe's terms permit training on these outputs (training/README.md, "Provenance and permission"). Check TypeSafe's current terms before redistributing the weights or data outside the project.

Credits

• Convai Innovations for Laya (convaiinnovations/laya), the base model and the laya library used for sequence building, the option head and serving helpers.
• TypeSafe for Jev (jev-1.13.0), the teacher whose judgments this model imitates, and for the System One API shape it serves.
• ModernBERT, the encoder architecture underlying Laya.

license: apache-2.0
base_model: convaiinnovations/laya
base_model_revision: 1c5edc17a7acd8701df6fc341c0d179f1c62c982
language:
  - en
library_name: laya
tags:
  - system-one
  - distillation
  - agent-simulation
  - decision-model