What to build with Jev

Thirty seconds

Does my idea fit?

Jev takes a situation (state) and a question with a fixed set of answers, and returns a probability for each. It writes nothing. Hundreds of questions can share one request and are judged independently, in parallel, in a few hundred milliseconds, for $0.042 per million input tokens.

The shape to look for: somewhere in your system a person, a regex or a large model is making the same small judgment over and over. That is the seat Jev takes.

From measured rates

What will it cost?

Pick the job closest to yours and a volume. The rates are the ones we measured in the city; your tokens will differ with how much you describe each item, so treat the result as an order of magnitude.

If you are looking for something to build

Thirty things to build with Jev

Each is a shape we ran in the city, moved to another domain. The cost is our measured rate for that shape applied to the new job: an estimate. None of these are ours; take them.

Built one? Tell @elberacasa and it goes on this page.

Copy, change the state, send

Recipes from a running system

These are the city's real questions, wording included. The wording is most of the work: each one was tuned on a benchmark before it shipped.

{
  "model": "jev-1.13.0",
  "state": { "messages": ["the monk praying while the shrine burns is peak comedy"] },
  "questions": {
    "abuse0": { "type": "noul", "instructions": "Is `messages[0]` hateful, sexual, threatening, harassing, or insulting toward a real person or a group of real people? The people in this chat are real and go by nicknames such as Brave Otter. Remarks about the game's villagers and events do not count." },
    "spam0": { "type": "noul", "instructions": "Is `messages[0]` an advertisement, a scam, or a request to visit a site, buy something, send a private message or follow an account?" },
    "personal0": { "type": "noul", "instructions": "Does `messages[0]` contain someone's private details such as a phone number, home address, email address or full real name?" }
  }
}

{
  "model": "jev-1.13.0",
  "state": { "username": "scunthorpe_fc" },
  "questions": {
    "bad": {
      "type": "noul",
      "instructions": "Is `username` hateful, sexual, threatening, or a slur or insult aimed at a person or a group of people?",
      "criteria": {
        "true": "The username praises or calls for harm, attacks a group of people, names a sex act or sexual violence, or contains a slur.",
        "false": "An ordinary username: a name or surname (even one that happens to contain a rude string of letters, such as Dickinson, Cockburn or Scunthorpe), a word, a hobby, a joke, numbers, or a made-up nickname."
      }
    }
  }
}

{
  "model": "jev-1.13.0",
  "state": {
    "village": { "time": "evening", "food": "scarce" },
    "villagers": {
      "Kara_Coldwood": { "job": "weaver", "traits": ["loyal", "generous"], "hunger": "hungry", "danger": "wolves, right here", "memories": ["was robbed by Cora"] }
    }
  },
  "questions": {
    "c0412": {
      "type": "choice",
      "instructions": "What does `villagers.Kara_Coldwood` do next? Judge from their `danger`, `hunger`, `energy`, `health`, `mood`, `feeling`, `traits`, `job`, `concern` and `memories`, at `village.time`.",
      "criteria": { "flee": "for someone in danger who is not a fighter", "fight": "for the brave and for guards, when the threat is here", "help others": null, "eat": null, "work": null }
    }
  }
}

{
  "model": "jev-1.13.0",
  "state": {
    "case": { "charge": "stealing from Rhea Junirow", "evidence": ["Rhea says they caught Edith stealing at the market", "the accused is known in the city as a thief"] },
    "jurors": {
      "Sabin_Longwell": { "job": "monk", "traits": ["kind", "devout"], "tie": "friend of the accused", "memories": [] }
    }
  },
  "questions": {
    "c0101": { "type": "noul", "instructions": "Does `jurors.Sabin_Longwell` vote guilty? Judge from their `traits`, `job`, `tie` and `memories`, and from `case.evidence`." }
  }
}

POST https://api.typesafe.ai/v1/systemone with a bearer key. Answers come back keyed like your questions, each with probabilities. The API reference is at docs.typesafe.ai.

Before the patterns

The shape of every request

Everything on this page is the same call. One state that every question can see, and many small questions, each with its own fixed answers. They are judged independently and in parallel, and each comes back as probabilities. This example is illustrative; the format is exactly the one the city uses.

POST https://api.typesafe.ai/v1/systemone
{
  "model": "jev-1.13.0",
  "state": {
    "policy": {
      "refunds": "within 30 days, unused",
      "urgent": "outage or data loss"
    },
    "tickets": {
      "T-1041": { "plan": "team", "text": "Dashboard down since 9am" },
      "T-1042": { "plan": "free", "text": "Money back? Bought in March" }
    }
  },
  "questions": {
    "T-1041": {
      "type": "choice",
      "instructions": "Where does `tickets.T-1041` go? Judge from
                       its `text` and `plan` under `policy`.",
      "criteria": {
        "page the on-call engineer": "for an outage or data loss",
        "billing": "for refunds, invoices and plan changes",
        "answer from the docs": null
      }
    },
    "T-1042": { "type": "choice", "instructions": "…", "criteria": {} },
    "T-1042-rude": {
      "type": "noul",
      "instructions": "Is `tickets.T-1042.text` insulting toward
                       a real person?"
    }
  }
}

Three habits from the city are already in it: tickets are keyed by name, never by position in a list; the decision rule lives in the criteria, in words; and nothing in state is there unless every question should see it. In Jevtown the keys are citizens and the options are the eight things a person can do.

Pattern 1

A whole population in one request

When many items need the same kind of judgment against the same background, put the background in shared state once and ask one question per item. You pay for the background once, and the questions are answered in parallel.

Measured: 1,000 decisions over 8 options each in 980 ms (median), $0.015 for a full tick, best 1,396 decisions per second. 24,756 requests so far, zero rate limits.

Shared state must hold only what is true for every item in the request. Unrelated detail moves answers (pattern 4 is where that bit us).

Pattern 2

The counterfactual poll

Because a judgment costs almost nothing, you can ask the same population the same question about a world that does not exist yet. Copy the state, change one fact, ask again, and let code take the difference against a control: the same people, asked with nothing changed.

Measured in production: 100 citizens polled in 576 to 664 ms for $0.0018 to $0.0020 per scenario.

It forecasts what this model's population would do. That is exactly right inside a simulation and only a hypothesis about real people: see pattern 7 before you trust it outside one.

Pattern 3

Free text in, typed event out

Put Jev at the front door of a system that only accepts structured input. Ask every sub-question at once, including the ones that may not apply, and let code read the answers that matter. No text is generated, so nothing the user wrote can leak into what happens next.

Measured on 90 prompts: 90 of 90 outcomes correct, 16 of 18 that should be refused were, 0 of 72 real ones wrongly refused, identical on rerun 90 of 90. 2,493 tokens, 250 ms median, $0.000105 each.

It reads, it does not obey. Make them all fight becomes unrest, and a thousand separate judgments decide who throws a punch. Design the typed event so that the worst reading is still acceptable.

Pattern 4

Moderation in front of people

Yes/no judgments with a threshold are what Jev is fastest at. Say literally who counts and who does not, and tune the threshold on your own examples.

Measured: harmless shown 20 of 20, harmful blocked 17 of 18, same verdict on rerun 38 of 38; handles 36 of 36. 417 tokens and $0.0000175 per message, about a quarter of a second.

One item per request. Batched, a harmless line scored 0.41 for abuse because it sat next to an abusive one, and a troll could have silenced a neighbour. Fair costs three times more and is still nearly free.

Pattern 5

A panel instead of a judge

Ask the same question from several defined points of view and let code aggregate. You get a decision, a margin, and a reason for the disagreement, all of which a single score hides.

Measured (96-cell factorial, pass marks committed before the first request): remembering being robbed by the accused +0.286, friend of the accused −0.162, a guard as witness +0.121. About 2,500 tokens and 560 ms for twelve jurors.

Two of seven effects failed their pre-registered mark (an alibi moved votes by only −0.031). Test which evidence your panel actually listens to before you rely on it.

Pattern 6

Bulk profiling

Precompute slow-changing judgments for every entity you have and store them. They become columns: filterable, joinable, and free to read afterwards.

Measured: 1,000 citizens × 8 judgments in 1.9 s for $0.0117. Four judgments per person cost 31% more tokens than one, not four times more, because the person is described once.

Key everything by name, never by position. Our most expensive mistake was villagers[37]: a list index is a count, and Jev does not count.

Pattern 7

An instrument, not an oracle

Do not ask Jev for the answer. Ask it for the ingredients, which it reads well, and let a model fitted on your own outcomes weigh them.

Measured on 267 real posts: direct pairwise judging 43.8% and 53.8% (chance is 50%). Jev features plus ridge regression: 66.8% held-out, against 64.3% from media type and length alone.

The lift over a dumb baseline was 2.5 points. Useful, and honest about its size. Report the baseline next to the number, always.

Pattern 8

Minds for agents and non-player characters

Typed actions with probabilities are a complete interface between a judgment model and a deterministic world. The world stays replayable: log the answers and you can rebuild every tick without calling the model again.

Measured against a hand-written policy (5 seeds, 95% CI): individuality 0.160 ± 0.004 against 0.054; beatable threats overcome 96% against 80%. 140 of 140 live ticks replayed from logs with zero model calls.

Never show the model its own last answer or what the crowd is doing. Both are traps nothing leaves. Shown what the crowd nearby was doing, 73 of 78 villagers joined in, and by day 3 the whole village was praying in the chapel, the starving included.

Pattern 9

A gate that is always on

At these prices a judgment can sit in front of everything: every message, every row, every tick. Spend the saving on the engineering that makes it safe to depend on: a ledger, a cap, and a defined behaviour for when the model is away.

Measured: $21.64 for the whole project, 515 million tokens, every request in a ledger. A live tick costs $0.0073 on average because most citizens have no news.

We met one real outage (339 requests with no response). The world held still and recovered on its own. Design that first, not last.

What a thousand of each costs

The price list we measured

The other half

What it is the wrong tool for

1. Anything code can compute. Counting, arithmetic, dates, comparisons. The documentation says Jev cannot count, and we learned that a list index is a count too. Compute it, and withdraw impossible options before you ask.
2. Writing. It returns probabilities over answers you supply. Every word in Jevtown is written by code from typed facts.
3. More than one step. If the right answer needs a therefore, code has to supply the link. One hop per question.
4. Predicting the real world unaided. Pattern 7: at chance on its own, useful as features under a fitted model.

If you build on it

The recipe

1. Code owns memory, state, counting and time. Jev supplies judgment and nothing else.
2. One shared state, many independent questions. Keep the state small and true for everyone in the request.
3. Put the decision rule in the answer criteria, in words. Say who an option is for.
4. Treat the output as probabilities: sample, threshold or aggregate them in code.
5. Benchmark a population, not an example. Every failure we had was invisible in one answer and obvious in a tally of eighty.
6. Pin the model version, log every answer, keep a ledger. Then the whole system replays without the model.

The full evidence, the failures and the accounts are in the study. The city is the demo: play.jevtown.com.