Split the work in two. A writing model, Claude in the examples below, drafts and explains. A second, much cheaper model makes the call: it picks one option, ranks a list, lets something through or holds it back. That second model never writes a sentence. It hands back a number and you act on the number.

The idea got a lot more concrete this week. Simon Willison wrote up Jev, a model from a company called TypeSafe that answers only three kinds of question. A yes or no question gets a probability. A pick-from-this-list question gets a probability for each option. A score question gets a number on a range. You can pour in all the text you like and you still get back a float. There’s no reasoning attached.

TypeSafe’s own launch post claims an end-to-end response time of 70 to 500 ms and input pricing at a small fraction of what a chat model costs, by its own comparison, with output free. Those are vendor numbers and I haven’t checked them. But even if you knock a zero off the enthusiasm, a model that can’t ramble is a different tool from a model that can.

Why pair a writer with a decider?

Because the two jobs have opposite needs. Writing wants range. You want the model to weigh tone and the person on the other end, then produce something a human would send. Deciding wants the opposite: a narrow, repeatable answer to a narrow question, fast enough to sit in the middle of a request without anyone noticing.

When one big model does both, you pay writing prices for deciding work. Every “is this spam?” or “which queue does this go to?” becomes a full generation with a paragraph of justification you’ll throw away.

A decision model skips all that. It reads the text and returns a number. Your code holds the threshold, so the policy lives somewhere you can read, test and change, instead of in a prompt anyone can quietly edit. I made a related argument about why a cheaper model wins once it clears the bar for a job. For some jobs the bar is a single number, and a model built to produce only that number is the right shape.

Before either model, check whether a plain rule already makes the call. When we went through Tallyfy looking for decisions a model should make, it usually did, and I wrote up that audit on tallyfy.com.

Three ways I plan to split the work

None of these are running yet. They’re the three patterns I went looking for when I read the Jev write-up, each tied to a writing job I’d give Claude.

Generate, then choose

Claude drafts a handful of candidates and the decider picks one. The first places I’d try it are email subject lines and the opening sentence of a help article. Claude is good at producing variety and bad at being ruthless about its own output, because it tends to like everything it wrote. A separate model asked “which of these would a busy operations manager open?”, and made to answer with a probability per option, has no such attachment. That makes the choosing cheap enough to run on every draft.

Decide first, write only where it matters

This flips the usual order. Take the step names in a Tallyfy template. Most are clear enough already, and asking Claude to rewrite every one of them produces churn nobody asked for. So the plan is to ask the decider one yes or no question per step, something like “would a new team member know what this step wants from them?”, and send only the low scorers to Claude for a clearer name. The expensive model writes where writing is needed and stays quiet everywhere else.

Let a number hold the door

A confident score lets a small change happen on its own, and a low one sends it to a person. The natural fit is small, reversible edits, like tagging a help article with its topic or flipping a status someone can undo in one click. Above the line, the change goes through. Below it, a human looks. The threshold is a setting I own, so when it’s wrong I move the line instead of rewriting a prompt.

Test the shape with Claude first

You don’t have to wait for API access to see whether the shape fits your problem. Ask Claude the same question in the same format: here is the text, here are the options, return only a probability for each, as JSON, nothing else. You get a stand-in with the same inputs and outputs, so you can wire the rest of the flow and find out whether the numbers are even useful.

TypeSafe publishes an MIT-licensed adapter that does this, a drop-in replacement for their API backed by Claude or another hosted LLM. That’s a smart move on their part, because it lets you compare cost, speed and accuracy before you commit.

There’s a catch in that stand-in. When Claude writes “0.9”, it’s writing a number as text, the same way it writes any other word, and nothing checks that 0.9 against how often Claude is right when it says 0.9. It’s a self-report of how confident the answer sounds, not a measurement. That’s fine for prototyping the plumbing. A threshold I pick while testing against Claude says nothing about where the line belongs on a model that produces its probabilities a different way, so I won’t tune one on it.

Where these models fall short

Text from strangers is the first thing I’d worry about. TypeSafe’s own known-limits page says the model doesn’t treat its input as hostile by default. The same page says content written to steer it, like an injected instruction or text that argues for its own classification, can move the answer. Outreach replies, support email and form submissions are all written by people you don’t control. If a stranger writes the input, a stranger can lean on the number. So anything a stranger wrote should need a higher score before it acts on its own, and a person should see the rest.

Calibration needs a careful read. The launch post says every answer comes with calibrated probabilities, so higher confidence should mean higher accuracy. The known-limits page is narrower about one output type: it says the score levels are weak in numerical calibration. It also warns against carrying a threshold tuned on a yes or no question over to a choice question. So I’d tune each threshold on my own data, for one question type, and tune it again when the question changes.

There’s no reason attached to any of it, either. When a decider puts something in the wrong bucket, there’s nothing to read, just the input and the score. The tempting fix is to hand the number to Claude and ask it to explain. That’s worse than no reason. A reason written after the fact for a number the writer didn’t produce is a story fitted to the answer, and it reads as convincing whether or not it’s true. Asking one chat model to decide and explain in the same breath goes wrong in its own way, because the explanation can argue itself into a different answer from the one it started with. Log the input and the score, and keep a human on the low ones.

Willison’s caution belongs here. He hopes nobody uses Jev to rank job applicants, because a single float can hide all kinds of bias baked into the model. I’d go further. Anything that decides something about a person, like who gets hired or who gets flagged, needs a reason a human can read and argue with. A bare number can’t give you one, and a reason bolted on afterwards only makes it look as if it did.

So I’ll keep the split where it belongs: numbers for sorting and routing text, words from Claude where a person will read them, a rule wherever a rule will do, and a human wherever the answer lands on someone’s life.