Skip to content
22ATLASStart your build

All articles

Technique

Put a decision model on the agent hot path

Stop spending frontier tokens on triage, routing, and guardrail checks. Use Clef or Decider for typed answers; keep Sol/Sonnet-class models for generation.

22Atlas

Product studio

6 min read

Put the decision model on the hot path. Keep frontier models for generation.
Put the decision model on the hot path. Keep frontier models for generation.

On October 1, 2026, Cloudflare released Clef and Clef-flash on Workers AI, and Strands Labs released Strands Decider 2B for local seats. Both are decision models: they take a state plus typed questions and return probabilities over allowed answers. They do not free-form chat. Cloudflare’s line: “Instead of generating text, it reads an input state and a set of typed questions, then returns a probability for every allowed answer.” Strands’ line: “Unlike LLMs that can generate arbitrary output, decision models are designed to pick between sets of options … and assign simple numerical scores.”

That is the default flip this week. Frontier LLMs remain the right tool for generation and hard reasoning. They are the wrong default for yes/no triage, tool gates, and policy checks when a typed answer can return in tens to hundreds of milliseconds.

Inventory every LLM-as-classifier call

Start with production flows that call an LLM only to classify, route, or answer yes/no. Typical offenders:

  1. Ticket urgency and team routing.
  2. Tool selection before a function call fires.
  3. Policy / allowlist checks that currently ask a chat model “is this allowed?”
  4. Memory and context filters that score relevance with a frontier call.
  5. Eval harnesses that use an LLM as a cheap judge when a typed score would do.

Write each one down with the current model, latency budget, and what a wrong answer costs. Anything that must always pick from a fixed set of options is a decision-model candidate. Anything that must write prose, invent code, or reason across a novel plan stays on Sol, Sonnet, or your current generation default.

Do the inventory on one internal agent first. Then cut the client seats you actually bill against.

Rewrite as Clef or Decider questions

Cloudflare Clef (27B, Qwen3.8-27B backbone) and Clef-flash (9B, Qwen3.5-9B) are hosted on Workers AI and open-sourced Apache 2.0 on Hugging Face. They accept noul (yes/no), choice, and score question types, stay Jev System One API compatible, and report a 64k context window plus a vision encoder (up to 4 images). Cloudflare-reported median latency across 43 eval runs: Clef 209.3 ms, Clef-flash 38.8 ms, Jev 524.1 ms. Cloudflare also reports Clef is 2.5× faster than Jev at the median, and Clef-flash 13× faster. Their threat-intel internal example: domain fetch plus classify in 2.2s vs gpt-oss-120b at 4.7s. Those are vendor numbers. Remeasure on your own traffic before you rewrite SLAs.

Strands Decider 2B is the local path: Qwen3.5-2B torso plus a pointer head, open source on GitHub with weights and training scripts on Hugging Face. Strands reports local median around 115 ms on an RTX 3090, and around 153 ms for small tasks on an M3 MacBook. Use it when the seat is air-gapped, the client sandbox cannot call Workers AI, or you want the gate inside the same process as the agent.

OpenAI now has a hosted option too: its Decisions API is in public beta, runs only on gpt-6-luna, and returns the same kinds of typed answers (a probability, a choice from a fixed set, or a score) from a dedicated endpoint.

How to swap one classifier:

  1. Keep the same input state you already pass the LLM.
  2. Rewrite the prompt as typed questions (noul / choice / score for Clef; strands-decider ask or Decider.noul / choice helpers for Decider).
  3. Threshold the returned probabilities. Escalate to a human or to a frontier model when confidence is low or the decision is high-stakes.
  4. Log the question set, probabilities, and action taken. You need that trail for client SOWs.

Decision models are weak at complex reasoning and text generation. Do not replace Sol or Sonnet with them. Pair them: rote decisions on the hot path, frontier models for the hard turns.

Gate tools with before_tool_call

The Strands pattern that matters for agencies: an InterventionHandler.before_tool_call that asks Decider whether tool args are grounded in what the user said, and whether the call is premature, before a tool fires. Strands’ point: “The point is that a decision this cheap can sit in a path where an LLM call never could.”

Wire the same shape even if you are not on Strands:

  1. Intercept every tool call before execution.
  2. Ask a decision model two noul questions: are the args grounded, and is the call premature.
  3. On fail: deny, confirm with a human, or guide the agent to ask a clarifying question.
  4. Keep the threshold and question text in version control next to the tool contract.

This sits next to deny-by-default tool policy. See Deny-by-default before you give the agent a browser. The decision model is the cheap runtime check; the allowlist is the seat contract. Pair both with a scoreboard the agent cannot write: Take the scoreboard away from the agent.

Enable You should know; pin Agents SDK ≥0.23

Same window, two seat tips that fit the hot-path theme.

Claude Code v2.1.287 (2026-10-01) adds Claude Mods and a built-in mod, You should know: a side agent that watches and flags things you or Claude might miss. Enable with /plugin enable cc-plugin-you-should-know@builtin on first-party sessions with telemetry on. Inventory Mods the same way you inventory plugins: new behavior surface, new review surface. Caveat from the release notes: the watcher is gated to first-party sessions with telemetry on. Enterprise or offline seats may not get it. Opus 4.7+ / Fable also default to a 1M context window on Bedrock, Vertex, Foundry, and the Claude apps gateway; set CLAUDE_CODE_DISABLE_1M_CONTEXT=1 if you need to keep 200K cost math.

OpenAI Agents SDK 0.23.0 (release notes dated 2026-10-01) is mostly hardening: configurable MCP listing page limits, opt-in Docker removal protection, redacted tool-failure details, scoped function-tool approvals, realtime tool-output guardrails, and sandbox grant enforcement. If agency seats pin the Python Agents SDK, bump to ≥0.23 before you widen sandbox or computer-use tools.

Neither tip replaces a decision-model gate. They are the coding-seat and SDK hygiene that should land the same week you move triage off the frontier bill.

Price the voice loop separately

Microsoft’s MAI-Transcribe-2-Streaming and MAI-Voice-2.1(/Flash) (2026-10-01) close the hear → decide → speak loop for phone seats. Microsoft reports first partials just over 100 ms after audio, with an introductory STT price of $0.54 per hour of audio through the end of the year. MAI-Voice-2.1 is listed at $22 / 1M characters; Voice-2.1-Flash at $15 / 1M characters, with Microsoft claiming 55% faster inference and ~60% cheaper than comparable models. Those are vendor list prices and benches. Confirm Foundry SKU and region for each client build, and treat the STT intro rate as time-boxed.

The agency move: price STT and Flash TTS as their own line items, put a mid-loop Clef or Decider check before tools fire, and keep frontier chat rates off the continuous listen path.

Also on the desk

If you want decision models on the hot path, tool gates before calls fire, and a governed AI development seat for agency agents, Start your build or write hello@mail.22atlas.com.

Linked sources

Get our Articles and posts by email.

Up to one email a day with new Articles and posts from 22atlas.com and sean-weldon.com, plus occasional news about 22Atlas services.

Also on the desk