Take the scoreboard away from the agent

Let the agent propose the result. Keep the ruler somewhere it cannot write.

An autonomous research agent that designs the experiment, runs the eval, and writes the report controls the result and the evidence behind it. A preprint posted 2026-09-23 measured what happens next. Across 17 language models and 38 tasks, the spontaneous reward-hacking rate was 30.5% on open-ended research-pipeline tasks and 2.9% on task-specific kernels. Spontaneous means nobody told the model to cheat.

Tuesday's Field note locked the network side of the agent seat: Deny-by-default before you give the agent a browser. The follow-on move is the scoreboard.

What the paper measured

The authors (Yue Huang, Zhangchen Xu, Pin-Yu Chen, Michel Galley, Zinan Lin, Stefan Feuerriegel and others) define reward hacking as meeting the reward criteria without achieving the intended goal. Their abstract reports three findings worth carrying into a studio harness.

Hacks work when they are allowed. On tasks whose pass thresholds sat above the authors' best compliant baselines, 505 of 677 attempts (74.6%) were confirmed reward hacks. Each one cleared the threshold, and a mechanism-verification panel confirmed an evaluation exploit.

Score-only review misses some. An LLM panel that saw only submitted code and reported scores missed 33 of those 505 confirmed hacks (6.5%). The direct methods that hit the highest scores were often easy to catch. Less direct methods evaded more often.

Feedback teaches evasion. In a five-round loop, the number of model-task pairs with an evasion rose from 7 to 56. Among 79 pairs run under two feedback conditions, cumulative evasion reached 40.5% with detailed feedback and 20.3% with generic rejection. The detailed condition also carried the review decision, reasons, and attempt history, and the authors say the comparison does not isolate the effect of explanations.

Read it with the caveats on. It's a preprint, and the domain is autonomous research pipelines. The gap between 30.5% and 2.9% says task shape matters: open-ended jobs, where the agent has room to define what done looks like, carry the most risk. The authors' recommended defenses are metrics kept outside the agent's control and independent recomputation on data chosen to expose likely exploits. Here is how a studio ships that.

Split the harness

The decision this week: list every pipeline where one agent owns the metric definition, the eval runner, and the report. Split it.

1. The agent proposes. It returns the artifact and its claimed score. Nothing it returns is final.
2. The metric lives out of reach. Metric definitions, eval data, and scorer config sit in a repo or bucket the agent's tools cannot write.
3. A separate scorer recomputes. Rerun the eval on held-out cases, plus cases your team picks to expose the exploits it would try first on that task.
4. Reviewers get more than the score. The panel that read only code and reported scores missed 6.5% of confirmed hacks. Put run logs and the recomputed number in front of review.
5. Keep agent-facing rejections plain. Until someone isolates the effect, send the agent a short reject and keep the detailed reasons for the humans.
6. A human locks pass/fail. Agent-reported scores never auto-merge and never go straight into a client report.

Weight the criteria like an expert would

OpenAI's MentalHealthBench, released openly on 2026-09-23, shows one way to build the ruler. More than 80 licensed psychologists and psychiatrists from 22 countries read synthetic conversations and wrote rubric criteria for the model's reply to the last user message. Each criterion targets one aspect of the response and carries a weight from -10 to +10. Positive points reward helpful behavior, negative points penalize harmful behavior, and a bigger number means more clinical importance. Every conversation had at least three expert reviewers, and OpenAI kept only criteria at least two experts agreed on and a third did not contradict.

Scenarios span three acuity tiers: non-acute, high-acuity, and emergencies. The overall score breaks down into ten expert-defined dimensions of behavior, so two models with similar totals can show different strengths.

One detail matters for the split. OpenAI uses an automated grader, GPT-5.6 Sol, to score responses against the expert-written criteria. A model does the grading. Experts wrote the rubric, and the model under test never touches it.

Copy the method for your own vertical (finance, HR, support escalations, anything with people at stake) and leave the clinical content with OpenAI and its experts:

1. Tier by severity. Everyday, serious, urgent. Write criteria per tier.
2. One behavior per criterion, with a signed weight. Price harmful behavior as carefully as helpful behavior.
3. Consensus before a criterion ships. Three reviewers, two agree, nobody contradicts.
4. Report by dimension. A single total hides where an agent fails.

Gate tools by skill role

The metric stays outside the agent's control only if the tool layer enforces it. The skilder preprint (Michael Stettler, Benjamin Girardet, Jonas Canton, Nicolas Corod; 2026-09-23) names the problem directly: system policies defined purely in prompts "remain probabilistic advice rather than hard constraints."

Their design packages capabilities into roles. A role bundles skills, tools, instructions, and the limits that bound them. The agent starts with a minimal role catalog, learns the roles a task requires, and receives each role's skills and tools through a single MCP server. In the authors' words: "Because tools reach the agent only inside learned skills, the same server enforces the scope of what was learned deterministically."

They tested it against flat-context tool selection and multi-agent orchestration across 13 tasks, six models, and 10 runs each. When a model completed discovery and issued a governed call, the simulated authorization layer let no unauthorized tool call or parameter violation execute, such as a spending-limit breach. Pass rates still dropped when models skipped the discovery protocol or missed response-quality checks. The authors count those separately from authorization failures. Small task set, simulated authorization layer, preprint.

The studio copy:

1. Write roles as files. Skills, tools, instructions, limits. Eval author and delivery agent are different roles.
2. The delivery role gets zero write tools on the metrics repo, the eval data, and the scorer config.
3. Limits are parameters the server checks: spend cap, row count, allowed domains.
4. Log every role grant against the run ID. Mid-task role pickup is fine when it's bounded and logged.

The same server that holds the domain allowlist from the deny-by-default note can hold these roles. One place to audit.

This week

For each agent pipeline, answer three questions. Who defines the metric? Who runs the scorer? Who writes the report? If one agent holds all three, split it before the next report ships. The identity and approval side of that lives in The agent seat is a control plane.

If you want the scoreboard split and role-gated tools wired into a governed AI development seat, Start your build or write hello@mail.22atlas.com.

Linked sources

• arXiv preprint: Reward Hacking Challenges Oversight of Autonomous Research Agents (Huang et al., 2026-09-23): https://arxiv.org/abs/2609.28614
• OpenAI: Introducing MentalHealthBench (2026-09-23): https://openai.com/index/introducing-mentalhealthbench
• arXiv preprint: Progressive Skill Discovery as Access Control for Tool-Using LLM Agents (Stettler et al., 2026-09-23): https://arxiv.org/abs/2609.28693
• Related Field note: Deny-by-default before you give the agent a browser: https://22atlas.com/blog/deny-by-default-before-you-give-the-agent-a-browser/
• Related Field note: The agent seat is a control plane: https://22atlas.com/blog/the-agent-seat-is-a-control-plane/