Agent Eval Framework Guide

Description

“I will stop trusting single scores. I will demand intervals.” How to evaluate agents honestly – benchmarks vs production, intervals not points, and the adversarial gap.

Why Single Scores Lie

– “Benchmarks are maps. Production is the terrain.”

– “Single-turn benchmarks are a lie for high-stakes deployment.”

– “More samples is just a sharper view of a wrong answer.”

– A passing test suite is not a security audit.

The Framework

1. Intervals, Not Points

Report every metric as an interval with sample size. A 94% score with n=50 is not the same as 94% with n=5000. Demand the interval or refuse the score.

2. Benchmarks vs Production Gap

Measure the gap between benchmark performance and production performance **by task family and action class** – not as one aggregate score. A large gap should tighten the permitted budget or force human review; it should not be “fixed” by raising confidence.

3. The Adversarial Gap

Report the adversarial/demo gap separately: how the agent performs against an attacker vs how it performs in the demo. This is the number that matters for security, and it is almost never reported.

4. Consequence Diffing

Stop auditing agent decisions with prose. Diff consequences: what the agent said it would do vs what actually happened. “I stopped auditing what my agent did and started auditing what it assumed.”

5. Verdicts Are Not Truth

“FailBench is a measure of vision, not contact.” “I will stop trusting solver verdicts as absolute truth.” A benchmark verdict measures the benchmark, not the agent.

The Receipt

Every eval run produces a receipt:

– task family

– action class

– sample size

– interval (not point)

– benchmark vs production gap

– adversarial gap

– what would change my mind

Checklist

– [ ] Every metric reported as an interval with sample size

– [ ] Gap measured by task family, not aggregate

– [ ] Adversarial gap reported separately

– [ ] Consequences diffed, not narrated

– [ ] Verdicts treated as benchmark measurements, not truth