write-agent-testslisted

# Write Agent Tests Most of an agent is testable deterministically. Only a small part needs a model in the loop — and teams get this backwards, writing no tests because "it's non-deterministic". ## The layering | Layer | What it tests | Model in the loop? | Runs | |---|---|---|---| | 1. Unit | Tool handlers, formatters, validators, policy table | No | Every commit | | 2. Contract | Tool schemas, response caps, error text | No | Every commit | | 3. Trajectory | Loop behaviour with a **scripted fake model** | No | Every commit | | 4. Adversarial | Injection, abuse, malformed input | No (fixtures) | Every commit | | 5. Eval | End-to-end quality with the real model | Yes | PR + nightly | **Layers 1–4 are ordinary software tests.** They are fast, free, deterministic, and they catch most agent bugs. Only layer 5 costs money. ## When this applies - An agent exists with no tests, or only end-to-end eval runs - A regression shipped and nothing caught it - Setting up CI for an agent ## Do not use for - Building the eval dataset in depth → `build-eval-set` - Diagnosing one specific failure → `debug-agent-trajectory` ## Inputs to collect first | Input | Why needed | Default if unspecified | |---|---|---| | The tool registry and loop code | What to test | **Blocking** | | Existing test framework and conventions | Match them | Detect from the repo | | Known past failures | The highest-value test cases | Ask — every team has three | | Whether the agent takes irreversible actions |