agent-evaluation

# Agent Evaluation Evaluate the whole AI system under realistic nondeterminism. Produce a decision backed by pinned candidate identity, representative scenarios, observable traces, calibrated graders, and repeated trials proportionate to the decision. ## Ownership - Use this skill for stochastic prompt, model, RAG, or tool-agent behavior where one successful run is insufficient evidence. - Use [quality-assurance](../quality-assurance/SKILL.md) for ordinary deterministic software tests, browser journeys, performance tests, and release test campaigns. - Use [agent-skills](../agent-skills/SKILL.md) for skill packaging and deterministic lexical trigger contracts. Those checks do not prove model behavior. - Use [test-driven-development](../test-driven-development/SKILL.md) to implement a behavior change and [production-readiness](../production-readiness/SKILL.md) to decide whether the exact candidate is operable. ## Evaluation Modes - **Development diagnostic:** Use frozen development cases, paired repeated runs, traces, and deterministic graders to localize a weakness or compare an iteration. Return `ITERATE` or `INCONCLUSIVE`; this mode cannot authorize adoption or release and does not consume a decision holdout. - **Release or adoption decision:** Add a sealed holdout, predeclared decision rule, calibrated blinded graders, statistically adequate repetitions, contamination controls, and immutable candidate identity. Use this mode when the result will select a model, change ...