design-agent-eval-workflowlisted
Install: claude install-skill gaelic-ghost/socket
# Design Agent Eval Workflow
Design evaluation workflows for agent, skill, prompt, and automation behavior.
This skill is a framework-neutral eval planning surface. It helps define what
good behavior means, how to measure it repeatably, which tasks can safely run
fully automatically, and where escalation remains necessary because safety
cannot be engineered into the workflow with reasonable confidence.
## Inputs
- Required: the agent, skill, prompt, automation, or workflow to evaluate
- Useful: target runtime, task examples, expected outputs, failure modes, write
surface, external services touched, acceptable risk, and current validation
commands
- Optional: preferred eval runner, existing case format, model/provider
constraints, budget, cadence, and reporting destination
## Workflow
1. Restate the behavior under evaluation and the real decision the eval should
support.
2. Define the automation target:
- `full-auto`: safe to run without human approval when it passes objective
gates
- `auto-with-escalation`: runs automatically but stops on bounded ambiguity,
failed checks, secret exposure, destructive writes, or external side
effects
- `human-review`: requires review because the task cannot yet be made safe
enough through scope, validation, sandboxing, rollback, or orchestration
- `manual-only-for-now`: behavior is too underspecified or high-risk for a
useful eval-backed automation claim
3. Build the case set:
- happy path