← ClaudeAtlas

design-agent-eval-workflowlisted

Design evaluation workflows for agent, skill, prompt, and automation behavior before implementation. Use when choosing eval cases, graders, thresholds, regression runs, full-auto gates, or escalation points for Codex skills, repo-local agents, OpenAI Agents SDK services, LangGraph graphs, or other agentic workflows.
gaelic-ghost/socket · ★ 6 · AI & Automation · score 71
Install: claude install-skill gaelic-ghost/socket
# Design Agent Eval Workflow Design evaluation workflows for agent, skill, prompt, and automation behavior. This skill is a framework-neutral eval planning surface. It helps define what good behavior means, how to measure it repeatably, which tasks can safely run fully automatically, and where escalation remains necessary because safety cannot be engineered into the workflow with reasonable confidence. ## Inputs - Required: the agent, skill, prompt, automation, or workflow to evaluate - Useful: target runtime, task examples, expected outputs, failure modes, write surface, external services touched, acceptable risk, and current validation commands - Optional: preferred eval runner, existing case format, model/provider constraints, budget, cadence, and reporting destination ## Workflow 1. Restate the behavior under evaluation and the real decision the eval should support. 2. Define the automation target: - `full-auto`: safe to run without human approval when it passes objective gates - `auto-with-escalation`: runs automatically but stops on bounded ambiguity, failed checks, secret exposure, destructive writes, or external side effects - `human-review`: requires review because the task cannot yet be made safe enough through scope, validation, sandboxing, rollback, or orchestration - `manual-only-for-now`: behavior is too underspecified or high-risk for a useful eval-backed automation claim 3. Build the case set: - happy path