write-eval
SolidWrite a live eval for new or changed runner/agent behavior using red/green TDD plus a falsification check that proves the eval fails when the behavior is broken. Use whenever you add or modify behavior that should be covered by an eval, when asked to "write an eval", "add an eval", "cover this with an eval", or after landing a feature that needs end-to-end proof it works.
Install
Quality Score: 87/100
Skill Content
Details
- Author
- dzhng
- Repository
- dzhng/duet-agent
- Created
- 3 months ago
- Last Updated
- 2 days ago
- Language
- TypeScript
- License
- Apache-2.0
Similar Skills
Semantically similar based on skill content — not just same category
eval-writer
Authors rigorous eval suites for AI agents, skills, and LLM systems — grounded in the 2026 eval-writing consensus (trace-driven error analysis, binary LLM judges, cross-family validation, α/κ agreement). Produces characterization, failure taxonomies, judge prompts, rubrics, and calibration protocols that harnesses (pmo-skill-refiner, CI) then execute. Two modes — Author (write from scratch) and Review (audit against the framework). First-class playbooks for per-skill evals and for pipeline stage-gate judgment content; generic fallback for arbitrary AI systems. Use whenever the user asks to write evals, audit evals, add eval coverage, calibrate a judge, build a rubric, write a judge prompt, or diagnose why a judge keeps passing broken outputs.
eval-skills
Eval and improve a skill against golden cases — run the target skill blind in a fresh, context-free subagent on each example input, grade the artifact against the expected outcome, and let the gaps drive the edits. Use when the user wants to test/eval/improve/harden a skill, says "this skill keeps producing X / keeps missing Y", or hands a skill plus example input→expected-output pairs. Pairs with [write-skills](../write-skills/SKILL.md) (the authoring principles every fix obeys).
build-agent-evals
Build automated evaluations for an AI agent from scratch: collecting tasks from real failures, choosing code/model/human graders, picking pass@k vs pass^k, building an isolated harness, and keeping the suite honest over time. Use this whenever someone wants to measure, benchmark, or regression-test an agent, write an eval harness for an LLM agent, decide how to grade non-deterministic output, set up an LLM-as-judge, or asks any version of "how do I know if my agent is actually getting better." Trigger even when they say "tests for my agent," "eval set," or "agent benchmark" rather than the word "evals," or when they ask about benchmark contamination or a model recognizing the eval. Not for container or resource limits making scores flaky across runs; that's calibrate-eval-infrastructure.