write-eval
SolidWrite a live eval for new or changed runner/agent behavior using red/green TDD plus a falsification check that proves the eval fails when the behavior is broken. Use whenever you add or modify behavior that should be covered by an eval, when asked to "write an eval", "add an eval", "cover this with an eval", or after landing a feature that needs end-to-end proof it works.
Install
Quality Score: 87/100
Skill Content
Details
- Author
- dzhng
- Repository
- dzhng/duet-agent
- Created
- 5 months ago
- Last Updated
- 2 days ago
- Language
- TypeScript
- License
- Apache-2.0
Similar Skills
Semantically similar based on skill content — not just same category
eval-writer
Authors rigorous eval suites for AI agents, skills, and LLM systems — grounded in the 2026 eval-writing consensus (trace-driven error analysis, binary LLM judges, cross-family validation, α/κ agreement). Produces characterization, failure taxonomies, judge prompts, rubrics, and calibration protocols that harnesses (pmo-skill-refiner, CI) then execute. Two modes — Author (write from scratch) and Review (audit against the framework). First-class playbooks for per-skill evals and for pipeline stage-gate judgment content; generic fallback for arbitrary AI systems. Use whenever the user asks to write evals, audit evals, add eval coverage, calibrate a judge, build a rubric, write a judge prompt, or diagnose why a judge keeps passing broken outputs.
eval-skills
Eval and improve a skill against golden cases — run the target skill blind in a fresh, context-free subagent on each example input, grade the artifact against the expected outcome, and let the gaps drive the edits. Use when the user wants to test/eval/improve/harden a skill, says "this skill keeps producing X / keeps missing Y", or hands a skill plus example input→expected-output pairs. Pairs with [write-skills](../write-skills/SKILL.md) (the authoring principles every fix obeys).
eval
Testing whether skills produce correct agent behavior. Use after modifying a skill, during periodic review, or when model upgrades may affect skill relevance.