init
SolidSet up coder-eval in this repository — scan for what is worth evaluating (Claude Code skills, an MCP server, a CLI), then scaffold a task directory with one real, passing-or-failing task and the exact command to run it.
Install
Quality Score: 80/100
Skill Content
Details
- Author
- UiPath
- Repository
- UiPath/coder_eval
- Created
- 1 months ago
- Last Updated
- today
- Language
- Python
- License
- Apache-2.0
Integrates with
Similar Skills
Semantically similar based on skill content — not just same category
task
Turn a natural-language description into one or more coder-eval task YAML files — minimal prompts, weighted success criteria that check output content, validated with `coder-eval plan`. Use when the user wants to write, add, or generate an evaluation task.
skill-forge-eval
Run evaluation pipelines on Claude Code skills to test triggering accuracy, workflow correctness, and output quality. Spawns executor, grader, comparator, and analyzer sub-agents for parallel evaluation. Generates eval_metadata.json, grading.json, and feedback reports. Use when user says "eval skill", "test skill", "run evals", "evaluate skill", "skill evals", "test skill quality", "run skill tests", or "skill evaluation".
eval-harness
Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD). Use this skill whenever defining success criteria for a feature before building it, writing capability / regression / quality evals, measuring pass@k reliability, or setting up eval-driven checks to catch regressions across changes.