init

Solid

Set up coder-eval in this repository — scan for what is worth evaluating (Claude Code skills, an MCP server, a CLI), then scaffold a task directory with one real, passing-or-failing task and the exact command to run it.

AI & Automation 116 stars 2 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 80/100

Stars 20%
69
Recency 20%
100
Frontmatter 20%
40
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Set up coder-eval in this repository Goal: leave the user with a task directory containing **one real task** they can run immediately, not an empty scaffold. The task must exercise something this repository actually ships. The user's request is: `$ARGUMENTS` ## Step 1 — Check prerequisites Run `coder-eval --version`. Installing this plugin did not install the CLI, and every later step needs it. If it is missing, follow `${CLAUDE_PLUGIN_ROOT}/reference/cli-setup.md`: offer the install, **ask before running it**, and confirm with `coder-eval --version` afterwards. Never install unprompted, and do not continue if the user declines. That reference also covers the other half of the version check — whether this project pins a coder-eval version, and what to do when the installed one does not match it. ## Step 2 — Check whether this repository is already configured This skill scaffolds a first suite. Run against a repository that already has one, it would write a "first task" beside an existing tree and report success — so find out before writing anything. Locate the eval tree per `${CLAUDE_PLUGIN_ROOT}/reference/repo-layout.md`. **If the repository already has tasks, report the inventory and stop**: how many task files and where, how many run directories, and whether there are experiments. Then point the user at the skills that act on what exists — - `/coder-eval:lint-tasks` to review the tasks already there; - `/coder-eval:analyze` to read a finished run. Only scaffol...

Details

Author
UiPath
Repository
UiPath/coder_eval
Created
1 months ago
Last Updated
today
Language
Python
License
Apache-2.0

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category