skill-eval

Featured

Measure whether a skill helps a named task or needs revision or removal. Use when: a bounded routing or coding evaluation is requested; conformance alone cannot show benefit.

AI & Automation 434 stars 40 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 97/100

Stars 20%
88
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# /skill-eval Answer one named maintenance decision: **retain, revise, remove, or insufficient evidence**. Choose the measurement that can answer that decision, use the caller's accepted cases and resource envelope, make one scoped recommendation, and stop. A completed evaluation does not require a positive difference. This is an optional specialist. The repository's selected runner owns execution and bounds; native results own measurements; BD and Git retain their authority. Do not add a core skill, AO evaluation command, scheduler, dashboard, second tracker, or mandatory review merely to run an experiment. ## Choose the question | Caller decision | Measurement | What it can establish | |---|---|---| | Does loading this skill change a specific observable act? | Behavioral probe with `scripts/probe-skill.sh` | Behavior change on that scenario; not correct code or productivity | | Does this package or version improve engineering outcomes at acceptable cost? | Repository-selected controlled coding comparison, such as `evals/skills-rpi` | Endpoint outcomes and cost on selected tasks; independent completion only when required exact-subject evidence exists | | Does a qualified memory update help later work? | Separate frozen-versus-updated memory transfer test | Narrow later-task reuse evidence with skill and runtime held fixed | | What happened in ordinary runs? | Existing native accounting and acceptance evidence | Observational failures, repairs and cost; not causal skill b...

Details

Author
boshu2
Repository
boshu2/agentops
Created
10 months ago
Last Updated
today
Language
Go
License
Apache-2.0

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category