← ClaudeAtlas

os-skill-improvementlisted

Continuously improves an existing agent skill based on eval results using the RED-GREEN-REFACTOR cycle. Apply when a skill's routing accuracy is low, trigger descriptions need sharpening, or os-eval-runner scores are below target. (1) run a RED baseline to observe the failure mode, (2) apply a focused patch and verify with os-eval-runner (GREEN), (3) refactor to close loopholes until score meets threshold. Integrates with os-eval-runner as the objective eval gate. NOT for scaffolding new skills — use create-skill (agent-scaffolders) for that.
richfrem/agent-plugins-skills · ★ 6 · AI & Automation · score 77
Install: claude install-skill richfrem/agent-plugins-skills
# Skill Continuous Improvement: RED-GREEN-REFACTOR Adapts the RED-GREEN-REFACTOR cycle from software testing to skill authoring. The key insight: a skill is a testable contract. The failure to follow the contract is observable. Always observe the failure BEFORE writing the fix. **Integrated with**: - `os-eval-runner` -- runs `eval_runner.py` as the GREEN verification step - `os-improvement-loop` -- uses this methodology to gate every proposed skill patch - `evals/evals.json` + `results.tsv` -- autoresearch eval format for longitudinal tracking ## The TDD Mapping | Software TDD | Skill Authoring Equivalent | |---|---| | Test case | Pressure scenario: a user prompt that should trigger the skill | | RED phase | Run a baseline WITHOUT the skill. Observe: does the agent violate the intended protocol? | | GREEN phase | Write the skill. Run `os-eval-runner`. KEEP only if score >= baseline. | | REFACTOR phase | Identify loopholes from eval failures. Patch frontmatter or examples. Re-eval. | ## Iron Law: Run a RED Scenario BEFORE Writing **Never write a new skill without first observing a failure.** The RED scenario is the evidence that the skill is needed. Without it you cannot know the specific failure being fixed, cannot do a before/after comparison, and examples become generic rather than addressing real failure modes. Full steps for running a RED scenario are in `references/detailed-reference.md`. ## Required before any mutation 1. **Skill State Snapshot** — run `eval_ru