← ClaudeAtlas

govkit-eval-suite-planninglisted

Plan a provider-neutral evaluation suite for an LLM feature. Use when the user asks to plan model evaluations or invokes /govkit-eval-suite-planning.
Accelerated-Innovation/governed-ai-delivery · ★ 20 · AI & Automation · score 68
Install: claude install-skill Accelerated-Innovation/governed-ai-delivery
# LLM Evaluation Suite Planning Plan the model evaluation suite for a feature. Determine the feature name from the user's request; if it is not provided, ask before proceeding. ## Inputs to read - `extensions/llm-application/manifest.yaml` - `extensions/llm-application/docs/backend/architecture/LLM_EVALUATION_CONTRACT.md` - `extensions/llm-application/docs/backend/architecture/MODEL_GUARDRAILS_CONTRACT.md` - `features/<feature_name>/nfrs.md` - `features/<feature_name>/acceptance.feature` - `features/<feature_name>/eval_criteria.yaml` - `features/<feature_name>/architecture_preflight.md` - `docs/backend/architecture/TECH_STACK.md` - applicable ADRs and implementation-profile guides If the LLM application extension is missing, stop and request its installation. Do not choose an evaluation product from model memory; use the configured adapter or propose a product decision through an ADR. ## Instructions 1. Define the evaluation subject by immutable model-routing, prompt, retrieval, guardrail, tool-definition, and configuration identities. 2. Select every applicable family: - deterministic structure and policy assertions - task correctness, relevance, completeness, and faithfulness - safety, prompt-injection, jailbreak, leakage, and policy-bypass cases - retrieval precision, recall, grounding, attribution, and source coverage - tool selection, argument validity, abstention, authorization, and recovery - latency, usage, cost, timeout, retry, and fallback be