hatch3r-ai-feature

Solid

Eval-driven development workflow for shipping AI features — write eval before prompt, measure, iterate, ship with caching + cost telemetry + model fallback + hallucination SLI

AI & Automation 26 stars 4 forks Updated 1 weeks ago MIT

Install

View on GitHub

Quality Score: 83/100

Stars 20%
48
Recency 20%
90
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# AI Feature Workflow (Eval-Driven) ## Quick Start **Applies when:** the project ships an LLM/AI feature. On a project with no AI feature, skip — do not scaffold AI machinery pre-emptively (per `rules/hatch3r-right-sizing.md`). Run this skill before shipping any LLM-driven feature. It defines the canonical eval-driven loop (write eval, write prompt, measure, iterate) and the production-readiness gates. Skipping any of the 9 steps = the feature is not done. This skill is the implementation counterpart to `rules/hatch3r-ai-evals.md` (backend governance) and `rules/hatch3r-ai-ux-patterns.md` (UI governance). The rules define the bar; this skill defines the route to clearing the bar. ## Step 0 — Detect Ambiguity (P8 B1) Before any work, scan the invocation for unresolved questions in scope, intent, acceptance criteria, target environment, or irreversibility. If any are found, ask the user via the platform-native question tool per `agents/shared/user-question-protocol.md`. Do not proceed under silent assumption. Default path, not an exception. Triggers for THIS skill: task class (classification vs open-ended vs RAG vs agentic), model pin (Sonnet vs Opus vs Haiku), eval threshold values, budget per request (cost cap), and fallback policy (graceful degrade vs hard fail). ## Fan-out Discipline (P8 B2) Fan-out scales with task size; token cost never justifies serializing independent work (`rules/hatch3r-fan-out-discipline.md` P8 B2; `agents/shared/efficiency-patterns.md`). Emi...

Details

Author
hatch3r
Repository
hatch3r/hatch3r
Created
5 months ago
Last Updated
1 weeks ago
Language
TypeScript
License
MIT

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Solid

hatch3r-feature

End-to-end feature implementation workflow. Covers data model, domain logic, API, and UI as a vertical slice. Use when implementing new features or working on feature request issues.

26 Updated 1 weeks ago
hatch3r
AI & Automation Solid

ai-feature-eval-harness

Design an evaluation plan for a product AI feature (LLM- or model-backed output): measurable success criteria, a held-out labeled eval dataset shape, per-criterion grading (code-based first, then LLM-based for nuanced judgment), and a pass threshold, then persist as AI_EVAL_PLAN.md. Use when the task ships or changes a feature whose output is model-generated or non-deterministic (assistant reply, classification, extraction, summarization, ranking, agent action) and needs a repeatable dataset-backed eval rather than only example-based tests. Do not use when the feature has no model-backed output (use test-strategy for deterministic behavior), when judging Fhorja's own command outputs against a rubric (use verify-against-rubric), or when no active task folder exists. The code-graded tier composes with ADR-0048 (a passing deterministic gate is Layer-1 evidence); the LLM-graded tier is added signal, not a replacement.

6 Updated 5 days ago
Mozurok
AI & Automation Listed

ai-eval-harness

Designs and runs evaluation harnesses for AI features and for agent skills themselves. Use when shipping LLM features or measuring skill quality. Emits EVAL_PLAN / EVAL_REPORT. Never claims production quality from a handful of vibes-only prompts.

0 Updated 1 months ago
willianbs