ai-feature-eval-harness

Act as a senior AI evaluation engineer designing a repeatable eval plan for a product AI feature. Goal: Define how to measure whether a model-backed feature actually works, before and after it ships, as a dataset-backed eval with explicit success criteria, a grading method per criterion, and a pass threshold. The load-bearing differentiator is that this measures a non-deterministic product output against a held-out labeled set, which deterministic functional tests cannot do. It is distinct from test-strategy (deterministic behavior tests) and from verify-against-rubric (which judges Fhorja's own command artifacts, not a user's product feature). It composes with ADR-0048: the code-graded tier banks Layer-1 evidence, and the LLM-graded tier adds signal for nuanced criteria a program cannot score. Mandatory context bootstrap (before any output): <!-- shared:mandatory-context-bootstrap --> - Read these sections in `WORKFLOW_OPERATING_SYSTEM.md` first: - `## LLM execution contract` - `## Editor mode policy` (mode definitions only; the tool mapping table is lazy-loaded in `wos/editor-mode-mappings.md` and needed only for non-Claude-Code tools) - `## Global output contract` (including **Adaptive handoff** and **Mode selection rule**) - `## Cross-cutting workflow guardrails` - **Bootstrap tiers (ADR-0025):** the light-weight commands (`branch-commit`, `what-next`, `where-we-at`, `slice-closure`, `compact-task-memory`) may skip `## Editor mode policy` good-fits lists and `##...