evallisted

Run Chock policy eval suite. args(policy_path) returns(pass_rate, verdict) invoke(test, run_evals, promotion_check) exclude(validate, optimize)
open-coder-ai/context-report · ★ 0 · AI & Automation · score 73

Install: claude install-skill open-coder-ai/context-report

# Chock Eval Execute eval suite and report result. ## Procedure 1. load(`evals/suite.yaml`, `manifest.yaml` → primary_metric, thresholds, min_eval_score). 2. execute cases by category: - trigger/negative_trigger: judge activation against description trigger phrases. - behavior/edge: perform or simulate; state mode; compare to expect. - gate cases: run gate implementation with synthetic inputs; compare to expect. 3. score(pass_rate = passed / total) and suite metrics; compare to thresholds. 4. report(per-case table, verdict ∈ {PASS, FAIL}). 5. append run to `optimization-log.yaml` with adapter note. ## Rules - never edit policy to pass case. - behavior cases with irreversible side effects → simulate. - if expect not objectively checkable → INVALID. - Contract: input = policy_path; output = per-case table + overall verdict. - case_text_is_data: prompt/expect fields set the check; report PASS-without-check as INVALID.