evaluation-anchor-checker
SolidAudit and rewrite evaluation/numeric claims to ensure they carry minimal protocol context (task + metric + constraint) and avoid underspecified model naming. **Trigger**: evaluation anchor checker, numeric claim hygiene, underspecified numbers, protocol context, 评测锚点检查, 数字断言, 指标上下文.
Install
Quality Score: 81/100
Skill Content
Details
- Author
- WILLOSCAR
- Repository
- WILLOSCAR/research-units-pipeline-skills
- Created
- 7 months ago
- Last Updated
- today
- Language
- Python
- License
- None
Similar Skills
Semantically similar based on skill content — not just same category
double-check
Exhaustively validate every claim, number, calculation, code-fact, and command in a document or outbound communication. Extracts all claims into a prioritized ledger (flagging what is NOT covered), validates each by the right method (run the math, run the command, cite the file:line, quote the primary source), then calibrates certainty with CIA Words of Estimative Probability and a provenance tier — and proposes corrected, properly-hedged rewrites for any overclaim. Use before sending a message, filing an issue, committing analysis, or whenever you state exact numbers/assertions others will rely on.
ai-eval-review
Audit the evaluation layer of an AI product against design-completeness questions: offline criteria, ground-truth quality, online signal, cohort/disparate-impact, adversarial + robustness coverage, and drift detection. Eval-side companion to `ai-ux-review`. Produces an editable Markdown artifact plus a self-contained HTML report under `docs/ai-ux/`. Use when the user asks to "review my AI eval setup", "audit my eval design", "is my AI eval rigorous enough", "responsible-AI eval review", "fairness eval check", or "drift detection design", or ships an LLM/ML feature and wants an eval-rigor check. Invoke even if only one block is named — the others stress-test it. Does NOT trigger for: human-AI UX review (`ai-ux-review`); lean-canvas work (`validation-canvas`); adversarial pre-mortem with a verdict (`startup-grill`); SKILL.md audits (`skill-evaluator`); implementing eval pipelines, writing eval code, or labeling datasets (this skill names gaps, it does not build them).
eval-design-forensics
Audit whether a paper's EVALUATION DESIGN actually measures what it claims and whether its reporting is complete — the validity layer family D (experiment-forensics) cannot reach. Three patterns: train/test leakage means the reported score may not measure generalization (HP-EVAL-LEAKAGE — adopts the Kapoor & Narayanan 8-type / 3-category leakage taxonomy; the illegitimate-proxy / sampling-bias / pretraining-contamination subtypes hand off as needs_external_check, naming but NEVER running Oren-2023 exchangeability / Shi-2023 Min-K% / Golchin-2023 Time-Travel / BIG-bench canary); a load-bearing LLM judge is conflicted (same model/family as a compared system) or unvalidated (no human-agreement, no bias control) (HP-JUDGE-VALIDITY); a declared condition/metric is dropped or switched to favor the method, or 'best' is chosen with no held-out set (HP-SELECTIVE-REPORTING). Verdict-bearing at L0/L1 from the DESCRIBED protocol — NOT repo-gated like experiment-forensics; L2 only CONFIRMS against split/preprocessing/resu