experiment-audit
FeaturedAudit experiment integrity before claiming results. Uses cross-model review (external reviewer backend) to check for fake ground truth, score normalization fraud, phantom results, and insufficient scope. Use when user says "审计实验", "check experiment integrity", "audit results", "实验诚实度", or after experiments complete before writing claims.
Install
Quality Score: 96/100
Skill Content
Details
- Author
- wanshuiyin
- Repository
- wanshuiyin/Auto-claude-code-research-in-sleep
- Created
- 5 months ago
- Last Updated
- today
- Language
- Python
- License
- MIT
Integrates with
Similar Skills
Semantically similar based on skill content — not just same category
experiment-forensics
Audit experiment integrity against the evidence ledger. At L2 (repo + result files present) a fresh cross-model reviewer reads the eval code line-by-line for fake/derived ground truth, score self-normalization, phantom results (a paper number with no backing file/key), dead/uncalled metric code, verified-scope inflation, method-described ≠ method-evaluated drift, synthesized-looking results, placeholder/fake data still wired into a released result, code-output ≠ reported-number mismatch, and missing reproducibility artifacts (an empirical/agent/LLM paper shipping neither code nor the prompts/configs its results need) — every finding span-anchored to a ledger claim_id. At L0/L1 (PDF / source only) the same patterns are surfaced as info-level 'could-not-verify' signals where the ledger gives an anchor (observability_level_required:2) — NEVER a fraud verdict from a PDF. The reviewer PROPOSES findings; tools/adjudicate_findings.py computes the verdict. Detect-only. Triggers: "experiment forensics", "audit the res
experiment-audit
Use this skill for scientific and ML-research reasoning work — evaluating experimental claims, auditing training runs or ablations, checking whether a statistical claim holds up, assessing reproducibility, reconciling contradictory results, reviewing a paper's methodology or results section, writing reviewer-style feedback, or producing a structured research report. This is a scientific reasoning discipline, not just a tool wrapper, so it applies even with no live data source — e.g. reviewing a pasted table of results, sanity-checking a claimed effect size, or evaluating an ablation described in prose. Trigger on phrasing like "did I mess up this experiment," "is this result real," "why did my loss/reward do X," "which run is better," "is this ablation confounded," "review this paper's claims," "write reviewer feedback," "is this reproducible," "what should I conclude from this," or "write up these results." When the user's data lives in Weights & Biases, this skill also covers the experiment-audit-mcp integr
paper-claim-audit
Zero-context verification that every number, comparison, and scope claim in the paper matches raw result files. Uses a fresh cross-model reviewer with NO prior context to prevent confirmation bias. Use when user says "审查论文数据", "check paper claims", "verify numbers", "论文数字核对", or before submission to ensure paper-to-evidence fidelity. Do not use for code/data/formula verification (use paper-verification).