evidence-ledger

Featured

Build the deterministic evidence ledger (artifact_manifest.json + claims.json) that every other Anti-Autoresearch auditor reads. One pass inventories artifacts, derives the observability level (L0 PDF-only / L1 +LaTeX / L2 +repo+results) by fixed rule, and extracts span-anchored, hashed, checkable claims (numbers, comparisons, scope, method, baselines, citations, captions, table cells) into claims.json. An OPTIONAL additive cross-model pass ADDS span-anchored semantic claims — method, theorem statements with their assumptions, definitions, proof/derivation steps and equations, scope, baselines, conclusions, the motivation span, and reproducibility-artifact references (the proof, derivation, and structure anchors the family B/D/G auditors need) — it never invents a number, emits a finding, or computes a verdict. Run FIRST, before any audit skill. Triggers: "build the ledger", "extract claims", "prep for integrity audit", "evidence ledger", "建证据账本".

AI & Automation 139 stars 8 forks Updated 3 days ago MIT

Install

View on GitHub

Quality Score: 89/100

Stars 20%
71
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Evidence Ledger — the spine every auditor reads > Infrastructure skill, **not an auditor**. It produces the *only* structure the > auditor skills are allowed to reason over, so they don't each re-read the PDF and > hallucinate a different table and a different list of numbers. It emits **no > findings and no verdict** — only `artifact_manifest.json` + `claims.json`. See > `references/integrity-forensics-contract.md` §"The pipeline" (stages [1]–[2]). Build the ledger for: **$ARGUMENTS** > 🔁 **Not verdict-bearing — but not a polling skill either.** The deterministic > backbone (Steps 1–2) is a *pure function of the hashed sources*: same source bytes > → byte-identical `claims.json`. Re-run it only when the sources change, never on a > wall-clock timer. The only non-deterministic part is the optional enrichment pass > (Step 3), which is additive and skippable. Do **not** wrap this skill in `/loop` / > `/schedule` / `CronCreate`; there is no verdict to re-fire and no external event to > wait on. ## Why this exists Five language-model auditors each independently parsing a PDF = five different hallucinated tables and five different number lists, none reproducible — and the obvious dismissal, *"an LLM grading another LLM's paper is just slop."* The structural answer is **one deterministic pass** that turns the paper into: - `artifact_manifest.json` — what was observable; this fixes the **observability level L**, the ceiling on every downstream finding's severity, and - `c...

Details

Author
wanshuiyin
Repository
wanshuiyin/Anti-Autoresearch
Created
1 months ago
Last Updated
3 days ago
Language
Python
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Featured

consistency-audit

Flagship intra-paper self-consistency forensics: does the paper contradict ITSELF across abstract/intro/tables/body/appendix, and does the method DESCRIBED match the method EVALUATED? Needs no external ground truth — works PDF-only (L0). Runs a deterministic arithmetic pass + a fresh cross-model semantic pass, every finding span-anchored to the evidence ledger (claims.json), reviewer≠adjudicator. Emits consistency-audit.findings.json; NEVER computes the verdict. Triggers: "consistency audit", "check the paper against itself", "self-consistency", "内部自洽".

139 Updated 3 days ago
wanshuiyin
AI & Automation Featured

experiment-forensics

Audit experiment integrity against the evidence ledger. At L2 (repo + result files present) a fresh cross-model reviewer reads the eval code line-by-line for fake/derived ground truth, score self-normalization, phantom results (a paper number with no backing file/key), dead/uncalled metric code, verified-scope inflation, method-described ≠ method-evaluated drift, synthesized-looking results, placeholder/fake data still wired into a released result, code-output ≠ reported-number mismatch, and missing reproducibility artifacts (an empirical/agent/LLM paper shipping neither code nor the prompts/configs its results need) — every finding span-anchored to a ledger claim_id. At L0/L1 (PDF / source only) the same patterns are surfaced as info-level 'could-not-verify' signals where the ledger gives an anchor (observability_level_required:2) — NEVER a fraud verdict from a PDF. The reviewer PROPOSES findings; tools/adjudicate_findings.py computes the verdict. Detect-only. Triggers: "experiment forensics", "audit the res

139 Updated 3 days ago
wanshuiyin
AI & Automation Featured

citation-forensics

Citation-integrity forensics: is every reference real, correctly attributed, and used in a context the cited work actually supports? Catches hallucinated references (no paper at the claimed arXiv id/DOI/venue, fabricated authors/year), metadata drift (wrong year/venue/version), and wrong-context citations (a real paper cited for a claim it never makes — or argues against). A hot zone for machine-generated papers. Decidable at L0 (text + canonical sources). Span-anchored to the evidence ledger (claims.json); the executor gathers canonical facts (DBLP / arXiv / DOI), then one FRESH cross-model thread per cited key proposes findings; reviewer != adjudicator. Emits citation-forensics.findings.json; NEVER computes the verdict. Triggers: "citation forensics", "check the references", "hallucinated citations", "wrong-context citation", "verify references", "引用核对".

139 Updated 3 days ago
wanshuiyin