consistency-audit

Featured

Flagship intra-paper self-consistency forensics: does the paper contradict ITSELF across abstract/intro/tables/body/appendix, and does the method DESCRIBED match the method EVALUATED? Needs no external ground truth — works PDF-only (L0). Runs a deterministic arithmetic pass + a fresh cross-model semantic pass, every finding span-anchored to the evidence ledger (claims.json), reviewer≠adjudicator. Emits consistency-audit.findings.json; NEVER computes the verdict. Triggers: "consistency audit", "check the paper against itself", "self-consistency", "内部自洽".

AI & Automation 139 stars 8 forks Updated 3 days ago MIT

Install

View on GitHub

Quality Score: 89/100

Stars 20%
71
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Consistency Audit — the paper vs itself Audit intra-paper self-consistency for: **$ARGUMENTS** (requires `claims.json` from `/evidence-ledger`). Emit span-anchored `consistency-audit.findings.json`; the deterministic adjudicator — not this skill — computes the verdict. > 🔒 **Do not wrap this skill in `/loop`, `/schedule`, or `CronCreate`.** It is > verdict-bearing input — it proposes the findings the deterministic adjudicator > turns into the report. Re-firing it on a wall-clock timer adds no signal: its > output changes only when the **paper / ledger** changes, not with the clock. > Schedule the *external wait that precedes it* — ledger built → audit **once**. > (Mirrors ARIS's external-cadence doctrine.) > The flagship instrument. Internal contradiction is the single most defensible > thing you can check on an unknown submission: it needs no external GT, runs at L0 > (PDF-only), and is exactly where machine-generated papers crack — they hallucinate > *local* coherence. Recall scales with the ledger: a PDF-text (L0) ledger extracts > only number/scope spans, so the table/caption/method-drift checks gain teeth at L1 > (LaTeX), where tables and captions are actually extracted. Adapted from ARIS > `paper-claim-audit`, reframed from "paper vs result files" to **"paper vs > itself."** There is no external ground truth in this skill. ## Why this exists An autoresearch pipeline (or rushed human) writes the abstract, the tables, the method section, and the appendix in separa...

Details

Author
wanshuiyin
Repository
wanshuiyin/Anti-Autoresearch
Created
1 months ago
Last Updated
3 days ago
Language
Python
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

paper-claim-audit

Zero-context verification that every number, comparison, and scope claim in the paper matches raw result files. Uses a fresh cross-model reviewer with NO prior context to prevent confirmation bias. Use when user says "审查论文数据", "check paper claims", "verify numbers", "论文数字核对", or before submission to ensure paper-to-evidence fidelity. Do not use for code/data/formula verification (use paper-verification).

2 Updated 3 weeks ago
wookat
AI & Automation Featured

paper-claim-audit

Zero-context verification that every number, comparison, and scope claim in the paper matches raw result files. Uses a fresh cross-model reviewer with NO prior context to prevent confirmation bias. Use when user says "审查论文数据", "check paper claims", "verify numbers", "论文数字核对", or before submission to ensure paper-to-evidence fidelity.

14,964 Updated today
wanshuiyin
AI & Automation Featured

evidence-ledger

Build the deterministic evidence ledger (artifact_manifest.json + claims.json) that every other Anti-Autoresearch auditor reads. One pass inventories artifacts, derives the observability level (L0 PDF-only / L1 +LaTeX / L2 +repo+results) by fixed rule, and extracts span-anchored, hashed, checkable claims (numbers, comparisons, scope, method, baselines, citations, captions, table cells) into claims.json. An OPTIONAL additive cross-model pass ADDS span-anchored semantic claims — method, theorem statements with their assumptions, definitions, proof/derivation steps and equations, scope, baselines, conclusions, the motivation span, and reproducibility-artifact references (the proof, derivation, and structure anchors the family B/D/G auditors need) — it never invents a number, emits a finding, or computes a verdict. Run FIRST, before any audit skill. Triggers: "build the ledger", "extract claims", "prep for integrity audit", "evidence ledger", "建证据账本".

139 Updated 3 days ago
wanshuiyin