template-reproducibility-audit

Solid

Deterministic reproducibility audit — fixed seeds, regenerate-from-clean, double-run diff before Zenodo/arXiv/release. USE WHEN outputs drift between runs, "worked on my machine", need regenerate-from-clean proof, or pre-release reproducibility check — even without naming docs/prompts.

AI & Automation 18 stars 1 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 84/100

Stars 20%
43
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Reproducibility audit Complements [manuscript-claim-verification](../manuscript-claim-verification/SKILL.md) (claim truth) by focusing on **stability of artifacts**. ## Natural invoke - "Prove template_code_project is reproducible before Zenodo" - "Numbers in the PDF don't match after a clean rebuild" - "Double-run the pipeline and diff outputs" ## Inputs to confirm - **Project** — from [`docs/_generated/active_projects.md`](../../_generated/active_projects.md). ## Workflow 1. **Determinism** — fixed RNG seeds, `MPLBACKEND=Agg`; no wall-clock/hostname/path leaks. List nondeterministic sources. 2. **Regenerate from clean** — wipe working outputs, run core pipeline, regenerate manuscript variables. Capture exit status per stage. 3. **Diff** — compare regenerated `output/<name>/` and manuscript variables vs prose assertions. Hand-typed numbers not from generated variables are findings even if equal. 4. **Double-run stability** — regenerate twice with clean tree between; any run-1 vs run-2 diff is hard failure. 5. **Fix** — seed injection, variable-ize hard-typed numbers, remove timestamp leakage. Update `projects/<n>/AGENTS.md` and README.md with regeneration command. Never hand-edit `output/`. ## Deliverables - Drift table: artifact | committed | regenerated | action. - Commands + raw output; no invented coverage numbers. ## Verification commands ```bash uv sync uv run python scripts/runner/execute_pipeline.py --project <project> --core-only uv run python proje...

Details

Author
docxology
Repository
docxology/template
Created
11 months ago
Last Updated
today
Language
Python
License
Apache-2.0

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category