← ClaudeAtlas

manage-paper-experimentslisted

Manage formal ML paper experiments from planning through reproducible reporting. Use when building or auditing the paper-wide experiment ledger, assigning paper experiment IDs, freezing code and protocols, scheduling GPU-node queues or waiters, generating send-and-forget remote Codex prompts, monitoring and reconciling W&B runs, retaining best/final checkpoints, or preparing paper curves and result tables.
vanzll/ai-research-accelerator · ★ 1 · AI & Automation · score 74
Install: claude install-skill vanzll/ai-research-accelerator
# Manage Paper Experiments Treat the experiment ledger as the single source of truth. Convert the paper plan into uniquely identified, reproducible runs; carry each run through scheduling, execution, W&B reconciliation, checkpoint retention, and paper reporting without silently changing the protocol. ## Start With Context 1. Read the applicable `AGENTS.md` before any local or remote action. 2. Locate and read the paper `plan.md`, experiment ledger, repository progress document, and relevant launcher/config files. 3. Read [ledger-schema.md](references/ledger-schema.md) before changing experiment state. 4. Read [queue-patterns.md](references/queue-patterns.md) before producing remote prompts or launch commands. 5. Read [reporting-protocol.md](references/reporting-protocol.md) before comparing runs or filling paper tables. 6. When installed, use `wandb-query` for W&B config, summary, history, state, and workspace inspection. Otherwise use an authorized API/export and record the evidence gap. Do not infer results from a run name or reward plot alone. If the plan and ledger disagree, stop scheduling affected experiments and surface the conflict. Do not invent extra seeds, ablations, evaluators, or method changes. ## Preserve Two Evidence Surfaces Always track these separately: - **Training dynamics:** speed, stability, threshold-crossing steps, peak, final, tail mean, drawdown, reward calls, and GPU-hours. - **Held-out evaluation:** pre-registered evaluation steps, best eva