← ClaudeAtlas

kg-experimentlisted

Run one disciplined experiment for a hypothesis — single change, frozen folds, same seeds, ledger logging with its parent baseline, paired comparison (per-fold deltas + OOF bootstrap) against the accepted baseline, recorded keep/discard decision.
TranBaDat2607/claude-code-kaggle-grandmaster · ★ 2 · AI & Automation · score 78
Install: claude install-skill TranBaDat2607/claude-code-kaggle-grandmaster
# /kg-experiment Run kgkit with the Bash tool, as `PYTHONPATH="${CLAUDE_PLUGIN_ROOT}" python -m kgkit <command>`: this command pre-approves that form for the kgkit calls it makes. Submitting, pushing or publishing still asks (so does a PowerShell `$env:` assignment). Hypothesis: `$ARGUMENTS` Follow the experiment protocol (`grandmaster-playbook` §2): 1. `python -m kgkit status` and `python -m kgkit ledger baseline` → the **accepted baseline** id (last KEEP, not the highest CV), its CV ± std and code path. If nothing is accepted yet, decide one first (`python -m kgkit ledger decide <id> baseline`). If the idea is in the backlog, mark it `python -m kgkit backlog set <n> --status running --first-experiment "<command you will run>"`; otherwise add it (same options). 2. Restate the hypothesis, the *single* change, the baseline id, and the success criterion: paired gain ≥ ~2 standard errors of the per-fold differences (or of the OOF bootstrap) with most folds improving. Not "Δ > fold std": the fold std measures fold difficulty, which cancels in a paired comparison. 3. Implement behind a flag/config in the existing script, never in a copy of it (`train_v2.py`): the record keeps the command it ran, so the flag says what changed. A quick probe that is not the experiment goes in `work/`. Smoke test, then full run on frozen folds with the baseline's seed(s). When the change touches the input pipeline, the resolution / `max_len` or the model size, run th