final-submission-selectionlisted
Install: claude install-skill TranBaDat2607/claude-code-kaggle-grandmaster
# Final Submission Selection
You usually get **2 final picks** (check the rules; some comps allow more or auto-select). The goal
is the best *private* score, which is a different sample (and sometimes a different time period)
than the public LB.
## 1. Gather evidence
`python -m kgkit ledger best -n 20` and `python -m kgkit status`. For each candidate: CV mean/std,
public LB, CV-LB agreement, number of models, use of risky elements (leaks, PL, LB-tuned weights,
aggressive post-processing), and its correlation with other candidates.
## 2. How much to trust public LB
Weight public LB by its statistical reliability:
- Public set small (a few thousand rows or fewer), noisy metric, or adversarial AUC high → trust CV.
- Public set large and drawn like private, CV-LB correlation strong → LB and CV mostly agree; use both.
- Private is a *future* period → favour robustness (simpler, ensemble-heavy, less feature drift).
Formalise per pair of candidates: `python -m kgkit ledger compare <a> <b> --public-n <rows>` gives the
paired SE of their gap at the public-set size (both are scored on the same public rows, so this is far
smaller than the SE of one score); treat LB gaps below ~2 paired SE as ties broken by CV. The SE of one
absolute score (`kgkit ledger noise <id> --public-n <rows>`) is for judging one candidate's CV-LB gap,
not the gap between two candidates. See `cv-lb-debugging`. A pair that includes an LB-only candidate has no
OOF: the same command then reads the two submission