← ClaudeAtlas

final-submission-selectionlisted

Choosing the final Kaggle submissions to maximise expected private leaderboard score and survive shake-ups — CV vs public LB weighting, hedging strategies, robustness checks, and endgame timeline. Use in the last days of a competition or when the user asks which submissions to select.
TranBaDat2607/claude-code-kaggle-grandmaster · ★ 2 · AI & Automation · score 75
Install: claude install-skill TranBaDat2607/claude-code-kaggle-grandmaster
# Final Submission Selection You usually get **2 final picks** (check the rules; some comps allow more or auto-select). The goal is the best *private* score, which is a different sample (and sometimes a different time period) than the public LB. ## 1. Gather evidence `python -m kgkit ledger best -n 20` and `python -m kgkit status`. For each candidate: CV mean/std, public LB, CV-LB agreement, number of models, use of risky elements (leaks, PL, LB-tuned weights, aggressive post-processing), and its correlation with other candidates. ## 2. How much to trust public LB Weight public LB by its statistical reliability: - Public set small (a few thousand rows or fewer), noisy metric, or adversarial AUC high → trust CV. - Public set large and drawn like private, CV-LB correlation strong → LB and CV mostly agree; use both. - Private is a *future* period → favour robustness (simpler, ensemble-heavy, less feature drift). Formalise per pair of candidates: `python -m kgkit ledger compare <a> <b> --public-n <rows>` gives the paired SE of their gap at the public-set size (both are scored on the same public rows, so this is far smaller than the SE of one score); treat LB gaps below ~2 paired SE as ties broken by CV. The SE of one absolute score (`kgkit ledger noise <id> --public-n <rows>`) is for judging one candidate's CV-LB gap, not the gap between two candidates. See `cv-lb-debugging`. A pair that includes an LB-only candidate has no OOF: the same command then reads the two submission