← ClaudeAtlas

leaderboard-boosterslisted

Advanced techniques that move a solid solution into the medal zone — pseudo-labelling, knowledge distillation, test-time augmentation, external data and pretraining, data cleaning/relabelling, multi-stage pipelines, auxiliary targets, leak exploitation, and full-data retraining. Use when a strong baseline exists and the user wants extra leaderboard gains.
TranBaDat2607/claude-code-kaggle-grandmaster · ★ 2 · AI & Automation · score 75
Install: claude install-skill TranBaDat2607/claude-code-kaggle-grandmaster
# Leaderboard Boosters Apply these only once validation is trustworthy — each one can silently leak. ## Pseudo-labelling (PL) 1. Train an ensemble; predict test (and/or external unlabelled data). 2. Select confident predictions (or keep *soft* labels for all), add them to training. 3. Retrain; repeat 1–3 rounds. **Leak-safe protocol:** pseudo-label only *test / unlabelled* data and keep validation folds pure (real labels only). For fold k, build the pseudo-labels with models that never trained on fold k — in practice the fold-k model's own test predictions (or an ensemble of fold-k models across seeds and architectures). The common shortcut — one PL set from the all-fold ensemble added to every fold — leaks: 4 of the 5 teachers learned fold k's labels and distil them into fold k's student through the pseudo-labels. No train/test duplicates are needed for this leak (near-duplicates make it much worse). Symptom: CV jumps far more than LB. If you use the shortcut anyway, judge the gain on LB or a clean holdout, never on that CV. Never pseudo-label *training* rows with predictions from a model that saw their validation fold. Gains are largest when test is big relative to train or domain-shifted. Soft labels + a weight < 1 for PL samples is usually safer than hard thresholds. ## Knowledge distillation Train a big ensemble (teacher), train a student on a mix of hard labels and teacher soft labels (OOF predictions for train rows, test predictions for PL rows). Use it to: fit in