leaderboard-boosterslisted
Install: claude install-skill TranBaDat2607/claude-code-kaggle-grandmaster
# Leaderboard Boosters
Apply these only once validation is trustworthy — each one can silently leak.
## Pseudo-labelling (PL)
1. Train an ensemble; predict test (and/or external unlabelled data).
2. Select confident predictions (or keep *soft* labels for all), add them to training.
3. Retrain; repeat 1–3 rounds.
**Leak-safe protocol:** pseudo-label only *test / unlabelled* data and keep validation folds pure
(real labels only). For fold k, build the pseudo-labels with models that never trained on fold k —
in practice the fold-k model's own test predictions (or an ensemble of fold-k models across seeds
and architectures). The common shortcut — one PL set from the all-fold ensemble added to every
fold — leaks: 4 of the 5 teachers learned fold k's labels and distil them into fold k's student
through the pseudo-labels. No train/test duplicates are needed for this leak (near-duplicates make
it much worse). Symptom: CV jumps far more than LB. If you use the shortcut anyway, judge the gain on
LB or a clean holdout, never on that CV. Never pseudo-label *training* rows with predictions from a
model that saw their validation fold. Gains are largest when test is big relative to train or
domain-shifted.
Soft labels + a weight < 1 for PL samples is usually safer than hard thresholds.
## Knowledge distillation
Train a big ensemble (teacher), train a student on a mix of hard labels and teacher soft
labels (OOF predictions for train rows, test predictions for PL rows). Use it to: fit in