← ClaudeAtlas

hyperparameter-tuninglisted

Efficient hyperparameter optimisation for Kaggle — when tuning is worth it, Optuna recipes with pruning on frozen folds, search spaces for LightGBM/XGBoost/CatBoost/NNs/transformers, budget allocation, and avoiding CV overfitting from excessive search. Use when the user wants to tune models or asks for an Optuna setup.
TranBaDat2607/claude-code-kaggle-grandmaster · ★ 2 · AI & Automation · score 75
Install: claude install-skill TranBaDat2607/claude-code-kaggle-grandmaster
# Hyperparameter Tuning **Priority:** validation > data/features > model family > loss/augmentation > hyperparameters. Tune *after* the feature set stabilises, and spend tuning budget where gains are measurable (beyond paired noise: `kgkit ledger compare` the tuned config against the accepted baseline). ## Optuna recipe (GBDT) ```python import lightgbm as lgb import numpy as np import optuna import pandas as pd from kgkit.cv import split_indices from kgkit import metrics as M metric = M.get("auc") folds = pd.read_csv("data/folds.csv")["fold"].to_numpy() def objective(trial): params = dict( objective="binary", learning_rate=0.05, n_estimators=10000, verbose=-1, num_leaves=trial.suggest_int("num_leaves", 15, 255, log=True), min_child_samples=trial.suggest_int("min_child_samples", 10, 300, log=True), colsample_bytree=trial.suggest_float("colsample_bytree", 0.2, 1.0), subsample=trial.suggest_float("subsample", 0.5, 1.0), subsample_freq=1, reg_alpha=trial.suggest_float("reg_alpha", 1e-3, 10, log=True), reg_lambda=trial.suggest_float("reg_lambda", 1e-3, 30, log=True), min_split_gain=trial.suggest_float("min_split_gain", 0, 1), ) scores = [] for k, (f, tr, va) in enumerate(split_indices(folds)): m = lgb.LGBMClassifier(**params).fit(X.iloc[tr], y[tr], eval_set=[(X.iloc[va], y[va])], callbacks=[lgb.early_stopping(200, verbose=False)]) score