← ClaudeAtlas

nlp-and-llmlisted

Winning playbook for Kaggle NLP and LLM competitions — transformer fine-tuning (DeBERTa-v3 and friends), LLM fine-tuning with LoRA/QLoRA, LLM-as-solver (math/reasoning/agents), retrieval & RAG, synthetic data generation, distillation, efficient inference with vLLM under Kaggle time limits, and text-specific validation. Use for any text, chat, code, or LLM-related competition.
TranBaDat2607/claude-code-kaggle-grandmaster · ★ 2 · AI & Automation · score 75
Install: claude install-skill TranBaDat2607/claude-code-kaggle-grandmaster
# NLP & LLM Playbook ## 1. Pick the paradigm | Problem | Strong default | |---|---| | Text classification/regression, ≤ ~1k tokens, 1k–100k samples | DeBERTa-v3 (base/large) fine-tune, 5 folds | | Token classification (NER, PII, span extraction) | DeBERTa-v3 token classification, long context via striding; threshold tuning | | Long documents | Longformer/BigBird, or chunk + aggregate; or decoder LLMs with long context | | Preference / pairwise judging, nuanced semantics | Fine-tune a 7–14B decoder LLM (Gemma/Qwen/Llama/Mistral family) with LoRA as a classifier/regressor head | | Reasoning / math / code generation | Strongest open reasoning LLM that fits the hardware; prompt engineering, self-consistency (majority voting), tool use (Python execution), test-time compute scaling, fine-tuning on curated solutions (SFT, then RL such as GRPO when feasible) | | Retrieval / matching | bi-encoder (e5/bge/gte) recall → cross-encoder or LLM re-rank | | Generation evaluated by metric (e.g. ROUGE, similarity) | fine-tuned seq2seq/decoder + decoding search tuned to the metric | Always check the rules about which pretrained models/APIs are allowed and model licences. ## 2. Encoder fine-tuning recipe (DeBERTa-v3) - max_len from token-length percentiles (cover 95–99%); tokenise once; dynamic padding with length-bucketed batches (the template's default), which removes most padding compute. - lr 1e-5–3e-5 (large: 1e-5), layer-wise LR decay 0.8–0.95, warmup 5–10%, cosine/linear decay, 2