nlp-and-llmlisted
Install: claude install-skill TranBaDat2607/claude-code-kaggle-grandmaster
# NLP & LLM Playbook
## 1. Pick the paradigm
| Problem | Strong default |
|---|---|
| Text classification/regression, ≤ ~1k tokens, 1k–100k samples | DeBERTa-v3 (base/large) fine-tune, 5 folds |
| Token classification (NER, PII, span extraction) | DeBERTa-v3 token classification, long context via striding; threshold tuning |
| Long documents | Longformer/BigBird, or chunk + aggregate; or decoder LLMs with long context |
| Preference / pairwise judging, nuanced semantics | Fine-tune a 7–14B decoder LLM (Gemma/Qwen/Llama/Mistral family) with LoRA as a classifier/regressor head |
| Reasoning / math / code generation | Strongest open reasoning LLM that fits the hardware; prompt engineering, self-consistency (majority voting), tool use (Python execution), test-time compute scaling, fine-tuning on curated solutions (SFT, then RL such as GRPO when feasible) |
| Retrieval / matching | bi-encoder (e5/bge/gte) recall → cross-encoder or LLM re-rank |
| Generation evaluated by metric (e.g. ROUGE, similarity) | fine-tuned seq2seq/decoder + decoding search tuned to the metric |
Always check the rules about which pretrained models/APIs are allowed and model licences.
## 2. Encoder fine-tuning recipe (DeBERTa-v3)
- max_len from token-length percentiles (cover 95–99%); tokenise once; dynamic padding with length-bucketed
batches (the template's default), which removes most padding compute.
- lr 1e-5–3e-5 (large: 1e-5), layer-wise LR decay 0.8–0.95, warmup 5–10%, cosine/linear decay,
2