ml-training-recipes

# ML Training Recipes Battle-tested patterns for PyTorch training across domains. Drawn from production codebases (Karpathy's autoresearch/nanochat, torchvision, HuggingFace) and modern training practice. ## Reference files (read when needed) - `references/architecture.md` — Transformer/LLM architecture code patterns, weight init - `references/optimizers.md` — Muon, AdamW hybrid, per-group LR, compiled optimizer steps - `references/domain-specific.md` — Vision, diffusion, contrastive, distributed, checkpointing, data loading - `references/scaling-and-selection.md` — Scaling laws, compute budget tables, decision trees, DGX Spark - `references/biomedical.md` — Drug discovery, protein models, medical imaging, genomics, clinical NLP - `references/experiment-loop.md` — Autonomous experiment loop (autoresearch keep/discard/revert) --- ## Architecture Selection Pick the right model by **data type** and **data scale**: | Data Type | < 10K samples | 10K-100K | > 100K | |-----------|--------------|----------|--------| | **Images** | Pretrained CNN + fine-tune | Fine-tune ViT or CNN | ViT from scratch | | **Text (gen)** | Few-shot prompting | Fine-tune GPT/LLaMA (LoRA) | Pretrain from scratch | | **Tabular** | XGBoost/LightGBM | Still XGBoost | Neural viable | | **Audio** | Pretrained Whisper | Fine-tune AST | Train from scratch | | **Molecules** | Pretrained GNN | Fine-tune molecular LM | Train GNN from scratch | | **Proteins** | ESM-2 embeddings + hea...