ml-emerging-techniqueslisted
Install: claude install-skill Lu1sDV/skillsmd
# ML Emerging Techniques
Six specialized LLM scaling, compression, and acceleration workflows consolidated
into two focused references. Open only the reference the task needs; do not
preload both.
| Task / Tool | Read |
|---|---|
| MoE training & inference (Mixtral, DeepSeek-V3, Switch, DeepSpeed EP, load balancing) or long-context extension (RoPE, YaRN, ALiBi, position interpolation) | [Architecture scaling reference](references/architecture-scaling.md) |
| Knowledge distillation (MiniLLM, logit/response KD), pruning (Wanda, SparseGPT, N:M), model merging (`mergekit`, SLERP, TIES, DARE), or speculative decoding (draft, Medusa, Lookahead) | [Compression and merging reference](references/compression-and-merging.md) |
## What each covers
- **Architecture scaling** — sparse MoE architectures, top-k / shared-expert /
auxiliary-loss-free routing, DeepSpeed expert parallelism and capacity factors,
plus RoPE math, YaRN NTK-by-parts, ALiBi slopes, HuggingFace `rope_scaling`,
and long-context fine-tuning memory optimization.
- **Compression and merging** — teacher-student distillation (forward vs reverse
KLD), one-shot and N:M pruning with calibration data, `mergekit` YAML recipes
and unsupervised coefficient tuning (AdaMMS), and lossless inference speedup
via draft-model speculative decoding, Medusa, and Lookahead decoding.
## Universal rules
- Check VRAM, interconnect, and architecture compatibility before launching:
MoE capacity scales with total experts even wh