torchforge-rl-training

# torchforge: PyTorch-Native Agentic RL Library torchforge is Meta's PyTorch-native RL library that separates infrastructure concerns from algorithm concerns. It enables rapid RL research by letting you focus on algorithms while handling distributed training, inference, and weight sync automatically. ## When to Use torchforge **Choose torchforge when you need:** - Clean separation between RL algorithms and infrastructure - PyTorch-native abstractions (no Ray dependency) - Easy algorithm experimentation (GRPO, DAPO, SAPO in ~100 lines) - Scalable training with Monarch actor system - Integration with TorchTitan for model parallelism **Consider alternatives when:** - You need production-ready stability → use **miles** or **verl** - You want Megatron-native training → use **slime** - torchforge is experimental and APIs may change ## Key Features - **Algorithm isolation**: Implement RL algorithms without touching infrastructure - **Scalability**: From single GPU to thousands via Monarch - **Modern stack**: TorchTitan (training), vLLM (inference), TorchStore (sync) - **Loss functions**: GRPO, DAPO, CISPO, GSPO, SAPO built-in ## Architecture Overview ``` ┌─────────────────────────────────────────────────────────┐ │ Application Layer (Your Code) │ │ - Define reward models, loss functions, sampling │ └─────────────────────┬───────────────────────────────────┘ │ ┌─────────────────────▼───────────────────────────────────┐ │ ...