nemo-mbridge-perf-moe-vlm-traininglisted

# MoE VLM Training Stable docs: @docs/training/moe-optimization.md Card: @skills/nemo-mbridge-perf-moe-vlm-training/card.yaml ## FSDP vs 3D Parallel | Approach | Strength | Best fit | |---|---|---| | FSDP | Simplest path to a working multimodal run | first bring-up, memory-first tuning, awkward PP boundaries | | 3D parallel | Higher ceiling after tuning | stable models with a clean PP layout and time for deeper sweeps | For MoE VLMs, the practical workflow is usually: 1. get the first reliable run with FSDP 2. stabilize real-data input, recompute, and memory behavior 3. move to 3D parallel only if the throughput headroom is worth the extra work ## Rounded Findings From Recent VLM Runs ### Qwen3-VL class models The main patterns were consistent across the tracker: - FSDP on GB200-class systems can already reach healthy high-teens utilization with a comparatively simple setup - B200 FSDP runs are viable, but more sensitive to recompute choice and frozen vision settings - 3D parallel can recover to a similar or better operating point, but only after tuning MBS, recompute, and the real vision path together ### Real data vs mock data Mock-data VLM runs are not trustworthy performance proxies. In the experiments, image-free mock runs looked closer to "roughly twice as fast" than "slightly optimistic" when compared with real multimodal input. Use real or realistic image payloads before drawing any conclusion about VLM throughput. ### Smaller multimodal MoE runs T