modal-serverless-gpu

# Modal Serverless GPU Comprehensive guide to running ML workloads on Modal's serverless GPU cloud platform. ## When to use Modal **Use Modal when:** - Running GPU-intensive ML workloads without managing infrastructure - Deploying ML models as auto-scaling APIs - Running batch processing jobs (training, inference, data processing) - Need pay-per-second GPU pricing without idle costs - Prototyping ML applications quickly - Running scheduled jobs (cron-like workloads) **Key features:** - **Serverless GPUs**: T4, L4, A10G, L40S, A100, H100, H200, B200 on-demand - **Python-native**: Define infrastructure in Python code, no YAML - **Auto-scaling**: Scale to zero, scale to 100+ GPUs instantly - **Sub-second cold starts**: Rust-based infrastructure for fast container launches - **Container caching**: Image layers cached for rapid iteration - **Web endpoints**: Deploy functions as REST APIs with zero-downtime updates **Use alternatives instead:** - **RunPod**: For longer-running pods with persistent state - **Lambda Labs**: For reserved GPU instances - **SkyPilot**: For multi-cloud orchestration and cost optimization - **Kubernetes**: For complex multi-service architectures ## Quick start ### Installation ```bash pip install modal modal setup # Opens browser for authentication ``` ### Hello World with GPU ```python import modal app = modal.App("hello-gpu") @app.function(gpu="T4") def gpu_info(): import subprocess return subprocess.run(["nvidia-smi"], capture_outp...