modal-serverless-gpu

Featured

Serverless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scaling.

DevOps & Infrastructure 264,439 stars 59691 forks Updated today MIT

Install

View on GitHub

Quality Score: 93/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Modal Serverless GPU Comprehensive guide to running ML workloads on Modal's serverless GPU cloud platform. ## When to use Modal **Use Modal when:** - Running GPU-intensive ML workloads without managing infrastructure - Deploying ML models as auto-scaling APIs - Running batch processing jobs (training, inference, data processing) - Need pay-per-second GPU pricing without idle costs - Prototyping ML applications quickly - Running scheduled jobs (cron-like workloads) **Key features:** - **Serverless GPUs**: T4, L4, A10G, L40S, A100, H100, H200, B200 on-demand - **Python-native**: Define infrastructure in Python code, no YAML - **Auto-scaling**: Scale to zero, scale to 100+ GPUs instantly - **Sub-second cold starts**: Rust-based infrastructure for fast container launches - **Container caching**: Image layers cached for rapid iteration - **Web endpoints**: Deploy functions as REST APIs with zero-downtime updates **Use alternatives instead:** - **RunPod**: For longer-running pods with persistent state - **Lambda Labs**: For reserved GPU instances - **SkyPilot**: For multi-cloud orchestration and cost optimization - **Kubernetes**: For complex multi-service architectures ## Quick start ### Installation ```bash pip install modal modal setup # Opens browser for authentication ``` ### Hello World with GPU ```python import modal app = modal.App("hello-gpu") @app.function(gpu="T4") def gpu_info(): import subprocess return subprocess.run(["nvidia-smi"], capture_outp...

Details

Author
NousResearch
Repository
NousResearch/hermes-agent
Created
1 years ago
Last Updated
today
Language
Python
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category