vastai-reference-architecture

# Vast.ai Reference Architecture ## Overview Production architecture for GPU compute workflows on Vast.ai. Covers the three-tier pattern (orchestrator, GPU workers, artifact storage), job queue design, and fault-tolerant training pipelines. ## Prerequisites - Vast.ai account with CLI - Cloud storage (S3, GCS, or MinIO) for artifacts - Understanding of ML training pipelines ## Instructions ### Architecture: Three-Tier GPU Compute ``` ┌─────────────────────────────────────────────────┐ │ ORCHESTRATOR (your server / CI / cloud function) │ │ - Job queue management │ │ - Instance provisioning via Vast.ai API │ │ - Status monitoring and auto-recovery │ │ - Cost tracking and budget enforcement │ └───────────────┬─────────────────────────────────┘ │ Vast.ai REST API ┌───────────────▼─────────────────────────────────┐ │ GPU WORKERS (Vast.ai rented instances) │ │ - Training / inference execution │ │ - Checkpoint saving to cloud storage │ │ - Health reporting back to orchestrator │ │ - Graceful shutdown on SIGTERM (spot preemption)│ └───────────────┬─────────────────────────────────┘ │ S3 / GCS / MinIO ┌───────────────▼─────────────────────────────────┐ │ ARTIFACT STORAGE (persistent) │ │ - Model checkpoints │ │ - Training logs and metrics │ │ - Dataset cache ...