databricks-reference-architecture

# Databricks Reference Architecture ## Overview Production-ready lakehouse architecture with Unity Catalog, Delta Lake, and the medallion pattern. Covers workspace organization, three-level namespace governance, compute strategy, CI/CD with Asset Bundles, and project structure for team collaboration. ## Prerequisites - Databricks workspace with Unity Catalog enabled - Understanding of medallion architecture (bronze/silver/gold) - Databricks CLI configured - Terraform or Asset Bundles for infrastructure ## Architecture ``` ┌─────────────────────────────────────────────────────────────────┐ │ UNITY CATALOG │ │ │ │ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌───────────┐ │ │ │ Bronze │ │ Silver │ │ Gold │ │ ML Models │ │ │ │ Catalog │─▶│ Catalog │─▶│ Catalog │ │ (MLflow) │ │ │ │ (raw) │ │ (clean) │ │ (curated) │ │ │ │ │ └────────────┘ └────────────┘ └────────────┘ └───────────┘ │ │ ▲ │ │ │ ┌────────────┐ ┌────────────────┐ │ │ │ Auto Loader│ │ Model Serving │ │ │ │ Ingestion │ │ Endpoints │ │ │ └────────────┘ └────────────────┘ │ ├─────────────────────────────────────────────────────────────────┤ │ Com...