data-pipeline-spec

# Data Pipeline Spec Skill This skill produces a complete data pipeline specification covering sources, transformations, destinations, scheduling, SLAs, error handling, data quality checks, and monitoring requirements. Output is ready for engineering handoff or architecture review. ## Required Inputs Ask the user for these if not provided: - **Pipeline purpose** — what business question or workflow does this pipeline serve? - **Source systems** — where does data come from? (databases, APIs, files, event streams) - **Destination** — where does data land? (data warehouse, data lake, downstream DB, reporting tool) - **Transformation type** — ETL (transform before loading) or ELT (load raw, transform in warehouse)? - **Frequency / SLA** — how often must data be fresh? (real-time / hourly / daily / weekly) - **Volume estimate** — approximate rows/events per run - **Data quality requirements** — completeness, deduplication, freshness, schema enforcement - **Team or stack** — any specific tools in use? (Airflow, dbt, Fivetran, Spark, Kafka, etc.) ## Output Structure --- # Data Pipeline Spec: [Pipeline Name] **Purpose:** [One sentence — what decision or workflow does this pipeline enable?] **Type:** [ETL / ELT / Streaming / Batch] **Owner:** [Team or individual] **Version:** [1.0] **Date:** [Date] **Status:** [Draft / Under Review / Approved] --- ## 1. Overview [2–3 sentences describing the pipeline end-to-end: what data moves, from where to where, at what cadence, and why....