reviewing-data-pipeline-codelisted

# Reviewing Data Pipeline Code ## When to use - Reviewing a PR that changes SQL, dbt models, Spark jobs, or orchestration. - Adding a new model, transformation, or ingestion step. - You want to catch data-correctness and cost issues generic review misses. - Do NOT use for pure application code review (use standard code review). ## The data-engineering review checklist Copy this into the review and check each item: ``` - [ ] Grain: is the output grain declared and enforced (unique key test)? - [ ] Idempotency: safe to re-run? upsert/partition-overwrite, not blind append? - [ ] Incremental: unique_key set? lookback for late data? no missed rows? - [ ] Correctness: joins can't fan out; NULL/timezone/dedup handled? - [ ] Tests: unique/not_null/relationships on keys; business-rule checks added? - [ ] Cost: partition/cluster pruning used? no SELECT *? bounded scans? - [ ] Backward compat: schema change additive? downstream/exposures considered? - [ ] PII/security: sensitive fields masked/limited? no secrets in code? - [ ] Observability: freshness/volume covered for a new critical dataset? ``` ## What to look for 1. **Grain and fan-out.** The most damaging bug is a join that multiplies rows. Confirm the grain is declared and a uniqueness test guards it. 2. **Idempotency.** Ask: what happens on retry or backfill? Reject blind `INSERT`-append; require MERGE/upsert or partition overwrite. 3. **Incremental logic.** Check `unique_key`, the `is_incremental()` filter, and a