medallion-designlisted

Design a medallion (bronze/silver/gold) ETL architecture interactively, objective-first. List the available data, confirm the objective, then design GOLD first to match the objective and get the user to review it before moving down to silver, then bronze (top-down default) — or bronze-up if the user asks. Asks the user to confirm at each layer boundary rather than designing all three in one shot. Wraps the reusable utils/ library (bronze.py, silver.py, scd.py, watermark.py, metadata.py, quality.py). Use this skill whenever the user wants a bronze/silver/gold or medallion/lakehouse layout, is building a layered Delta/warehouse pipeline, or wants an objective-driven top-down layer design. For extraction/idempotency mechanics reference pipeline-design; this owns the layered design conversation.
Methasit-Pun/data_engineer_claude_skills · ★ 1 · Web & Frontend · score 65

Install: claude install-skill Methasit-Pun/data_engineer_claude_skills

# Medallion Design (Interactive, Objective-First) Medallion = **bronze** (raw, as-ingested) → **silver** (cleaned, conformed) → **gold** (business-ready, objective-shaped). This skill designs it as a **guided conversation**, one layer at a time, so the gold layer actually matches what the user needs — not a generic three-tier that misses the goal. ## Core principle: design gold first (top-down default) Most medallion designs fail because they build bronze→silver→gold and *discover* at the end that gold doesn't answer the question. Invert it. **Start from the objective, define gold, then derive what silver and bronze must contain to feed it.** Bottom-up (bronze→gold) is available too — use it when the user explicitly asks, or when the sources are fixed/unknown-purpose and you're exploring what gold *could* be. ## The interactive protocol — ask at every boundary Do **not** design all three layers in one response. Walk the user through it: 1. **Inventory + objective.** List every dataset available (pull from [[data-sourcing]] / [[data-profiling]] if present). Confirm the objective in one sentence: *"Gold exists to answer ___."* 2. **Design GOLD.** Propose the gold tables — grain, columns, metrics, dimensions, SCD needs — shaped directly to the objective. **Stop and ask the user to review before continuing.** 3. **On approval, design SILVER.** Derive the cleaned/conformed tables that gold requires: standardized names, cast types, dedup keys, validated rows. **Stop and ask