data-engineeringlisted

Data platform engineering on Databricks, Microsoft Fabric, Snowflake and open source. Use when choosing a platform, layering or boundaries; when picking an engine or dataframe library or a job outgrew one machine; when onboarding a source and weighing CDC against cursor polling; when a dbt or SQLMesh project is laid out or reviewed; when an incremental model leaves stale rows or a MERGE runs far too long; when quality checks must be placed; when a bill grew or must be capped; when catalogs, grants, masking or record matching are set up; and when judging an inherited platform. MUST be used when rows go missing or duplicate while every run reports green. Routes grain and SCD to dimensional-data-modeling, RAG to ai-engineering, quality-threshold choice and anomaly judgment to statistics.
thefilesareinthecomputer/dotagents · ★ 0 · Data & Documents · score 72

Install: claude install-skill thefilesareinthecomputer/dotagents

# data-engineering Building and running a data platform, from the decision that picks a shape down to the line that makes a load wrong. The companion skill `dimensional-data-modeling` is the theory; this is the practice. **Altitude is the organizing principle, not the topic list.** "Which platform" and "why is this MERGE slow" are the same question at different heights, and the value here is moving between them without losing the thread. Every reference file carries all three altitudes for its topic, plus explicit up-links and down-links. | Altitude | Question it answers | What it produces | |---|---|---| | Architecture | What shape, which platform, which boundaries, what drives the bill | A decision with its conditions and its reversal cost | | Implementation | How is this built here, in what order, gated by what | A pipeline, a model, a bundle, a test suite | | Line of code | Why is this specific thing wrong, slow, or silently incorrect | A diff | The seams matter more than the levels, because they run downward and they are one-way. A platform choice forecloses implementation options; an implementation choice forecloses line-of-code options. A partitioned table forecloses row-level concurrency. A serverless-only feature forecloses where pipelines can run. When you answer at one altitude, name what you just foreclosed at the one below. ## Route by situation | The situation | Go to | |---|---| | Choosing a platform, layering, or boundaries; judging an inherited estate;