data-engineering-data-pipeline
FeaturedYou are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.
Install
Quality Score: 99/100
Skill Content
Details
- Author
- sickn33
- Repository
- sickn33/antigravity-awesome-skills
- Created
- 10 months ago
- Last Updated
- today
- Language
- Python
- License
- MIT
Integrates with
Bundled in these plugins
Similar Skills
Semantically similar based on skill content — not just same category
data-engineering-data-pipeline
You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.
data-engineering-data-pipeline
You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.
data-pipeline-architect
Designs robust ETL/ELT data pipelines covering ingestion, idempotency, schema evolution, orchestration, and data quality validation. Use this skill when the user asks to design, build, or review a data pipeline, ingest data from APIs/databases/files into a warehouse or lake, set up batch or streaming ETL/ELT, choose an orchestrator (Airflow, Dagster, Prefect, dbt), make a pipeline idempotent or backfill-safe, handle late-arriving or duplicate data, manage schema drift/evolution, add data quality or freshness checks, or model incremental/CDC loads.