airflow-dagslisted

Use when creating, debugging, or configuring Apache Airflow 3 DAGs — builds data pipelines with TaskFlow API or traditional operators, configures scheduling and asset-driven triggers, wires XCom data passing, sets up sensors and deferrable operators, generates dynamic task mappings, and structures multi-layer test suites. Triggers on DAG authoring, TaskFlow API, operators, sensors, scheduling, assets, dynamic tasks, XCom, or pipeline testing.
wagneripjr/skills · ★ 0 · AI & Automation · score 75

Install: claude install-skill wagneripjr/skills

# Writing Airflow 3 DAGs Airflow 3 DAGs are Python code that define data pipeline workflows. Every DAG must follow three principles: **atomicity** (each task does one thing), **idempotency** (same input = same output on rerun), and **modularity** (reusable functions and operators, DRY). ## Project Structure ``` project/ dags/ # One .py file per DAG, filename = dag_id include/ # Support code (NOT parsed by scheduler) sql/ python_functions/ custom_operators/ custom_hooks/ tests/ dag_validation/ # DagBag-based structural checks unit_tests/ # Custom code with mocked dependencies integration_tests/ # Real external systems, no mocking plugins/ cluster_policies/ # Environment-level enforcement (@hookimpl) ``` ## Quick Decision Guide | Need | Approach | Reference | |------|----------|-----------| | Standard multi-task pipeline | TaskFlow API (`@dag`/`@task`) | [dag-authoring.md](references/dag-authoring.md) | | Single data producer task | Asset-oriented (`@asset`) | [dag-authoring.md](references/dag-authoring.md) | | Legacy code / specific operators | Traditional syntax (`DAG` class) | [dag-authoring.md](references/dag-authoring.md) | | Variable number of task copies | Dynamic Task Mapping (`.expand()`) | [dynamic-tasks.md](references/dynamic-tasks.md) | | 50+ similar DAGs from config | Dynamic DAGs (`dag-factory`) | [dynamic-tasks.md](references/dynamic-tasks.md) | | Time-based