template-pipeline-debugging

Solid

Systematic pipeline DAG failure triage for the Research Project Template. USE WHEN ./run.sh or execute_pipeline.py fails, a stage stalls (setup, tests, analysis, render, validate, LLM, copy), pytest/coverage gate fails mid-pipeline, PDF render or validate breaks, Project Analysis finishes too fast with no figures, or user says pipeline debug, stage failed, resume checkpoint, core-only triage — even without naming this skill or docs/prompts.

Data & Documents 19 stars 1 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 84/100

Stars 20%
43
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Pipeline debugging ## Natural invoke - "My pipeline failed at PDF render for template_code_project" - "Project Analysis finishes in under a second with no figures — help debug" - "run.sh --pipeline failed; what's the first real error?" - "Resume from checkpoint after fixing project tests" ## Inputs to confirm - **Project** — from [`docs/_generated/active_projects.md`](../../_generated/active_projects.md); infer from context if obvious. - **Invocation** — full vs `--core-only`, `--resume`, `--skip-infra`, multi-project. - **Failing stage** — if unknown, reproduce first. ## Workflow 1. **Reproduce** — re-run the failing invocation verbatim; capture full stderr/stdout and exit status. Quote the **first** real error, not the last line. Name the failing stage (setup, infra tests, project tests, analysis, render, validate, LLM review, LLM translations, copy). 2. **Isolate** — run that stage's underlying command directly (pytest target, analysis script, validation CLI, renderer). Use `--resume` to skip good upstream stages while iterating. 3. **Classify** — dependency/uv gap, missing system tool (LaTeX/pandoc-crossref), nondeterministic input, coverage gate, undefined citation/cross-ref, thin-orchestrator violation (logic in `scripts/` not `src/`), or logic bug. When **Project Analysis** completes in under a second with no figures, treat as import/dependency failure and isolate via `scripts/pipeline/stage_02_analysis.py`. Trace where bad state **enters**; fix at ingestion....

Details

Author
docxology
Repository
docxology/template
Created
1 years ago
Last Updated
today
Language
Python
License
Apache-2.0

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

Data & Documents Listed

debugging-data-pipelines

Systematically root-cause data pipeline failures and data incidents — job errors, wrong or missing data, duplicates, and freshness misses — by tracing lineage upstream, isolating the failing stage, reconciling against source, and planning a safe fix and backfill. Use when a pipeline fails, numbers look wrong, data is missing or duplicated, a dashboard is stale, or a stakeholder reports a data discrepancy.

17 Updated 1 weeks ago
Unknown-333
DevOps & Infrastructure Listed

pipeline-incident-triage

Triage a failed or wrong data pipeline on GCP and produce a fix plus a postmortem. Use when the user says a pipeline failed, a DAG is red, a table is stale, numbers look wrong, an assertion failed, a dashboard is broken, a job is stuck, or costs spiked unexpectedly. Also use when the user pastes an error from BigQuery, Airflow, Dataform, or Cloud Run.

0 Updated 5 days ago
rk-chavali
Data & Documents Listed

pipeline-doctor

Diagnose and safely repair a stuck ingest pipeline using only whitelisted CLI commands — stale vault locks, crashed running stages, corrupt window-done JSON, a forward-only state machine that refuses to re-run preprocessing. Use when the user says "the pipeline is stuck / 状态机卡住了 / the lock won't release / window-done keeps failing / it won't let me re-run profile / diagnose the pipeline". Never for content quality (kb-qa / wiki-lint-semantic), ingesting a source (ingest), post-publish retrospectives (kb-postmortem), or editing skills (skill-evolve); it never hand-writes SQL or edits the SQLite file directly.

2 Updated 3 weeks ago
Iabstergo1