← ClaudeAtlas

data-processinglisted

Process, transform, import, export, and handle large datasets using bounded and reliable technology-agnostic patterns.
soden46/engineer-flow · ★ 3 · Data & Documents · score 77
Install: claude install-skill soden46/engineer-flow
# data-processing Use this skill for imports, exports, transformations, batch processing, files, large datasets, and data pipelines. ## Principles Make data boundaries explicit. Validate external data before relying on it. For large workloads prefer bounded processing such as: - streaming - chunking - pagination - iterators - batches Avoid loading unbounded datasets into memory. Define behavior for: - malformed records - partial failure - retries - duplicates - ordering - checkpointing - resumability Keep transformation logic separate from transport/storage concerns where useful. For destructive or bulk operations, consider dry-run or preview mechanisms when appropriate. Verify record counts and important invariants after processing. ## Adaptation Use project evidence to determine the actual language, framework, runtime, and existing conventions. When stack-specific implementation guidance is needed, prefer project evidence, native framework or language mechanisms, and relevant user-installed specialist skills. Technology-specific guidance must not redefine or weaken the core engineering requirement.