data-processinglisted
Install: claude install-skill soden46/engineer-flow
# data-processing
Use this skill for imports, exports, transformations, batch processing, files, large datasets, and data pipelines.
## Principles
Make data boundaries explicit.
Validate external data before relying on it.
For large workloads prefer bounded processing such as:
- streaming
- chunking
- pagination
- iterators
- batches
Avoid loading unbounded datasets into memory.
Define behavior for:
- malformed records
- partial failure
- retries
- duplicates
- ordering
- checkpointing
- resumability
Keep transformation logic separate from transport/storage concerns where useful.
For destructive or bulk operations, consider dry-run or preview mechanisms when appropriate.
Verify record counts and important invariants after processing.
## Adaptation
Use project evidence to determine the actual language, framework, runtime, and existing conventions.
When stack-specific implementation guidance is needed, prefer project evidence, native framework or language mechanisms, and relevant user-installed specialist skills. Technology-specific guidance must not redefine or weaken the core engineering requirement.