distributed-job-safety

# Distributed Job Safety Patterns and anti-patterns for concurrent job management with pueue + mise + systemd-run, learned from production failures in distributed data pipeline orchestration. **Scope**: Universal principles for any pueue + mise workflow with concurrent parameterized jobs. Examples use illustrative names but the principles apply to any domain. **Prerequisite skills**: `devops-tools:pueue-job-orchestration`, `itp:mise-tasks`, `itp:mise-configuration` --- > **Self-Evolving Skill**: This skill improves through use. If instructions are wrong, parameters drifted, or a workaround was needed — fix this file immediately, don't defer. Only update for real, reproducible issues. ## The Nine Invariants Non-negotiable rules for concurrent job safety. Violating any one causes silent data corruption or job failure. Full formal specifications: [references/concurrency-invariants.md](./references/concurrency-invariants.md) ### 1. Filename Uniqueness by ALL Job Parameters Every file path shared between concurrent jobs MUST include ALL parameters that differentiate those jobs. ``` WRONG: {symbol}_{start}_{end}.json # Two thresholds collide RIGHT: {symbol}_{threshold}_{start}_{end}.json # Each job gets its own file ``` **Test**: If two pueue jobs can run simultaneously with different parameter values, those values MUST appear in every shared filename, temp directory, and lock file. ### 2. Verify Before Mutate (No Blind Queueing) Before qu...