background-jobslisted

# Background jobs A job system's contract is at-least-once execution at some later time. Design every job for the "at least" and the "later": it will run twice, and it will run after the world changed. ## Method 1. **Enqueue references, not state.** The payload is IDs plus intent (`{"type": "send_receipt", "order_id": 123}`); the job reloads current state at run time. Snapshotting state into payloads acts on stale data after any delay or retry. 2. **Make every job idempotent.** Natural idempotency where possible (setting a status, upserting), otherwise an idempotency key checked in the job's own transaction. The queue will redeliver after crashes between work and ack; only the job can make that safe. 3. **Retry with backoff, jitter, and a ceiling.** Exponential backoff starting at seconds, jittered to avoid thundering herds, capped attempts (5-10). Distinguish retryable failures (timeouts, 429s) from permanent ones (validation): permanent failures skip retries and go straight to the dead-letter queue. 4. **Dead-letter with context, and staff the queue.** Failed-forever jobs land in a DLQ with error, attempt count, and first-failure time. A DLQ nobody monitors is a silent data-loss pit: alert on depth and age, and build the one-click requeue for after the fix ships. 5. **Set visibility timeouts above p99 runtime.** Too short and the queue redelivers mid-run (guaranteed duplicates); heartbeat/extend for long jobs. Bound job runtim