chaos-tdd-fault-injectionlisted
Install: claude install-skill shimo4228/chaos-tdd-fault-injection
# Chaos-TDD Fault Injection
**Seeded fault schedules as test-first specification.**
LLM agent pipelines fail in one recurring family: an LLM or external I/O
returns something unexpected — truncated text, a 429, valid JSON of the wrong
shape — and the pipeline degrades *silently*. These bugs get diagnosed after
they burn a production run, one at a time. Chaos-TDD front-loads them:
> operational fault catalog → fault-injection test asserting the DESIRED
> guarded behavior (RED) → minimal production guard in the same PR (GREEN).
This is not Netflix-style random production chaos. A single-process agent
needs no daemon to inject faults into itself, its stored data is often
irreplaceable, and randomness makes CI flaky. Everything here is
deterministic, in-process, and pytest-native.
## 1. Build the fault catalog from operational history
Do not invent faults from imagination — mine them from incidents you already
had:
1. List the incidents. Classify each by *where the unexpected response
entered*: LLM text? HTTP status? embedding rows? the parse layer?
2. Diff that list against existing test coverage — grep for the **fault**,
not the feature (`finish_reason`, `429`, `JSONDecodeError`, ...).
3. The uncovered remainder is the catalog. A typical first catalog:
- mid-generation read-timeout (only connection-refused was covered)
- direct HTTP faults on the embedding endpoint (429 / timeout / ragged
rows / fewer rows than requested)
- syntactically-valid JSON vi