← ClaudeAtlas

chaos-tdd-fault-injectionlisted

Chaos-TDD for LLM agent pipelines — deterministic fault injection at existing seams, where the fault-injection test states the desired guarded behavior FIRST and the minimal production guard lands in the same PR. Use when hardening a pipeline against LLM/external-I/O fault families (truncated output, timeouts, 429s, wrong-shaped-but-parseable JSON, flapping backends), when building a fault catalog from operational bug history, when adding property-based fuzz over LLM output shapes, or when reviewing whether a pipeline's failure paths abstain with reason codes instead of degrading silently. NOT for infra-level chaos on distributed systems (chaostoolkit/toxiproxy are the right altitude there, not here), NOT for single-bug regression pinning after the fact (this skill is that discipline's front-loaded, catalog-driven counterpart), and NOT for audit-log schema design (a sibling concern this skill only asserts against).
shimo4228/chaos-tdd-fault-injection · ★ 0 · Testing & QA · score 73
Install: claude install-skill shimo4228/chaos-tdd-fault-injection
# Chaos-TDD Fault Injection **Seeded fault schedules as test-first specification.** LLM agent pipelines fail in one recurring family: an LLM or external I/O returns something unexpected — truncated text, a 429, valid JSON of the wrong shape — and the pipeline degrades *silently*. These bugs get diagnosed after they burn a production run, one at a time. Chaos-TDD front-loads them: > operational fault catalog → fault-injection test asserting the DESIRED > guarded behavior (RED) → minimal production guard in the same PR (GREEN). This is not Netflix-style random production chaos. A single-process agent needs no daemon to inject faults into itself, its stored data is often irreplaceable, and randomness makes CI flaky. Everything here is deterministic, in-process, and pytest-native. ## 1. Build the fault catalog from operational history Do not invent faults from imagination — mine them from incidents you already had: 1. List the incidents. Classify each by *where the unexpected response entered*: LLM text? HTTP status? embedding rows? the parse layer? 2. Diff that list against existing test coverage — grep for the **fault**, not the feature (`finish_reason`, `429`, `JSONDecodeError`, ...). 3. The uncovered remainder is the catalog. A typical first catalog: - mid-generation read-timeout (only connection-refused was covered) - direct HTTP faults on the embedding endpoint (429 / timeout / ragged rows / fewer rows than requested) - syntactically-valid JSON vi