vacuous-checklisted

# A Success Test That Accepts Any Output Is Not a Test ## Problem A check reports success and the operation did not happen. The check is not lying about what it saw — it is reporting truthfully on a predicate that cannot distinguish success from failure, because **failure output satisfies the predicate too.** This is not the same bug as a missing check. A missing check is loud: nothing claims the work is done. This one is silent and worse, because it manufactures positive evidence. ## Use when Any of these, in any language: - A shell retry loop whose success test is `[ -n "$out" ]`, `[ $? -eq 0 ]` on a command that exits 0 on API errors, or a grep for a substring that appears in both outcomes. - `gh api ... --jq .some_field` — `gh` prints the API's error JSON to stdout, so `--jq` on a 4xx/5xx body yields a non-empty string. - A test helper that stringifies both sides before comparing: `String(got) === String(want)` passes on `check('x', 1, '1')`. - A verification script that counts lines, checks a file exists, or greps for a header, where the failure mode also produces lines, a file, or that header. **The tell:** you can describe the failure mode and it still satisfies the predicate. If you cannot construct a failing input that the check rejects, the check is decorative. ## Solution Four rules, in order of how often they pay. ### 1. Assert the shape success produces, not the absence of nothing ```sh # WRONG — an error body is non-empty url=$(gh api "repos/