false-green-tests

Solid

Find out why the tests pass but the app is broken. Catches false greens: a green suite over a feature that does not work, a mocked API standing in for a real one, an assertion that holds no matter what the app does, a click handler wired to nothing. Use when the suite is green and the user says it is broken, when a test never fails, when coverage looks fine but bugs still ship, or before trusting a passing run you did not watch.

Testing & QA 888 stars 150 forks Updated today NOASSERTION

Install

View on GitHub

Quality Score: 84/100

Stars 20%
98
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# The suite is green and the app is broken A green test is evidence about the test, not about the app. This skill separates the two by running the real app and comparing what it _does_ against what the test _claims_. It uses **Reticle**, which observes the running app from the inside: DOM, network, console, routing, and framework state. Not installed? `RETICLE_INSTALL_SOURCE=npx_skill npx @reticlehq/server@latest init`, then see the [`install-and-verify`](https://github.com/reticlehq/reticle/blob/main/skills/install-and-verify/SKILL.md) skill. ## The five shapes, in the order they are worth checking **1. The assertion cannot fail.** Read the test the user trusts. If it only asserts absence (no console error, no thrown exception, no rejected promise) it passes on a control wired to nothing. A dead button throws nothing, fires nothing, and changes nothing. Prove it in the app instead: ``` reticle_act_and_wait({ sessionId, ref, action: "click", until: { kind: "allOf", predicates: [ { kind: "net", method: "POST", urlContains: "/api/...", status: 200 }, { kind: "signal", name: "..." }, ]}}) ``` A verdict of `no` here, against a green suite, is the false green. **2. The test drove a mock and the app drives an API.** Compare what the app actually requested with what the test stubbed: ``` reticle_observe({ action: "network", sessionId, since }) ``` No request where the test asserted one means the suite verified a fixture. A stale client cache is the same failure with ...

Details

Author
reticlehq
Repository
reticlehq/reticle
Created
3 months ago
Last Updated
today
Language
TypeScript
License
NOASSERTION

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category

Testing & QA Listed

falsifiable-tests

Empirically prove a test can fail before trusting it — break the code under test, confirm the test turns red for the right reason, then restore. Use this whenever you write, modify, or review tests; whenever a suite goes green and you are about to tell the user something "works"; and whenever the user asks to audit tests, check test quality, or hunt for false negatives, flaky tests, vacuous assertions, or tests that pass no matter what. Trigger it even when the user only says "add a test" or "make the tests pass" — a test never observed failing is unverified, and this is what verifies it.

0 Updated 1 weeks ago
ArnauFerma
Testing & QA Listed

build-test-truth

Use when building a project, running its tests, or reasoning about a CI result. A passing command is not a passing suite. Knows the "green-by-skip" traps - a run that discovered or executed zero tests, a suite that self-skipped on a missing environment flag, a tool/flag combination that reports success while running nothing, or the wrong test project running - and applies an execution gate that asserts the tests actually covering the change ran and passed. Triggers on build, compile, run the tests, CI, "the tests pass", vstest/xunit/jest/pytest, green build.

0 Updated 5 days ago
scott-garvin
Testing & QA Listed

tests

What it is — a standard for tests that mean something when they are green: a verdict no pipe can swallow, logs read even at exit 0, every check watched failing before it is trusted, test-first with a falsification pass behind it, commits granular enough to bisect, and no compatibility shims for a shape that was never released. Use when writing, running or reviewing tests, judging whether a green run or an exit code proves anything, finding the commit that broke a test or the earlier test that pollutes it, deciding what to do about a test that passes and fails on the same code, or setting up testing in a new repo. Triggers: tests, test suite, coverage, TDD, flaky, mock, fixture, golden file, bisect, which commit broke it, test pollution, .only, exit code, pipefail, debugging, mutation, property-based, e2e, quarantine, напиши тест, покрытие, тесты падают, зелёный прогон, нестабильный тест, какой коммит сломал, падает только в общем прогоне, код возврата, отладка, карантин.

0 Updated yesterday
rokokol