rseng-numerical-accuracy

Solid

Covers floating-point correctness in research code: why 0.1 + 0.2 != 0.3, choosing absolute vs relative tolerances in tests, accumulation error and safe summation, precision choices (float32 vs float64), catastrophic cancellation, NaN and infinity handling, and cross-platform or cross-library result drift. Use PROACTIVELY when floating-point comparisons fail mysteriously, when writing numerical tests or choosing tolerances, when results differ across machines, compilers, BLAS builds or library versions, or when precision or numerical stability questions arise in analysis or simulation code.

Code & Development 14 stars 2 forks Updated 4 days ago MIT

Install

View on GitHub

Quality Score: 83/100

Stars 20%
39
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Numerical accuracy in research code Floating-point numbers are approximations with well-defined rules, and research conclusions can hinge on respecting them. The two recurring failures: treating floats as exact (== comparisons, accumulating error blindly) and treating all differences as noise (tolerances loosened until tests pass). Both are avoidable with a small set of habits - and both matter more in research than elsewhere, because the numbers ARE the result. ## The ground rules - Never compare floats with == (except against an exactly representable sentinel like 0.0 you assigned yourself). Use tolerance-based comparison: `math.isclose`, `numpy.isclose`, or the testing helpers below. - Decimal literals are usually not representable: 0.1 is stored as the nearest binary fraction, which is why 0.1 + 0.2 != 0.3. Format-rounding for display hides this; arithmetic does not. - NaN propagates and never equals anything, including itself; test with isnan, and decide explicitly whether NaN in data means missing, invalid or bug (rseng-data-management's missing-data discipline). Silent NaN propagation into published numbers is the classic silent failure. - Precision-degrading operations (subtracting nearly equal numbers - catastrophic cancellation; summing numbers of very different magnitude) lose precision structurally; restructure the formula (e.g. use expm1/log1p, two-pass variance algorithms) rather than adding digits. ## Tolerances: choose, do not t...

Details

Author
fdiblen
Repository
fdiblen/rseng-agent-skills
Created
4 days ago
Last Updated
4 days ago
Language
Python
License
MIT

Similar Skills

Semantically similar based on skill content — not just same category

Code & Development Solid

rseng-defensive-coding

Covers defenses against silently wrong research results: validating data at boundaries (schemas, assertions, sanity checks), explicit physical units and quantities in code (pint/astropy-style), disciplined randomness (explicit seeded generators, parallel streams), and fail-loud handling of NaN and missing data. Use PROACTIVELY when code ingests external or instrument data, when values carry physical units, when randomness enters simulations or sampling, or when NaN or missing-data handling is implicit; also when the user mentions data validation, unit errors, seeds or silent bugs, or reviews analysis code whose failure would be invisible. For floating-point behavior and tolerances see rseng-numerical-accuracy; for diagnosing an existing bug see rseng-debugging.

14 Updated 4 days ago
fdiblen
AI & Automation Listed

integer-issues

Detect integer over/underflow in `unchecked` blocks, downcasting losses, fixed-point precision errors, division-before-multiplication, signed/unsigned mixing. Activate on any arithmetic in `unchecked { }`, `SafeCast`, `uintN(uintM(x))` casts, division and modulo, percentage/basis-points math, AMM share/asset math.

37 Updated 3 weeks ago
iktok90-design
Code & Development Listed

state-the-noise-floor

Before you put a NUMBER in front of a maintainer — "N tickets affected", "X% of cases", "this run is 13% slower" — establish what that same number would be if nothing were wrong, and report the ratio. Build the floor by comparing against something that SHOULD be identical: repeat the measurement (N≥5), or shuffle the one variable your claim depends on and keep everything else fixed. If the signal is under ~2x the floor, say so yourself before the reviewer does. Use when you are about to quote a count, a percentage, or a performance delta in an issue, PR description or mailing-list post. Trigger terms: benchmark, regression, faster, slower, N tickets, X percent, how many, count, delta, improvement, is this significant, noise, variance, baseline.

1 Updated 3 weeks ago
serhiy-bzhezytskyy