backtest-overfittinglisted
Install: claude install-skill howard-lynn-ye/fin-skills
# Backtest overfitting
**Seven guards in this repo ask whether a backtest cheated. This one asks the question that is
still open after they all pass: the backtest is clean, and it is *the maximum of a search*.**
Two numbers answer it. **PBO** asks how often the in-sample winner lands in the bottom half out
of sample — 0.5 means the winner is a coin flip. **MinBTL** asks how many years of data a
claimed Sharpe needs before the honest trial count stops explaining it on its own.
Every number below is printed by `scripts/overfitting.py` (numpy + scipy, seeds fixed and
printed, **24 s**). Nothing here is quoted from a paper without being reproduced.
## 1. CSCV, exactly as defined
✅ Source: Bailey, Borwein, López de Prado & Zhu, *The Probability of Backtest Overfitting*,
Journal of Computational Finance 20(4), 2016 — read from the authors' hosted preprint
(`davidhbailey.com/dhbpapers/backtest-prob.pdf`; the published JCF text is paywalled and was not
reached). Algorithm 2.3:
1. Build a **T x N** performance matrix — one column per configuration you tried, rows aligned
on the same calendar.
2. Split the **rows** into an even number **S** of disjoint contiguous blocks.
3. Form all **C(S, S/2)** ways of choosing S/2 blocks as in-sample; the complement is
out-of-sample. Both halves are the same size — that is what "symmetric" means, and it is why
IS and OS performance are directly comparable.
4. `n*` = the configuration with the best IS performance. Find its **rank** amo