---
name: backtest-integrity
description: "Audit any trading backtest for overfit before you trust it. Runs the Lopez-de-Prado integrity battery: purged & embargoed combinatorial-CV (CPCV), Probability of Backtest Overfitting (PBO), Deflated Sharpe Ratio (DSR) with the FULL trial count, sample-uniqueness down-weighting, and triple-barrier labels. Turns 'it backtested great' into a calibrated honesty verdict. By TradeHouse (Bloomberg-terminal-of-prediction-markets)."
license: MIT
homepage: https://th-validator.eliteaiempire.com
tested_with: claude-code v2.x
---

# Backtest-Integrity

A great-looking backtest is the default outcome of searching enough strategies — not
evidence of edge. This skill applies the **Lopez-de-Prado** integrity battery so you only
trust a result that survives it.

## When to use
- Before promoting a trading strategy / signal / persona to live capital.
- When a backtest Sharpe looks "too good" — quantify the overfit probability.
- Reviewing someone else's backtest claim.

## The battery
1. **Purged + embargoed CV / CPCV** — remove train/test leakage from overlapping labels; an
   un-purged CV inflates Sharpe by leaking future info across the split boundary.
2. **PBO (Probability of Backtest Overfitting)** — across combinatorial train/test splits,
   how often does the in-sample-best config underperform out-of-sample? PBO > 0.5 = the
   selection is no better than noise.
3. **Deflated Sharpe Ratio (DSR)** — deflate the observed Sharpe by the number of trials
   ACTUALLY searched (the whole grid, not the survivors). Undercounting trials is the #1
   way DSR lies — count the full search space.
4. **Sample-uniqueness weighting** — concurrent/overlapping labels are not independent
   observations; down-weight them or N is fiction.
5. **Triple-barrier labels** — label by which of {profit-take, stop, time} is hit first,
   not fixed-horizon returns.

## How to run
1. Take the strategy's returns/labels + the SIZE of the search that produced it (number of
   configs/params/personas tried — be honest, include the discarded ones).
2. Run purged-CPCV → PBO. Run DSR with trials = full search space. Apply uniqueness weights.
3. Emit the verdict: `SUPPORTED` (survives) / `OVERFIT_RISK` (PBO high or DSR ≤ 0) with the
   numbers and what would change the verdict.

## Output
`{ PBO, DSR, deflated_sharpe, trials_counted, uniqueness_factor, verdict }` + a plain-English
"would I bet real money on this?" line.

## Upgrade
The hosted **Overfit Validator** (free lead-magnet) + Research (hypothesis→p-value) + Copilot
are at **https://th-validator.eliteaiempire.com**, backed by 16-venue prediction-market
settlement + L2 depth. This skill is the portable integrity logic.