Trading statistics: test what a backtest result actually says · 3 / 5
Overlapping trades: why twenty results may contain four market episodes
Twenty trades opened around four announcements are twenty executions, but they may contain far fewer independent market experiences. Preserve the individual trades while grouping the common episodes that can move them together.
Athenum7 minUpdated:
Count observations and dependence separately
An observation count is a literal count under your grouping rule. Independence is an assumption about relationships among outcomes. Trades can share entry timing, holding windows, instruments, signals or market shocks. A larger row count does not by itself justify narrower uncertainty intervals.
Overlap does not prove perfect dependence, and non-overlap does not prove independence. A persistent regime can link trades across days. Examine the mechanism, time series and sensitivity to plausible grouping choices. Autocorrelation diagnostics can reveal some dependence but do not certify that all relevant dependencies are absent.
Keep the account constraint in the replay
If five trades are open simultaneously, the simulation must allocate capital and risk to all five at once. Replaying each as if it alone controlled the entire account can create impossible aggregate exposure. The same issue arises when counting many related signal variants from one price move as independent confirmations.
For uncertainty analysis, group outcomes using a rule set before evaluating their success: for example, event episode, session or a justified time block. Keep the member trades and report both counts. Do not choose the grouping that produces the smallest uncertainty after seeing the results.
Worked example: five copies of each common episode
Construct four independent episodes only for this teaching model. Each episode contains five perfectly identical net trade outcomes: +2R, −1R, +1R and −2R respectively. There are twenty trade rows, but within each episode the five copies provide no additional information about its shock. The average is zero under either representation.
If episode outcomes have population variance σ² and episodes are independent, the mean’s variance is σ² / 4. Incorrectly treating all twenty identical copies as independent uses σ² / 20, understating variance by a factor of five and standard error by √5. This exact factor depends on the constructed perfect dependence; it is not a general formula for all overlapping trades.
| Episode | Trade rows | Outcome of each row | Distinct modeled shock |
|---|---|---|---|
| A | 5 | +2R | 1 |
| B | 5 | −1R | 1 |
| C | 5 | +1R | 1 |
| D | 5 | −2R | 1 |
Open full-size diagram- Recorded trades: 20 count
- Independent episodes in this model: 4 count
Do not replace evidence with an arbitrary effective count
Simply dividing every strategy’s trade count by five is as unjustified as assuming independence. The example’s factor follows from its explicit structure. Real analysis requires an appropriate covariance or resampling model, sufficient history and sensitivity checks. Preserve raw counts even when the uncertainty model uses groups.
Before acting
- Map holding windows and common event drivers.
- Keep literal trade count separate from independence assumptions.
- Replay simultaneous capital and risk constraints.
- Declare grouping before using it to quantify uncertainty.
Check your understanding
A sample has six independent sessions, each containing three perfectly identical outcomes under a teaching model. By what factor does treating all eighteen rows as independent understate the variance of their mean?
Show the explained answer
The correct variance under the stated model is σ² / 6; the incorrect independent-row variance is σ² / 18. The variance is understated by a factor of three, and standard error by √3. This conclusion relies on independent sessions and perfectly identical within-session outcomes.