Why Backtesting Is Non-Negotiable
Every trading strategy is a hypothesis: "If condition X occurs, then entering in direction Y with risk Z will be profitable over a large sample of trades." Backtesting is the process of testing that hypothesis against historical data before risking real capital. Trading without backtesting is the equivalent of launching a product without any testing — you might get lucky, but the odds are against you.
Ernest Chan, in Quantitative Trading, emphasizes that backtesting is not about proving a strategy works — it is about disproving the null hypothesis that the strategy has no edge. This distinction matters: you are looking for evidence that your results are not due to random chance.
Manual vs. Automated Backtesting
Manual Backtesting
Manual backtesting involves scrolling through historical charts and recording trades as if you were trading in real time. This method is slower but builds deep pattern recognition and intimacy with your setup. For discretionary strategies — especially those involving price action concepts like order blocks, FVGs, and CHoCH signals — manual backtesting is essential because these patterns require contextual judgment that is difficult to fully automate.
Automated Backtesting
Automated backtesting uses code to simulate a strategy across historical data. It is faster, eliminates human bias in trade selection, and can test across far larger datasets. However, it requires precise rule definition and is susceptible to coding errors that can produce misleading results. Platforms like TradingView's Pine Script strategy tester make automated backtesting accessible even for non-programmers.
Sample Size and Statistical Significance
One of the most common backtesting mistakes is drawing conclusions from too few trades. A strategy that wins 8 out of 10 trades might look exceptional, but that sample is far too small to be statistically meaningful. As a general rule, aim for a minimum of 100 trades — and ideally 200 or more — before evaluating a strategy's viability. This sample size allows the law of large numbers to smooth out the natural variance in trade outcomes.
Avoiding Curve Fitting
Curve fitting — also called overfitting — occurs when you optimize a strategy's parameters so extensively that it perfectly fits historical data but fails on new data. Academic research on overfitting in financial models, including work by Marcos Lopez de Prado in Advances in Financial Machine Learning, demonstrates that the more parameters you optimize, the higher the probability that your backtested results are illusory.
- Keep rules simple. A robust strategy has few, clearly defined rules. If your strategy requires more than five or six conditions to trigger a trade, it is likely overfit.
- Avoid optimizing to the decimal. If your strategy only works with a 14.3-period moving average but fails with a 13 or 15, it is fragile and unlikely to hold up in live conditions.
- Test across multiple instruments and timeframes. A genuine edge should be observable across related markets, not confined to a single asset.
Walk-Forward Analysis
Walk-forward analysis is the gold standard for validating a backtested strategy. The process works as follows: optimize your strategy on an "in-sample" data window, then test it on the immediately following "out-of-sample" window without any changes. Slide the window forward and repeat. If the strategy performs consistently across multiple out-of-sample periods, you have genuine evidence of an edge.
This method directly addresses the curve-fitting problem because out-of-sample data was not used in parameter selection. If your results deteriorate dramatically between in-sample and out-of-sample windows, the strategy is overfit to historical patterns that did not persist. Tools like the MoQ indicator suite can help define objective, repeatable entry criteria — such as a BOS at an order block with oscillator confirmation — that translate cleanly from backtest to live execution.
Sources & Further Reading
- Ernest P. Chan, Quantitative Trading (2008) — practical guide to developing and backtesting algorithmic trading strategies.
- Marcos Lopez de Prado, Advances in Financial Machine Learning (2018) — rigorous treatment of overfitting, cross-validation, and statistical pitfalls in strategy development.
- Robert Pardo, The Evaluation and Optimization of Trading Strategies (2008) — the definitive reference on walk-forward analysis methodology.
- Investopedia, "Backtesting Definition" — accessible introduction to backtesting concepts and best practices.
- TradingView, "Strategy Tester" — guide to using TradingView's built-in strategy backtesting engine.