Backtesting: How to Properly Test a Trading Strategy Before Going Live

The Three Biggest Backtesting Biases

Survivorship Bias

Survivorship bias occurs when you test only on stocks or assets that still exist today. Indexes like the S&P 500 regularly add and remove companies. In 2000, the S&P 500 included Enron, WorldCom, and Lehman Brothers — all of which went to zero. If you backtest a strategy using only current S&P 500 members, you are excluding the companies that failed. Your backtest looks better than reality because you only include the survivors. The fix: use databases that include delisted securities, such as CRSP (Center for Research in Security Prices) or Compustat. For forex and crypto, survivorship bias is less severe because currency pairs and major cryptocurrencies tend to persist, but data quality remains a concern. See how quants handle survivorship bias →

Look-Ahead Bias

Look-ahead bias occurs when your strategy uses information that was not available at the time of the trade. The most common example: using the daily close price to generate an entry signal for the same day's open. If your strategy says "buy at the open if yesterday's close was above the 20-day moving average," you are fine — you had yesterday's close before the open. But if your strategy says "buy at the close if today's close is above the 20-day moving average," you cannot enter until the next day's open because you did not have today's close at the close. Other examples: using revised GDP data that was not available at the time, or using corporate earnings data from restated filings. Every piece of data in your backtest must have been available at the time the trade signal occurred.

Overfitting (Data Snooping)

Overfitting means optimizing your strategy's parameters so precisely to historical data that you fit the noise instead of the signal. If you test 1,000 parameter combinations and pick the one that performed best on 2018-2023 data, that combination almost certainly captured random patterns specific to that period. The classic sign: optimal parameters are specific and non-standard. A 14-period RSI is standard; a 13-period RSI that happened to work better on 2018-2023 data is suspect. To avoid overfitting, use out-of-sample testing — reserve 20-30% of your data (typically the most recent period) for final validation. Never look at out-of-sample results during development. Use walk-forward analysis to test parameter stability across multiple time periods. Use Monte Carlo to test for overfitting →

Walk-Forward Analysis and Out-of-Sample Testing

Walk-forward analysis is the gold standard for backtesting robustness. Divide your data into multiple periods: an in-sample period for parameter optimization and an out-of-sample period for validation. Then roll forward: optimize on 2018-2019, test on 2020; optimize on 2018-2020, test on 2021; and so on. This creates a sequence of out-of-sample results that simulate how the strategy would have performed if you had been trading it forward in time. If the out-of-sample results are consistently weaker than in-sample results (which they will be), the key question is whether the out-of-sample results are still profitable. A robust strategy degrades gracefully; an overfitted strategy collapses in out-of-sample testing. Analyze rolling returns for strategy robustness →

How many trades do I need for a statistically valid backtest?

A minimum of 100 trades is required for basic statistical significance, with 200-300+ being preferable. Fewer than 30 trades is statistically meaningless — you cannot distinguish skill from luck. Also ensure your backtest covers multiple market regimes: bull markets, bear markets, sideways markets, high volatility periods, and low volatility periods. A strategy that only works in trending bull markets is not robust. Test across at least 5-10 years of data covering multiple economic cycles. For higher timeframe strategies (daily charts), this may require 15-20 years of data to get enough trades.

What metrics should I evaluate in a backtest?

The key metrics are: total return (net profit over the test period), win rate (percentage of profitable trades), average win vs average loss, profit factor (gross profit / gross loss — aim for 1.5+, excellent above 2.0), maximum drawdown (largest peak-to-trough decline), Sharpe ratio (risk-adjusted return — above 1.0 is good), and number of trades (minimum 100 for significance). Additional metrics include the Calmar ratio (return / max drawdown), Sterling ratio, and the percentage of profitable months. Also examine the equity curve visually — a smooth upward curve with small drawdowns is far better than a volatile curve with large swings, even if total returns are similar.

Should I include slippage and commissions in my backtest?

Yes, always include slippage and commissions in your backtest. Realistic slippage estimates: $0.01-$0.02 per share for liquid US stocks, 0.5-1 pip for major forex pairs, $0.10-$0.20 per contract for liquid futures. Commission estimates depend on your broker. A strategy that makes $10,000 in backtesting without costs could lose $2,000 after slippage and commissions — the difference between a winning and losing strategy. Always be conservative with your cost estimates. If a strategy barely breaks even with realistic costs, it will lose money in live trading. Add a buffer of 20-30% to your cost estimates for safety.

What is forward testing and why is it important?

Forward testing (also called paper trading) is the step between backtesting and live trading. You run your strategy in real-time on a demo account for at least 50-100 trades over 2-3 months. Forward testing catches issues that backtesting misses: execution problems (slippage, fills, spread costs), psychological challenges (watching a drawdown develop in real-time), and data quality issues (your backtest data may have been cleaner than live data). A strategy that passes both backtesting and forward testing has a much higher probability of succeeding in live trading. Never skip this step — it is the cheapest insurance against a bad strategy.

Related Resources