Backtesting

Testing on the past

Before a new roller coaster opens, engineers run it empty many times. A backtest is the trading version: you check how your rules would have done on old price data, before you risk a dollar.

There are two ways to do it:

• By hand. Scroll a chart back in time, hide the candles to the right, then move forward one candle at a time. Each time your setup shows up, write down the entry, stop, target and what happened. Many charting apps have a "replay" button that hides the future for you.
• With a computer. You write the rules as code and the computer tests years of data in seconds. Fast, but it's also easier to fool yourself.

Either way, you end up with a list of pretend trades you can measure.

The honest traps

Doing a backtest is easy. Doing an honest one is hard. These traps make bad strategies look amazing.

1. Cherry-picking

Cherry-picking means counting only the examples that worked. When you scroll a chart, your eyes jump to the beautiful winners and slide past the ugly ones. The fix: write down every time your setup appears, even the ones that look bad.

2. Overfitting

Overfitting means adding so many rules, tuned so exactly to the past, that the strategy only fits the past. "Buy when RSI is below 31, but only on Tuesdays, and only if the 17-period average is rising, except in March." That might have won 95% of the time last year. It's like a student who memorized last year's test answers. Give them a new test and they fail. Simple rules with few settings usually hold up better.

3. Look-ahead bias

Look-ahead bias means using information you couldn't have known at the time. Example: "Buy at the open if today closes green." At the open, you don't know how the day will close! Always ask: "At the exact moment I got in, could I have known this?"

4. Forgetting fees and slippage

Real trades cost money. There's the broker's commission, and there's slippage, the difference between the price you wanted and the price you got. Say your backtest made $8 per trade. If each real trade costs $5 in fees and $5 in slippage, you actually lose $2 per trade.

5. Too few trades

Flip a coin 10 times and you might get 7 heads. That doesn't make it a magic coin. Ten trades prove almost nothing. Many traders want at least 50 to 100 trades before trusting a test, and more is better.

6. Testing on one period only

Markets have moods: trending years, choppy years, calm years and wild years. A strategy tested on one rising year has only seen one kind of weather. Test it across different kinds of markets.

Keep some data locked away

The best defense against overfitting: before you start, split your history in two.

• The in-sample part is where you build and tweak your rules.
• The out-of-sample part stays locked. Don't peek. When your rules are finished, test them on it once.

If the strategy only works on the part you tuned it on, it was overfit:

The overfit strategy looks perfect where it was tuned, then falls apart. The simple one is bumpier, but it keeps working. Bumpy and real beats smooth and fake.

Remember

• A backtest checks your rules on past data before you risk money.
• Record every setup, not just the pretty ones, and never use information from the future.
• Include fees and slippage, test lots of trades, and test different kinds of markets.
• Keep out-of-sample data locked away and test on it once, at the end.