Is Your Backtest Lying to You? How to Spot Overfitting

The most dangerous thing in systematic trading isn’t a losing backtest. It’s a gorgeous one. A smooth, rising equity curve is the easiest thing in this field to manufacture by accident — and it’s what empties the most accounts. Here’s how I decide whether a strategy is real or just curve-fit, after building and stress-testing more than 2,000 of them.

What overfitting actually is

Overfitting is when your strategy has learned the noise in your historical data instead of a real, repeatable edge. It looks brilliant on the data it was built on and falls apart the moment it meets anything new — because there was never a genuine pattern, just a story your optimizer told you.

The tells I look for

Too many parameters. Every knob you turn is another chance to fit noise. When a strategy needs a dozen finely-tuned inputs to look good, it’s usually memorizing history. My viable strategies tend to run on very few parameters, each of which I can explain why it exists.

No out-of-sample proof. This is the big one. If a strategy has only ever been judged on the same data it was built on, you know nothing about it. Real validation means testing on fresh data the strategy has never seen — I cover exactly how in how I validate a strategy before real money.

A tiny trade sample. Fifty trades with a big average win can look amazing and mean nothing. I want hundreds of trades across years before I believe anything.

It only works on one market. A genuine edge usually survives, at least partially, on related markets. A strategy that works on exactly one symbol and nowhere else is a red flag, not a specialty.

No execution costs. A backtest run without realistic commission and slippage is fiction. Add costs per market, on both entry and exit, and a surprising number of “winners” disappear.

The honest test

Here’s the uncomfortable question I make every strategy pass: if I showed you only the out-of-sample portion — the trades on data that didn’t exist when the strategy was built — would you still trade it? If the answer is no, the backtest was lying, however pretty it looked.

Want to know whether your own process is exposed to this? The free Trading System Health Check scores exactly these risks in two minutes. And the full validation framework is in The BreakOut Trading Revolution.