No ads. No sponsors. Self-funded. Built for you, not advertisers.
Ghost Man Encyclopedia · Risk & testing

What is backtesting, and how does survivorship bias distort it?

By Ghost Man Trades · Last reviewed · Education only, not advice

Short answer. Backtesting is testing a trading rule by replaying it on historical prices. Survivorship bias is a common flaw: if the test only uses companies that still exist, it leaves out the ones that went bankrupt or were delisted, so results look better than reality. Every backtest is hypothetical and guarantees nothing about the future.

Backtesting in plain English

A backtest asks: "If I had followed these exact rules in the past, what would have happened?" You write down the rules precisely (when to buy, when to sell, how much), run them over years of historical prices, and count the results: number of trades, win rate, average gain and loss, worst drawdown.

Done carefully, it is the best way to find out whether an idea had any edge at all before risking money. Done carelessly, it produces beautiful numbers that fall apart in real trading. The most common reason is not bad math. It is bad data.

Survivorship bias

Suppose you backtest a strategy on the stocks in an index today, going back 20 years. Every one of those stocks survived 20 years, and many of them grew enough to join the index. The companies that went bankrupt, got delisted or shrank out of the index are missing. Your test only ever "bought" winners, because the losers were removed from the list before you started.

That is survivorship bias, and it is well documented. Researchers studying mutual funds found that databases that drop dead funds overstate average performance, because poorly performing funds are the ones that tend to close.

Example (hypothetical numbers).

  • In 2006, 100 stocks meet your rule. Over the next 20 years, 80 survive and return an average of +150%. 20 fail or get delisted, losing an average of 80%.
  • A backtest using only today's survivors sees 80 stocks and reports an average of +150%.
  • The real average across all 100 stocks: (80 × 150% + 20 × −80%) ÷ 100 = (12,000 − 1,600) ÷ 100 = +104%.
  • Same rule, same years. The survivor-only test overstated the result by 46 percentage points, and it never showed you a single wipeout.

Other backtest traps

  • Look-ahead bias: using information that wasn't known yet, like buying at today's close on a signal that only exists after the close.
  • Overfitting: adjusting settings until the past looks perfect. More rules and more tuning usually mean worse real results.
  • Ignoring costs: commissions, bid-ask spreads, slippage and taxes all reduce real returns.
  • Too few trades: ten trades can look great by luck.
  • One market regime: a test that only covers a long bull market says little about a bear market.

Common mistakes when reading someone else's backtest

  • Not asking which stocks were included. If the answer is "today's index," assume survivorship bias.
  • Looking only at win rate. Check the size of losses and the worst trade too. See win rate vs expectancy.
  • Treating hypothetical returns as a promise. In the U.S., registered investment advisers face specific SEC requirements before they can advertise hypothetical performance, for good reason.

How Ghost Man uses this

Every signal record on this site is a backtest of the Ghost Man rules on one stock and one chart: buy at the next open after a buy signal, sell at the next open after a take-profit signal, long only, no stop loss. Trades are filled at the next open, not at the signal's close, to avoid look-ahead bias. We list every trade so anyone can check it.

Here is the limitation we want you to know: the records are calculated on stocks that are listed and trading today. Companies that went bankrupt or were delisted are not in our universe, so our records carry survivorship bias, and the true historical results of the same rules across all stocks would likely be worse. The records also leave out commissions, spreads and taxes, and the open trade is shown separately rather than counted. Treat them as evidence about how the rules have behaved, not as a forecast.

Related terms

Learn it step by step in Swing School

Frequently asked questions

Is a backtest the same as real trading results?

No. A backtest is hypothetical: it shows what rules would have done on past data, with no real money, emotions or execution problems. Real results usually come in worse.

How do you avoid survivorship bias?

Test on a historical universe that includes companies that were later delisted, acquired or went bankrupt, using point-in-time index memberships and prices. This data costs more and is harder to get, which is why many free backtests skip it.

What is look-ahead bias?

Using information in a test that was not available at the time of the trade, for example today's index members, later-revised earnings or a closing price before the close happened.

What is overfitting?

Tuning a strategy's settings until it matches the past almost perfectly. An overfit strategy has learned the noise in old data and usually performs much worse on new data.

How many trades does a backtest need?

There is no magic number, but a handful of trades tells you very little. Results based on dozens or hundreds of trades across different market conditions are more meaningful.

Sources

We check every definition and formula against these sources. Our own rules come from the Ghost Man scan code that runs this site.

Education only. Education only, not financial advice. Ghost Man Trades is not a registered investment adviser or broker-dealer. Past results, including historical signal records, do not guarantee future results. Trading involves risk of loss. All examples on this page are hypothetical and use made-up numbers to show the math.

All encyclopedia entries · Glossary · Swing School

See it on a live chart

The Ghost Report shows which quality stocks sit near or below their 4-year average right now. Members also get the Ghost Man indicator on their own TradingView charts.

60 days free, no card needed. Then $49/month for founding members, locked in for life. $99/month regular price later.

Start the free 60-day beta