A backtest is only honest if three conditions hold: it's tested on data it has never seen, real trading costs are included, and the rules are fixed before you look at the results. Most backtests that look amazing fail live because the trader peeked — tweaking rules until the equity curve looked good on the same data used to build the strategy. That is curve-fitting, not testing. In this article, you'll learn the five traps that make backtests lie, a clean workflow you can follow, and a worked example of a great-looking strategy falling apart on unseen data.
What a backtest can — and can't — tell you
A backtest answers one question: "If I had traded these exact rules over this past period, what would have happened?" That tells you whether your rules are coherent, how much heat the strategy takes in losing streaks, and what drawdowns to expect.
What it cannot tell you is what will happen next. Markets change: volatility regimes shift, and a setup that worked for two years can go quiet for the next two. A backtest gives you a range of plausible outcomes, not a promise. Treat a smooth, ever-rising equity curve with suspicion — real strategies have ugly patches, and a curve without them usually means something was massaged.
The five traps that make backtests lie
Almost every misleading backtest falls into one of these traps, and each one flatters the result in a different way.
| Trap | How it sneaks in | The fix |
|---|---|---|
| Look-ahead bias | Using information that wasn't available at the time of the trade — e.g. entering on a candle's close as if you knew it mid-candle, or using a revised indicator value | Only use data that existed at the decision point; enter on the next candle's open after a signal |
| Overfitting (curve-fitting) | Adding rules and filters until the backtest looks great on one dataset | Keep rules simple; test on unseen data (see below) |
| Survivorship bias | Testing only on assets that survived — e.g. today's index members, or pairs still listed | Include delisted or dropped instruments, or accept the bias and note it |
| Ignoring costs | Leaving out spread, commission, swap, and slippage | Add realistic costs to every trade before judging the result |
| Cherry-picked periods | Testing only on a stretch where the strategy happened to work | Test across different market conditions: trends, ranges, high and low volatility |
Look-ahead bias is the sneakiest of the five, because it often comes from the testing tool itself rather than from the trader. If your software fills orders at the close of the signal candle, you are trading with knowledge of that candle's final price — something a live trader never has. The fix: assume your fill happens on the next candle's open, or worse, and see if the strategy still holds up.
Overfitting deserves a longer look — it's the trap most traders walk into willingly. You test a strategy, it loses, so you add a filter. It still loses, so you tweak the indicator settings. Eventually the curve turns green — and you feel like you've found something. What you've actually done is tailor the rules to the random noise in that dataset. The strategy memorized the past instead of learning a pattern that repeats.
A clean backtest workflow
Follow this order — each step protects you from a specific trap.
- ☐ Write the rules down before testing — entry, exit, stop loss, take profit, position size, and which sessions or pairs you'll trade
- ☐ Split your test period: roughly 70% for building (in-sample), 30% held back for validation (out-of-sample)
- ☐ Add realistic costs first: spread, commission, and a slippage allowance on every trade
- ☐ Run the in-sample test and record the result without tweaking anything yet
- ☐ If the in-sample result is poor, change the idea itself — not the settings — and start over
- ☐ Run the untouched out-of-sample period with the final rules, no changes allowed
- ☐ Compare: if out-of-sample results are far worse, the strategy was overfit
- ☐ Test across a different market regime (a ranging year if you built it in a trending year, and vice versa)
- ☐ Only then consider a small live test with minimum size
Writing rules down first is the whole game. If you adjust a stop loss mid-test because "that one loss shouldn't count," you've stopped testing and started storytelling.
Worked example: the strategy that passed, then failed
Example: Trader A builds a simple trend strategy — buy when price crosses above the 50-period moving average, exit on a cross back below, with a fixed stop. She tests it on two years of EUR/USD hourly data.
Her in-sample test (the first 70% of the data) looks strong:
| Metric | In-sample result |
|---|---|
| Trades | 140 |
| Win rate | 44% |
| Average win | 1.9R |
| Average loss | 1.0R |
| Expectancy per trade | 0.28R |
Expectancy is the average profit per trade in units of risk: (0.44 × 1.9) − (0.56 × 1.0) = 0.28R. Over 140 trades, that compounds into a very respectable curve.
Then she runs the same untouched rules on the remaining 30% — the out-of-sample period:
| Metric | Out-of-sample result |
|---|---|
| Trades | 60 |
| Win rate | 38% |
| Average win | 1.4R |
| Average loss | 1.0R |
| Expectancy per trade | −0.09R |
The strategy went from +0.28R per trade to slightly negative. The pattern it had "learned" was mostly noise in the first dataset. This is a normal outcome, and it's exactly why the out-of-sample step exists: it caught the problem before real money did. Trader A has two honest options — a different idea, or accepting this setup doesn't hold up. What she should not do is tweak the moving average until the out-of-sample period also looks good — then it stops being out-of-sample.
If you want to dig deeper into the math behind that table, this guide to trading expectancy and profit factor walks through the same calculation with more examples.
Costs: the silent strategy killer
Costs are where many backtests quietly lie. A strategy that trades often can look profitable before costs and bleed slowly after them.
Example: Trader B's scalping strategy averages 0.4R profit per trade before costs, across 500 trades — that looks like 200R of profit. But each trade costs roughly 0.15R in spread and commission combined, a realistic figure for many retail forex setups. After costs, the edge is 0.25R per trade, not 0.4R. If slippage adds even 0.1R on fast entries, it drops to 0.15R — less than half of what the raw backtest showed.
Always subtract costs before celebrating. If a strategy only works with zero costs assumed, it doesn't work.
Position size interacts with costs too — a strategy tested with one risk level can behave very differently at another. If you haven't settled your risk per trade yet, read this breakdown of how much to risk per trade before finalizing your backtest settings.
How many trades is enough?
There is no magic number, but there are sensible floors. A backtest with 15 trades tells you almost nothing — one or two lucky winners can dominate the result. Aim for at least 100 trades before taking a strategy seriously, and prefer 200 or more if it trades infrequently. More trades mean the win rate and average win/loss figures stabilize, and a single outlier stops moving the whole result.
Also watch the time span, not just the trade count. A hundred trades in one trending month test one market condition; a hundred across two years test several. You need both: enough trades for statistical weight, and enough calendar time for regime variety.
One more check that costs nothing: look at your worst losing streak in the backtest, then imagine living through it with real money. If a 12-trade losing streak would make you abandon the strategy, the backtest just told you something about fit — not the strategy's math, but whether you can actually trade it. Many traders quit sound strategies during normal drawdowns, then blame the strategy.
A common mistake and how to fix it
The mistake: testing a strategy on one pair, in one year, with settings tuned until the curve looks good — then going live with full size the next week.
The fix is slower but cheaper than the alternative:
- Re-run it on at least two other pairs or instruments you didn't tune it on. If it collapses everywhere else, it was tuned to one dataset.
- Run a walk-forward test: optimize on one window, test on the next, then roll both forward and repeat. Consistent results across windows are worth far more than one great result on one window.
- Forward-test with minimum size for a month. Live fills, real spreads, real emotions — paper results never fully replicate these.
- Journal every live trade and compare it against the backtest's expectations — win rate, average win, drawdown. If live results drift far, pause and investigate before sizing up.
Discipline after the backtest matters too: abandoning the rules at the first losing streak is the same pattern as revenge trading, where frustration, not the plan, makes decisions. If you catch yourself changing rules mid-drawdown, that's a journaling problem, not a strategy problem.
FAQ
How is backtesting different from forward testing?
Backtesting runs your rules over past data; forward testing runs them live (or on demo) as new candles form. Backtesting is fast and cheap but can't replicate real fills, slippage, or emotions. Forward testing is slower but honest about execution. Use both: backtest to filter ideas, forward-test to confirm them.
Can I backtest manually instead of using software?
Yes, and for many traders it's the better first step. Scroll through a chart bar by bar, covering the right side, and log each setup as if trading it live. It's slow, but it forces you to confront every ambiguous setup. A journal like Tragene Journal works well for logging these manual backtest trades alongside your live ones.
What is walk-forward testing?
It's a rolling in-sample/out-of-sample split: optimize on one window, test on the next, then roll both forward and repeat. Consistent results across many windows mean far more than one great result on a single period.
How do I avoid overfitting without complicated statistics?
Keep the rules simple, limit the filters, and always validate on data you didn't use while building the strategy. If a rule only improves one specific period and hurts others, drop it. A strategy with three rules that works across regimes beats one with twelve rules that works on one dataset.
Should I backtest on multiple timeframes?
Testing on other timeframes is a useful robustness check, but not a substitute for out-of-sample testing. A strategy tuned on the 1-hour chart that also works on the 4-hour chart is encouraging — but if both used the same historical period, they share the same data. Different time periods first, other timeframes second.
My backtest looks great but live results are worse. What went wrong?
The usual suspects, in order: costs and slippage weren't modeled, the strategy was overfit to past data, or your live execution differs from the tested rules (late entries, skipped setups, moved stops). Compare your live trade log against the backtest — the gap usually shows up within the first 20–30 trades.
A backtest won't hand you a winning strategy — but it is a cheap way to kill bad ideas before they cost you money. Test honestly, validate on data you haven't touched, include every cost, and let the losers die on paper instead of in your account.
This article is for educational purposes only and is not financial advice. Trading involves risk. Author: Kunal D., Founder, Tragene Journal