Imagine you had a time machine that could take you back to any point in your life. You could relive your experiences, but with one twist: you already know what’s going to happen. Wouldn’t it be amazing to tweak your choices and see how different actions might lead to better outcomes? What if you could avoid mistakes or optimise decisions to get the best possible result every time?
If you’re new here, you might be wondering what this has to do with finance. In our previous blog, we talked about a "Magical Diary" that records everything about you, much like how historical data captures every detail about the stock market. If you haven't read it yet, I’d highly recommend going back and giving it a read—it sets the stage perfectly for what we’re about to dive into.
Now, let’s get back to that time machine. While time travel might sound like science fiction, in the world of quantitative finance, quants have a tool that’s almost as powerful. It's called backtesting.
Backtesting is like using that magical time machine for trading strategies. It allows quants to take a strategy they've developed and test it against historical data. Essentially, they’re asking, “If I had used this strategy in the past, how would it have performed?”
But why is this so crucial? Let’s bring back our "Magical Diary" analogy. Imagine you could go back and rewrite certain moments of your life based on the knowledge you have now. For example, if you knew you’d get a cold every winter, you might start taking vitamins early or avoid certain situations that made you sick in the first place. Similarly, backtesting allows quants to see if their strategies would have “gotten sick” or “stayed healthy” during different market conditions.
Think of backtesting as a practice exam before the final test. Just like how you’d take a mock test to see how well you might do on the actual exam, backtesting simulates a trading strategy using historical market data to see how it would have performed.
For example, say you’ve developed a strategy that buys stocks when they drop by 5% and sells when they rise by 10%. Instead of jumping into the market and risking real money right away, you can backtest this strategy against years of past market data. You might find that it worked beautifully during some periods but failed during others. This insight is priceless because it allows you to tweak the strategy before deploying it in the live market.
Understanding key metrics is crucial in backtesting because they help you evaluate how well a trading strategy might perform under different conditions. Let’s dive into some of the most important metrics used in backtesting:
Drawdown: Drawdown measures the decline in your portfolio from its peak to its lowest point before it begins to recover. This metric is critical because it shows the potential risk involved in your strategy. A large drawdown can be concerning because it indicates that, during a period of losses, your portfolio could suffer significantly before bouncing back. Even if a strategy is profitable in the long run, a deep drawdown might make it unsuitable for certain risk-averse investors.
Imagine you have a strategy that, over the past five years, achieved a 20% annual return. However, during one particularly volatile year, your portfolio value dropped by 40% before recovering. This 40% drop is the drawdown. While the strategy is profitable, such a significant drop might be too risky for some investors, much like experiencing a steep fall on a rollercoaster might be too much for certain thrill-seekers.
Sharpe Ratio: The Sharpe Ratio is a measure of the risk-adjusted return of your strategy. It considers not just the returns you make but the level of risk you take to achieve those returns. A higher Sharpe Ratio means you’re earning more return per unit of risk, which indicates a more efficient use of risk. This metric helps you understand whether the returns of a strategy are worth the risks being taken.
Consider two strategies: one with a 10% return and a Sharpe Ratio of 0.5, and another with a 7% return but a Sharpe Ratio of 1.2. Although the first strategy has a higher return, the second strategy is more efficient because it takes on less risk for every percentage of return it generates. It’s like comparing two drivers—one who drives fast but recklessly and another who drives slightly slower but safely. The safer driver represents a higher Sharpe Ratio.
Profit Factor: The Profit Factor is the ratio of gross profits to gross losses, and some people also refer to it as the Risk-Reward Ratio. It tells you how many units of profit you make for each unit of loss. A Profit Factor greater than 1 indicates a profitable strategy, while a higher Profit Factor suggests a more robust strategy. This metric is important because it directly reflects the overall profitability of your strategy.
Suppose your strategy has a Profit Factor of 2. This means that for every $1 you lose, you make $2 in profit. For instance, if over a period you earned $30,000 in gross profits and incurred $15,000 in gross losses, your Profit Factor would be 2. This is similar to a business that spends $1 on marketing to generate $2 in sales revenue—clearly, a profitable and scalable approach.
Win Rate and Loss Rate: Win Rate and Loss Rate show the percentage of trades that result in a profit versus those that result in a loss. These metrics help in understanding the consistency of your strategy. However, it’s crucial to consider them in conjunction with the size of wins and losses. A strategy might have a high win rate, but if the losses are significantly larger than the wins, the strategy may still be unprofitable.
Imagine a strategy that wins 80% of the time, which sounds excellent at first glance. However, each winning trade only makes $50, while each losing trade loses $250. Over 10 trades, you might win 8 trades and make $400, but lose 2 trades and lose $500. Despite the high win rate, the strategy results in a net loss of $100. It’s like a business that makes frequent small profits but incurs occasional large losses, which can wipe out all the gains. The overall success depends not just on how often they profit, but on the balance between profits and losses.
Beta and Alpha: In finance, Beta measures a strategy’s sensitivity to market movements, while Alpha represents the excess return a strategy generates beyond what would be expected based on its Beta. A strategy with a high Alpha is considered to be outperforming the market, while Beta indicates how much of the strategy’s returns can be attributed to market movements rather than the strategy itself.
Imagine a car designed for both speed and control. Alpha is like the engine that gives the car extra speed beyond what’s expected, while Beta is like the car’s handling, which determines how well it performs under different road conditions. A strategy with a high Alpha and a Beta that matches your risk tolerance is like having a car that not only accelerates quickly (outperformance) but also handles well on various terrains (market conditions).
While backtesting is an invaluable tool, it’s important to be aware of its limitations and potential pitfalls. Here are some reality checks that every quant should consider:
One of the fundamental truths in finance is that past performance is not always indicative of future results. Backtesting relies on historical data to evaluate a strategy, but markets evolve, and the conditions that made a strategy successful in the past might not exist in the future. Relying solely on past performance can lead to overconfidence in a strategy, which may falter when market conditions change.
A strategy that performed exceptionally well during a prolonged bull market might suffer significant losses during a bear market. It’s like using an umbrella that worked perfectly in a light drizzle, only to find it inadequate during a torrential downpour. The conditions have changed, and the tool that once worked is no longer effective.
Overfitting occurs when a strategy is excessively tailored to fit historical data, capturing noise rather than actual patterns. This creates the illusion of a highly successful strategy during backtesting, but such a strategy is likely to fail in live trading because it’s not robust enough to handle new, unseen data.
A strategy that appears to perform well because it’s optimized for specific historical events might fail when applied to new data. It’s like tailoring a suit to fit perfectly in a specific pose and lighting, only to find that it doesn’t look as good in different settings. The over-optimization makes it impractical for real-world use.
Survivorship Bias
Survivorship bias occurs when backtests only consider data from entities that have survived until the end of the data period, ignoring those that failed or were delisted. This can lead to overly optimistic results, making a strategy appear more successful than it actually is.
A backtest that only includes companies still listed on a major exchange might overlook those that went bankrupt, resulting in an inflated view of the strategy’s success. It’s like studying only the most successful businesses and assuming their strategies guarantee success, while ignoring the countless businesses that followed similar paths but failed.
Look-ahead bias happens when future data is inadvertently used to make decisions in a backtest, leading to results that are not achievable in live trading. This bias can significantly distort the perceived effectiveness of a strategy.
Using quarterly earnings data released after a trade date to inform a decision in a backtest creates an unrealistic advantage, much like playing chess with knowledge of your opponent’s next move. The strategy looks brilliant, but only because it’s based on information that wasn’t available at the time.
Transaction costs, such as commissions, spreads, and slippage, are often overlooked in backtesting. Ignoring these costs can make a strategy appear more profitable than it would be in reality, especially for high-frequency or short-term strategies.
A high-frequency trading strategy might show impressive returns in a backtest that doesn’t account for transaction costs. However, once you factor in the costs of executing thousands of trades, the strategy’s profitability could disappear. It’s like running a business that seems profitable until you realize you haven’t accounted for overhead costs like rent and utilities, which significantly reduce your bottom line.
Backtesting is like having that magical time machine, offering a glimpse into how a strategy might perform before it’s actually put to the test. However, it’s not perfect—just as our "Magical Diary" analogy has its limits. The insights gained from backtesting can minimize surprises and maximise success, but it’s essential to remain aware of its limitations and avoid common pitfalls.
Thanks for Reading!
We hope you enjoyed this deep dive into the world of backtesting. If you’re curious about how we apply these strategies in real-world scenarios or have any questions, feel free to reach out. And if you haven’t already, don’t forget to check out our previous blog—it’s where the magic began!
