One problem with traditional backtesting is that it relies on the presupposition that there are repeating predictive patterns in the market. In fact, most trading methodologies rely on this assumption. And yet we know the disclaimer that past performance is not indicative of future results.
And yet backtesting largely assumes that the future will be similar to the past. Yet, we can imagine the possibility for non-repeating but predictable profit opportunities. Even without getting into those possibilities, we can imagine that if we can model the dynamics of the market accurately, we can predict new outcomes that cannot be extrapolated from the past.
The way this is accomplished is by simulation. Simulation offers the powerful promise of allowing us to make use of historical market data under varying conditions of future similarity. Simulation, massive simulation is also poised to impact every aspect of our lives.
Imagine for a moment that you are a world class MMA fighter or boxer, and you’re competing against a similar top-ranked fighter. What should your strategy be? In the past, you might have studied your opponent and intuited a strategy. Perhaps, if you were more sophisticated you might have even used crude statistics such as counting to figure out the risk and probability of a given working move. But today, it is surely possible to feed your moves into a computer with precise timing and force calculations. Next, it is possible to infer the same regarding your opponent by using previous fight videos. In addition, by using the fighter's height, weight, and other statistics it is possible to model how well he could perform, including moves that were not recorded. Once all the data is put into the computer then you can run thousands or hundreds of thousands of fight simulations. The best performing simulations will yield the best strategies. The strategies that are discovered may be non-intuitive and completely innovative. These can be used with human cognition and consideration as the basis for your game plan.
Now, imagine how this would work for a trader. It is not just running thousands of simulations on past data. But you must infer how future traders will react to changing market conditions. This is the difficult part because you need to know how the combination of variables will impact their behavior.
Even if that level of simulation is beyond the average developer's capability or can only provide rough approximations due to the difficulty in modeling, it is still possible to start thinking more along the lines of simulation to explore creative opportunity and risk management.
Some ideas on how you might do this:
- Use random and partially randomized entries, and exits to try to find more universal or robust settings for your strategies.
- Create synthetic market data where you change the amount of volatility, trend, and mean reversion to see how it might impact your strategies.
- Create models of how traders might act in certain scenarios and look for situations that might offer predictive advantage.
- Use Monte Carlo analysis with randomized entries to come up with pessimistic capital requirements.
- Try to find optimal strategies for given market conditions.
- Build self-learning strategies with limited capacity for memory and try to find the optimal rules for trading.
–by Curtis White from blog, Beyondbacktesting