As I learn how to build trading systems I find that curve fitting is one of the easiest trap to fall into. Curve fitting consist of over tweaking your system until it can predict almost exactly where the stock price will go in the range you are testing. The more curve fitting you do the less it will “fit” with the future stock prices.
Here is a strategy that was tweaked too much to fit SPY from November 2008 to November 2009, you can see that buy (green dots), sell (red dots), short (yellow dots) and cover (purple dots) signals have an amazing ability to get the maximum returns (mostly when the stock is going down):
And here is what the same strategy looks like 2 years after, it became very volatile and bad returns:
Here are some rules I try to follow to avoid this:
- Reduce the number of parameters your system has, the more parameters, the more opportunities you have of falling into the curve fitting trap.
- Increase the number of trades and time range of your back tests, while this might reduce the overall performance of your system, it will also reduce the risk (draw-down), it is very challenging to find an equilibrium between returns and risk.
- Always take into consideration periods where the stock had huge uncommon gaps when back-testing as they can disproportionately affect your results. Visualize on a graph the range you are testing on.