Finding Out What Works, And What Doesn’t Work

Many traders who try system trading have previously had difficulty at discretionary or manual trading. Most of these folks eventually recognize the benefit of trading a system with well defined rules – a system that has performed well in the past. It is nice to know a trading approach has historically worked, but as with all things related to trading, past performance is no guarantee of future results.

Unfortunately, many people who try systematic/algorithmic/mechanical/rule-based trading for the first time bring along a lot of the baggage that they have acquired from their previous method. Depending on the pre-conceived notions they bring into mechanical trading, these new systematic traders may run into a lot of frustration and trouble.

Many times, for example, traders will always test with a few core concepts, such as always closing by the end of the day. This is what they were used to as a discretionary or manual trader, and therefore they never even think to test ideas out of their old comfort zone. Perhaps removing these “comfort” rules would dramatically improve performance.

In this article, I will examine three common items that new systematic traders test, and see how these items actually work when they are subjected to rigorous testing.

Ground Rules

In all the testing that follows, I will test in 7 different markets, across a range of commodities:

• Wheat (W)
• 10 Year Treasury Notes (TY)
• Lean Hogs (LH)
• Australian Dollar (AD)
• Heating Oil (HO)
• Cotton (CT)
• e-mini Nasdaq (NQ)

I picked these at random, one from each major market group. The test period will be from 1/1/2007 to 12/31/2011, a 5-year period that includes some quiet, and some very chaotic markets.

Finally, I will assume $5 round turn commissions, and $30 round turn slippage. The $30 slippage might be excessive, but I’ve always found it is better to be conservative than to underestimate slippage. I’d rather not be disappointed in real time with larger than expected slippage.

The System

I will use an extremely simple strategy for the tests I am running: a simple breakout based on closing prices. The system is always long or short, and will enter at the next bar open, long if the close is the highest close of the past X bars, and short if the close of the lowest close of the last X bars. One version of the system always exits at the end of the day, and the other version is a swing system. X will be varied from 5 bars to 100 bars.

Here is the code for the swing version, Strategy A:

Here is the code for the day trading version, Strategy B:

Day, or Night?

Since most markets these days run nearly 24 hours, we can run into issues when testing historically, as many markets traded “pit” hours years ago. Because of that, and because most of the volume is during these traditional pit hours, I will use the old pit session times, but still use electronic trading data.

Exit 1 Minute Before Close

If you use Tradestation to test, you may be familiar with their keyword “setexitonclose.” This is a neat function for backtesting, but in live trading the order is sent after the market is closed, rendering it ineffective. So, I set up the custom sessions above to exit 1 minute before the old “pit” closing time. Then, I know my strategy will exit properly, and will therefore work in backtest and in real time.

Final Pre-Test Info

Now that we have everything properly setup, it is time to run some tests. For each instrument given above, I will run tests with 5, 15, 30, 60, 120, and Daily bars. That gives us 7 instruments x 6 bar sizes = 42 tests. Then, for each test, I will run the breakout length X from 5-100 in steps of 1, which yields 96 iterations per test. Since I am running two strategies overall, I am running 42 x 96 x 2 = 8,064 unique performance sets.

For each of the 42 tests for each strategy, I will record the best Net Profit (out of the 96 iterations) and its corresponding maximum drawdown (if best Net Profit is greater than zero). For each of the 42 tests, I will also record the percentage of iterations that are profitable.

All the tests being run are shown graphically in Figure 3.

Questions To Answer

Once all the tests are run, I will see if I can answer the three questions below. These questions are directly related to the desires of many discretionary day traders.

1. Many traders love strategies that are the same across multiple instruments. An example of this is price action trading – many feel the principles of price action hold across all instruments. So, when these traders test algorithms, one demand is that a strategy is viable ONLY if it is profitable across many instruments. Question: Is multiple market profitability a reasonable requirement?
2. Most day traders need to be out by the end of the day. ** Question: Does forcing an exit at the end of each day improve or decrease strategy performance?**
3. Many day traders try to go for the smallest (shortest) bar size possible. The theory is that trades will be more frequent, losses will be smaller, and exits will be more responsive to market conditions. Question: What kind of impact does bar size have on performance?

Before I get to the results, I should point out that this is one study, with two strategies, across only seven markets. So, the conclusions I reach may not hold up over all markets, and may be different for different strategies. I’m guessing, though, that the conclusions probably do hold in general.

Results

Results of all 8,046 performance tests are shown in Table 1 for the swing Strategy A, and Table 2 for the intraday Strategy B. I will refer to these results as I discuss each of the three questions.

Strategy A- Breakout, Swing Trading

$ Thousands ($K) Maximum Net Profit, 2007-2011

$ Thousands ($K) Maximum Drawdown for Max Net Profit case, 2007-2011

% of iterations Profitable (100%=goal) 2007-2011

Tables- I

Strategy B- Breakout, Exit End Of The day

$ Thousands ($K) Maximum Net Profit, 2007-2011

$ Thousands ($K) Maximum Drawdown for Max Net Profit case, 2007-2011

% of iterations Profitable (100%=goal) 2007-2011

Tables- II

1. Is profitably across multiple instruments a reasonable requirement?

Figures 4 and 5 show the best case Net Profit for both the swing and intraday version of the strategy. With one notable exception (Heating Oil intraday trading), what is good in one market is generally good in another market. Although the maximum profit varies wildly from market to market, all the swing markets are profitable, and all but one of the intraday markets are unprofitable. This should give you some confidence that the strategy is sound (or not). Of course, it doesn’t mean the strategy is tradable in each market, since I am looking only at maximum net profit based on optimization. But, clearly the intraday Strategy B is not viable in most markets.

So, the results show that profitably across multiple markets is possible, even with markedly different instrument, suggesting that is indeed a valid requirement.

2. Is exiting end of day (intraday trading) a good idea?

Figure 5 shows the answer loud and clear – the answer is no. You’ll get more profit by swing trading. Unfortunately, it might also mean enduring more drawdowns, but think of it like this: the market “pays” people for holding overnight and weekends, and taking on that risk.

3. Are shorter timeframes better?

Since Strategy A is clearly the better strategy, I will use those results to look at the impact of bar length (timeframe). This is shown in Figure 6, where I look at the percent of iterations that were profitable for each combination of instrument and bar size. So, for example, Wheat with 5-minute bars shows that 62% of iterations were profitable (denoted with a red circle in Figure 6). This means that as I varied the breakout length from 5 to 100 in increments of 1 (96 total iterations), 60 of those runs resulted in a profitable end result. Obviously, the best case is 100% – where no matter what value you use for breakout length, the end result is positive.

Results of Figure 6 show that in general profitability increases as the bar length increases, although this effect is not very pronounced, and it does not hold for all instruments. This is confirmed by the results of Figure 4, where the maximum Net Profit generally increase with increasing bar period, but not dramatically.

Why is this so? Profitability could be better as bar period increases, since random market noise plays a smaller role in longer period bars (i.e., the true trend is more easily seen in larger period bars). Of course, drawdown should also be considered when looking at bar size, but in this study it does not seem to change greatly with bar size.

Conclusion

Results with this simple strategy lead to three conclusions:

1. Striving for profit in multiple markets, as a confirmation of a strategy, is indeed possible.
2. Swing trading is likely more profitable than intraday trading.
3. Longer timeframes are generally superior to short time periods, but this is not a major effect.

Follow On

Of course, I made these conclusions based on one study. What if the strategy was different? What if the timeframes or markets were different? What if different years were used for the test period? Will the conclusions reached here still hold? I’ll examine those questions in Part 2.

If you would like to learn more about building trading systems be sure to get a copy of my latest book, Building Winning Algorithmic Trading Systems.

— Kevin J. Davey of KJ Trading Systems