In the October issue of Futures magazine, author Jean Folger discusses an important aspect in selecting two or more indicators when developing a trading system. While I don't recommend simply combining indicators to create a trading system, and I don't think that's what Folger is suggesting either, when there comes a time to introduce two or more technical indicators to a trading system, this is when Folger's advice is relevant. The author highlights a common mistake when selecting two or more indicators that could really hinder the performance of your system. By following Folger's advice you can multiply the effectiveness of your system by selecting two or more indicators when done properly.

Types of Indicators

When it comes to technical indicators we are talking about mathematical formulas that are applied to price or volume. These technical indicators include MACD, Moving Averages, Stochastics, ADX, ATR, CCI, and many others. Folger first organized these indicators into different categories based upon what they are measuring.

• Trend - ADX, Moving Averages, MACD, Parabolic SAE
• Momentum - CCI, RSI, Stochastics
• Volatility - ATR, Bollinger Bands, Standard Deviation
• Volume - Chaikin Oscillator, OBV, Rate of Change

Selecting Two Indicators

When it comes to selecting two indicators the mistake can be from selecting two from the same category. By selecting from the same category you are measuring the same market characteristics (Trend, Momentum, Volatility, or Volume). In this case you're not getting new information about the market. For example, if you select ADX and Moving Average you are simply looking at the trending characteristics of the market. I'm a believer in keeping things simple and if you are introducing two indicators that are telling you the same thing, this is not helpful and it needlessly complicates your trading system. Each indicator should be dedicated to a specific purpose, not telling you the same thing two different ways. The point is to look at different market characteristics to expand your view. This can be done by selecting two indicators from different groups, say from Trend and Momentum. Now you are gathering complementary information about the market and are better prepared to make a decision.

Example

An example strategy will make this concept even more clear. I'll take Folger's lead and create a similar strategy used in the original article. Let's create a simple strategy for the S&P E-mini futures market. We'll use a daily chart just to keep things simple. No slippage or commissions will be deducted. Entry signals will generate with the stochastic indicator move out from its overbought/oversold regions. The system will simply reverse its current position thus, we are always in the market.

• Go Long when the SlowD line crosses above 20
• Go Short when the SlowD line crosses below 80

On August 31, 2015 an indicator called “Death Cross” appears on the S&P cash index. This has not happened since 2011. The Death Cross is simply when a 50-period simple moving average (SMA) crosses under a 200-period SMA on a daily chart. Such a signal is believed to warn of upcoming bearish market activity. The opposite signal is called a “Golden Cross”, and is alleged to warn of bullish market activity. But does the Death Cross or the Golden Cross hold any merit? What can we learn from these signals and how can we use this information in our trading? In this article I’m going to demonstrate why the Death Cross is important and why its opposite signal, the Golden Cross, may be even more important.

Both the 50-period SMA and the 200-period SMA are well known moving averages. Thus a lot of eyes are watching price around these levels. It’s no surprise when these moving averages cross the event and might be worth noting.  One way to test how well the Death Cross and Golden Cross perform historically  is to create a simple trading system based upon these signals. Using TradeStation’s EasyLanguage coding language it’s simple to create a trading system that is always in the market switching between a long position and a short position based upon a moving average crossover. Here is what the entire trading system code looks like:

In the September 2012 issue of Futures magazine, author Neil Rosenthal began a multi-part series on system development. During the first series Neil uses MetaTrader 4 (MT4) to code a simple system and demonstrates how he uses Excel to analyze the results as the first step to building a trading system. After discovering a market edge – what I call a key concept – Neil demonstrates his process of finding an initial hard stop value for his system. I found his method similar to what I use. I thought it would be helpful if I recount his method here.

While many people focus on the specifics of a trade entry, trade exit is also vitally important. After you have discovered what you think is a solid “key concept” the next question you may wish to explore is: Where should the initial stop be placed? Within Neil’s article he calls this his ISL (initial stop loss).

The Trading System

Because we are not focusing on a specific trading system I’m going use Neil’s idea of using a random entry method. That is, upon the open of the market a virtual coin will be flipped. If the coin comes up heads, we go long. If it comes up tails, we go short. This will take the focus off the trading system and place the emphasis on the true point of this article, how to determine where to place your ISL. Indeed, this process is applicable to all system trading development.

I’m going to trade this on the Euro currency futures market and will use the 830 open (Central) as the time to open a new trade and the 1500 close (Central) to close the trade. Trades will be executed and managed on a 5-minute bar chart. Below is the TradeStation code that will act as the foundation of our trading system. If you are familiar with the steps I use to develop a trading system, you will recognize this stage of development as the “Baseline” system. This Baseline system will act as our stake-in-the-ground or reference point to compare the modified version against.

The fundamental idea behind predictive modeling is indicators may contain information that can be used to predict a forward looking variable, called a target. The task of a predictive model is to find and exploit any such information.

Date                           Trend        Volatility          Day_return

19950214                 0.251          1.572                   0.144

19950215                 0.101          1.778                   0.055

19950216                -0.167          2.004                  -0.013

...

Suppose we provide several years of this data to a model and ask it to learn how to predict day_return, the one day forward return, from two indicators, one called trend and the other volatility. In the lingo of machine-learning this process is called model training. Then, we may at a later date calculate from recent prices that trend=0.225 and volatility=1.244 as of that day. The trained model may then make a prediction that the target variable day_return will be 0.152. (These are all made-up numbers.) Based on this prediction that the market is about to rise substantially, we may choose to take a long position.

Converting Predictions to Trade Decisions

Intuition tells us that we should put more faith in extreme predictions than in more common predictions near the center of the model’s prediction range. If a model predicts that the market will rise by 0.001 percent tomorrow, we would not be nearly as inclined to take a long position as if the model predicts a 5.8 percent rise. This intuition is correct, because our research has shown in general there is a large correspondence between the magnitude of a prediction and the likelihood of success of the associated trade. Predictions of large magnitude are more likely to signal profitable market moves than predictions of small magnitude. The standard method for making trade decisions based on predicted market moves is to compare the prediction to a fixed threshold. If the prediction is greater than or equal to an upper threshold (usually positive), take a long position. If the prediction is less than or equal to a lower threshold (usually negative), take a short position. The holding period for a position is implicit in the definition of the target. This will be discussed in detail in our book Statistically Sound Machine Learning for Algorithmic Trading of Financial Instruments (SSML) . It should be obvious that the threshold determines a tradeoff in the number of trades versus the accuracy rate of the trades. If we set a threshold near zero, the magnitude of the predictions will frequently exceed the threshold, and a position will be taken often. Such trades carry a relatively high rate of failure. Conversely, if we set a threshold that is far from zero, predicted market moves will only rarely lie beyond the threshold, so trades will be rare but have a relatively high success rate. We already noted that there is a large correspondence between the magnitude of a prediction and the likelihood of a trade’s success. Thus, by choosing an appropriate threshold, we can control whether we have a system that trades often but with only mediocre accuracy, or a system that trades rarely but with excellent accuracy.

TSSB automatically chooses optimal long and short thresholds by choosing them so as to maximize the profit factor for long systems and short systems separately. Profit factor, a common metric of trading system performance is the ratio of total gains on successful trades to total loses on failed trades. In order to prevent degenerate situations in which there is only one trade or very few trades, the user specifies a minimum number of trades that must be taken, either as an absolute number or as a minimum fraction of bars. In addition, TSSB has an option for using two thresholds on each side (long and short) so as to produce two sets of signals, one set for ‘normal reliability’ trades, and a more conservative set for ‘high reliability’ trades. Finally, in many applications, TSSB prints tables that show performance figures that would be obtained with varying thresholds.

Computation of thresholds and interpretation of trade results based on predictions relative to these thresholds are advanced topics that will be discussed in detail in SSML. For now, the reader needs to understand only the following concepts:

• The user specifies indicator variables based on recent observed history and target variables that portray future price movement
• TSSB is given raw historical market data (prices and perhaps other data, such as volume) and it generates an extensive database of indicator and target variables. One or more models are trained to predict the target given a set of indicators. In other words, the model learns to use the predictive information contained in the indicators in order to predict the future as exemplified by the target.
• Every time a prediction is made, the numerical value of this prediction is compared to a long or upper threshold. If the prediction is greater than or equal to the long threshold, a long position is taken. Similarly, the prediction is compared to a short or lower threshold, which will nearly always be less than the long threshold. If the prediction is less than or equal to the short threshold, a short position is taken.
• The holding period for a position is inherent in the target variable. This is discussed in detail in SSML.
• TSSB will report results for long and short systems separately, as well as net results for the combined systems.

Testing the Trading System

TSSB provides the ability to perform many tests of a predictive model trading or filtering system. The available testing methodologies will be discussed in detail SSML. However, so that the reader may understand the elementary trading/filtering system development and evaluation presented in the next chapter, we now discuss two general testing methodologies: cross validation and walkforward testing. These are the primary standards in many prediction applications, and both are available in TSSB in a variety of forms.

The principle underlying the vast majority of testing methodologies, including those included in TSSB, is that the complete historical dataset available to the developer is split into separate subsets. One subset, called the training set or the development set, is used to train the predictive model. The other subset, called the test set or the validation set, is used to evaluate performance of the trained model. (Note that the distinction between the terms test set and validation set is not consistent among experts, so the increasingly common convention is to use them interchangeably. The same is true of training set and development set.)

The key here is that no data that takes part in the training of the model is permitted to take part in its performance evaluation. Under fairly general conditions, this mutually exclusive separation guarantees that the performance measured in the test set is an unbiased estimate of future performance. In other words, although the observed performance will almost certainly not exactly equal the performance that will be seen in the future, it does not have a systematic bias toward optimistic or pessimistic values. Having an unbiased estimate of future performance is one of the two main goals of a trading system development and testing operation. The other goal is being able to perform a statistical significance test to estimate the probability that the performance level achieved could have been due to good luck. This advanced concept is beyond the scope of this brief overview but its is discussed in depth in  SSML.

In the earliest days of model building and testing, when high speed computers were not readily available, splitting of the data into a training set and a test set was done exactly once. The developer would typically train the model using data through a date several years prior to the current date, and then test the model on subsequent data, ending with the most recent data available. This is an extremely inefficient use of the data. TSSB makes available both cross validation and walkforward testing. These techniques split the available data into training sets and test sets many times, and pool the performance statistics into a single unbiased estimate of the model-based trading system’s true capability. This extensive reuse of the data for both training and testing makes efficient use of precious and limited market history.

Walkforward Testing

Walkforward testing is straightforward, intuitive, and widely used. The principle is that we train the model on a relatively long block of data that ends a considerable time in the past. We test the trained model on a relatively short section of data that immediately follows the training block. Then we shift the training and testing blocks forward in time by an amount equal to the length of the test block and repeat the prior steps. Walkforward testing ends when we reach the end of the dataset. We compute the net performance figure by pooling all of the test block trades. Here is a simple example of walkforward testing:

1) Train the model using data from 1990 through 2007. Test the model on 2008 data.

2) Train the model using data from 1991 through 2008. Test the model on 2009 data.

3) Train the model using data from 1992 through 2009. Test the model on 2010 data.

Pool all trades from the tests of 2008, 2009, and 2010. These trades are used to compute an unbiased estimate of the performance of the model.

The primary advantage of walkforward testing is that it mimics real life. Most developers of automated trading systems periodically retrain or otherwise refine their model. Thus, the results of a walkforward test simulate the results that would have been obtained if the system had been actually traded. This is a compelling argument in favor of this testing methodology.

Another advantage of walkforward testing is that it correctly reflects the response of the model to nonstationarity in the market. All markets evolve and change their behavior over time, sometimes rotating through a number of different regimes. Loosely speaking, this change in market dynamics, and hence in relationships between indicator and target variables, is called nonstationarity. The best predictive models have a significant degree of robustness against such changes, and walkforward testing allows us to judge the robustness of a model.

TSSB’s ability to use a variety of testing block lengths makes it easy to evaluate the robustness of a model against nonstationarity. Suppose a model achieves excellent walkforward results when the test block is very short. In other words, the model is never asked to make predictions for data that is far past the date on which its training block ended. Now suppose the walkforward performance deteriorates if the test block is made longer. This indicates that the market is rapidly changing in ways that the model is not capable of handling. Such a model is risky and will require frequent retraining if it is to keep abreast of current market conditions. On the other hand, if walkforward performance holds up well as the length of the test block is increased, the model is robust against nonstationarity. This is a valuable attribute of a predictive-model based approach to trading system development. Look at Figure 1 which depicts the placement of the training and testing blocks (periods) along the time axis. Figure 1 above shows two situations.