Two problems that often result from system optimization are: 

(1) a reduction in the number of trades, a system that generates too few trades can both be more difficult to trade and a low number of trades does not inspire confidence; and...

(2) there is a selectivity or specialization risk that, by sheer bad luck, the specific chosen parameter values will under perform in the future even while the basic system continues to work across most other parameters.

It is thought that hedge funds often run classes of similar systems to avoid the risk of simply choosing a set of unlucky parameters but individual futures traders may not have the capital to deploy a class of similar systems for essentially a single trading concept.

We present the concept of signal multiplexing as a method to increase the number of trades which offers the possibility of greater confidence and profits– while also providing the added bonus of reducing the specialization risk of trading a single parameter set. To demonstrate the concept, we build a RSI swing trading system for the ES but instead of optimizing for a single specific optimum: we optimize across three signals any of which can get us into or out of a trade. We chain the signals using conditional “OR statements”.

RSI-2 Standard - Click For Larger Image

RSI-2 Multiplex - Click For Larger Image

RSI-2 Standard - Click For Larger Image

RSI-2 Multiplex - Click For Larger Image

Every system developer has a list of rules and indicators that they want to test out. The problem every developer faces is narrowing down that list of rules and indicators to the select few that are best worth focusing on. A quite similar and related problem that every developer has to deal with is having some inputs or indicators that they believe to be relevant and important but not knowing exactly in what way they are significant. I will show you how you can quickly find the best rules and indicators to focus on using an automated search approach which will work on most traditional retail trading platforms (such as Tradestation, Multicharts, Ninjatrader, Amibroker, etc.). But first I should point out that there are two basic approaches to developing systems.

The manual method also known as the hypothesis and test method (guess and check), quite similar to the scientific method, and the generative method also often thought of as the “data mining” approach. It seems that most professionals use the scientific method because the generative method lacks sufficient control over the process. The problem with the traditional method is that it can be really slow and self-limiting.

Computers today are tremendously powerful and fast, and yet the system developer using the traditional approach derives very little benefit from all that increased computing power and resources. Clearly, a developer who can’t utilize the advances in technology will be at a significant handicap.

Fortunately, it is not an either-or scenario because the optimizers found in retail trading platforms can be used for not just optimizing inputs but also for discovering the best trading inputs and rules to use. The key idea is to use the optimizer to find the most relevant rules or indicators. One of the first ideas that I had was to chain conditions together using operators such as “or” and “and”. This can be achieved by creating an array of rules with associated bit states which the optimizer can turn on and off.

Tradestation has the limitation that it can’t optimize Boolean fields. The solution is to simply convert everything to “0” and “1” numeric. Create a list of rules and allow the optimizer to turn them on or off.

One problem with traditional backtesting is that it relies on the presupposition that there are repeating predictive patterns in the market. In fact, most trading methodologies rely on this assumption. And yet we know the disclaimer that past performance is not indicative of future results.

And yet backtesting largely assumes that the future will be similar to the past. Yet, we can imagine the possibility for non-repeating but predictable profit opportunities. Even without getting into those possibilities, we can imagine that if we can model the dynamics of the market accurately, we can predict new outcomes that cannot be extrapolated from the past.

The way this is accomplished is by simulation. Simulation offers the powerful promise of allowing us to make use of historical market data under varying conditions of future similarity. Simulation, massive simulation is also poised to impact every aspect of our lives.

Imagine for a moment that you are a world class MMA fighter or boxer, and you’re competing against a similar top-ranked fighter. What should your strategy be? In the past, you might have studied your opponent and intuited a strategy. Perhaps, if you were more sophisticated you might have even used crude statistics such as counting to figure out the risk and probability of a given working move. But today, it is surely possible to feed your moves into a computer with precise timing and force calculations. Next, it is possible to infer the same regarding your opponent by using previous fight videos. In addition, by using the fighter's height, weight, and other statistics it is possible to model how well he could perform, including moves that were not recorded. Once all the data is put into the computer then you can run thousands or hundreds of thousands of fight simulations. The best performing simulations will yield the best strategies. The strategies that are discovered may be non-intuitive and completely innovative. These can be used with human cognition and consideration as the basis for your game plan.

Now, imagine how this would work for a trader. It is not just running thousands of simulations on past data. But you must infer how future traders will react to changing market conditions. This is the difficult part because you need to know how the combination of variables will impact their behavior.

Even if that level of simulation is beyond the average developer's capability or can only provide rough approximations due to the difficulty in modeling, it is still possible to start thinking more along the lines of simulation to explore creative opportunity and risk management.

Some ideas on how you might do this:

• Use random and partially randomized entries, and exits to try to find more universal or robust settings for your strategies.
• Create synthetic market data where you change the amount of volatility, trend, and mean reversion to see how it might impact your strategies.
• Create models of how traders might act in certain scenarios and look for situations that might offer predictive advantage.
• Use Monte Carlo analysis with randomized entries to come up with pessimistic capital requirements.
• Try to find optimal strategies for given market conditions.
• Build self-learning strategies with limited capacity for memory and try to find the optimal rules for trading.

–by Curtis White from blog, Beyondbacktesting

In my experience trading futures, I have found that most damaging losses are caused by taking consecutive losing trades, i.e. runs of losing trades or correlated loses. In this post, I share 2 simple methods and 1 advanced method to avoid consecutive losses which can be applied to your trading systems and discretionary trading.

The first method is based on time, and the concept is simple– you require a minimum time (min bars after loss) after a losing trade or trades before a new trade can be placed. The logic is that if you have a method that is normally consistent and you wait long enough after a loss that whatever was driving the market when you took the loss or your psychology has more likely played out.

The second technique is that after you have a losing trade or trades on either long or short side then you only allow your system (or yourself) to take trades that are opposite to the previous loss.

The third technique is based on a previous work of mine where I explained a method to score your winning and losing trades to learn from them. In that process, you need to acquire data on some statistically relevant number of trades. However, this new technique does not require training because it is only concerned with your most recent loss. In this approach, you need to develop a market similarity score and you record that statistic at the entry of a losing trade. Next, you wait for the similarity score to either change meaningfully before your next trade (must change by a statistically relevant or minimum amount to reset it) or to change meaningfully from your last losing trade.

While any of these technique in isolation could be used to improve your system performance, it makes sense to think about situations where they could be combined. Notice, I refer to loss or losses– the reason is that if your system normally shows a higher probability to win after a losing trade then you may want to reserve these techniques for only situations after you have 2 or more losses.

-- By Curtis White from blog beyondbacktesting

A good rule of thumb is that a reasonable risk per day for a day trader is from 2% to 5%. A moderate level of risk is around 3% of your capital. The new CME e-micros makes it possible to get started with trivial amounts of risk (from $500 to $1,000 or even less). However, the e-micros are only a small part of the futures landscape.

For day traders, the energy complex can provide good volatility– that would be products like CL (Crude Oil), HO (Heating Oil), and Natural Gas (NG). They can compliment the equity day trading e-minis like the ES (S&P 500), NQ (Nasdaq 100), and the RTY (Russel 2000).

Now, most traders know that the CME sets the margin requirements and many brokers offer very low day trading margins as low as $500 per contract. However, here I am concerned with calculating up the reasonable minimum account size for day trading to maximize the probability of success– not the absolute minimum.

I have found that for e-mini day trading that trying to trade with less than $600 risk per day can be restrictive and makes it more difficult. If we take $600 at 3% account risk then that yields $20,000 as your minimum reasonable starting capital. Of course, it can be difficult to utilize a given risk without exceeding it. As such, we might want a risk limit that is above what we anticipate and to provide for changing market conditions. Given that, a 1k daily loss limit at 3% yields $33,000 while a $1,200 risk limit at 3% would yield an account size of $40,000.

What about a more quantitative method that can also look at the individual contracts? In the method below, I calculated the notional value for the most popular day trading contracts. Next, I took the required capital to trade each contract at no more than 4x leverage. A leverage of 3x to 4x is often considered sufficient to enable meaningful returns while still allowing for adversity. However, that method does not take into account the volatility differences among contracts. As such, I developed a volatility adjusted method which is computed by taking a smoothed moving average of the Average True Range (200 SMA of the 14 ATR) and converting that into a real dollar amount (multiply by tick value), and finally calculating the capital that would be required to keep that at no more then 5% of our account. The logic is basically if we want to capture large intraday moves then it is a given that there is some associated level of precision which also, due to the fixed multiplier in futures, defines the type of capital we should be trading with.

While calculated independently, the results from the 4x leverage rule-of-thumb and the 5% ATR are mostly in agreement, and more or less support my original guesstimates.

Mean reversion systems, such as those based on RSI, suffer significant performance degradation when stop losses are applied for risk control. In previous posts, I shared how one could create more powerful RSI systems by using adaptive periodicity and multi-state regime classifiers.

The key idea for better RSI entries is that there may be a unique optimal RSI threshold for any given bar size. As to why this might be relevant, it might be capturing some component of the speed of the move, or the type of results might depend on the volatility regime.

We do not need to be concerned with the exact influence the bar size might have. For example, we might reason that a large fast move might be a better buy. On the other hand, it might also be that a large move implies greater risk of future big moves, and that we should be even more selective. The good news is that we do not need to know the precise way that the adjustment should be made. We merely need to know that there is an influence and the optimizer will do the rest.

For this sample long only S&P 500 E-mini system, I classify each trigger bar into 1 of 4 classes based on the bar’s range. For each class, I optimize a specific threshold adjustment factor which will be added to the threshold level (cross under 30) that is used for buys. The adjustment can be positive or negative. For the exit, I use standard overbought level (cross over 70) and 5 bar time-based exit.

First, we examine how the long only base RSI(14) system does. While the trade quality surprised me, the poor sample size and low overall profits were not impressive.

The traditional way to treat the RSI is to treat low RSI levels as good buying opportunities while treating high RSI levels as selling opportunities. However, we seek to gain fresh insight into the nature of RSI, with an eye toward discovering possible momentum return, by exploring the RSI using a visual quantitative approach.

Exporting And Visualizing The Data

We are interested in the next day returns of the RSI(2) for the E-mini S&P 500 (ES) futures contract. We will be using Tradestation and Excel for this analysis. For computing next day returns, we export the RSI(2) level and the close to close returns. We can export CSV data from Tradestation using the toolbox. However, I often find it convenient to print the data directly to the output window using Easylanguage, and then simply copy and paste it into Excel.

Today’s topic is on finding the optimal periodicity for the RSI indicator and the techniques should apply to other sorts of indicators as well. The good news is that it won’t require any rocket science. John Ehlers has written a lot about this topic, and his articles spurred my interest in it, which frankly I had never considered. I might should note now that I have not reviewed the actual methodology that Ehlers used in any depth but my impression was that the methods were very advanced. However, I realized I could tackle this problem using just basic statistics.

Before we start analyzing specific solutions, I want to take a moment to describe what we are actually attempting to do. The question that we are trying to answer is how do we find the optimal length to use when calculating an indicator. A poor length value may result in the RSI spending too much time in either the overbought or oversold states, or failing to reach the proper levels.

Obviously, the optimal lookback will change as market conditions change and therefore our methods should be adaptive and responsive to the market. The goal is to find a metric to rank how well behaved the RSI is. A well behaved RSI is one that spends about an equal amount of time in overbought and oversold levels. During a trending market, if the RSI gets stuck into overbought conditions then that would be an example of a poorly behaved RSI. The solution thus is simply to measure the amount of time that the RSI spends in overbought and oversold conditions.

My original inspiration came from the idea of a histogram. We simply add 1 for every bar that the RSI is above our threshold and subtract 1 for every bar that the RSI is below our threshold. The absolute value of the resulting summation is our error metric, which we seek to minimize. Key Concept: Sum the amount of time that the RSI spends above 50 and below 50, and take the difference. The absolute value of the difference is our error metric, which we seek to minimize.

One technique for optimizing systems is to create a regime filter. A most common example is a binary classifier that classifies the market into either bull or bear markets based on closing above or below the 200 day moving average. But, there are problems with binary classifiers and we will demonstrate why a multi-state classifier is often a more powerful approach. In fact, the multi-state classifier combined with optimizer is actually a basic but powerful form of machine learning.

There are some obvious problems with binary classification. First, assuming that the market only has 2 regimes is somewhat an arbitrary decision. Second, it is a rather crude tool. For example, we might find it reasonable to classify various variables into multiple classes such as bearish, bullish, and neutral. The multi-state classifier is a powerful technique that allows for such classifications and more. In the moving averaging example, we might take the difference between the close and the moving average and let that specify our trading style. This allows our system to trade adaptively to the current market conditions. And the possibilities are really extensive for application.

One benefit of multi-state classifiers over a linear regression approach is that it is easy to encode non-linear logic. For example, imagine the hypothetical scenario where we want to trade conservatively when the market is very bearish, aggressive when bearish, aggressive when bullish, and conservatively again when the market is very bullish. The idea in this case is that very bullish or bearish markets are higher risk. This is an example of a nonlinear matrix.

Simple

Multi-State