It’s easy to find a trading model that has a positive average profit of $100 and then conclude it could be profitable into the future. But is there a metric we can use to help us predict what might happen into the future? A complicated approach would be to use the Monte Carlo method, but not everyone has access to this however, we all have access to a simple calculator. By visiting a topic in statistics called Confidence Intervals (CI) we can obtain a hint at what’s possible and perhaps find weaknesses in our seemingly profitable trading model.

The average net profit of a trading model is simply the historical P&L for each trade over a given time period. Let’s imagine a trading model that has produced 60 trades. Some of the trades are winners and some are losers. We add the total P&L together for each individual trade and divide it by the number of trades – 60 in this case – and we get $100. Clearly this is well above zero so in the long run this system appears profitable.

However, we also know that individual trades can be very different from our average profit per trade. Some trades produce much larger winning trades while others produce smaller winning trades. Still, other trades produce a range of losing trades. If we graph each trade’s P&L and then draw a line representing our average P&L we would see each individual trade falls around our mean value of $100. In other words, the P&L for any trade will vary around this mean value. We can measure this variation and use it to estimate the likelihood the system might remain profitable.

Statistically speaking, a trading model that exhibits a large standard deviation of profit per trade will have an increased chance of failing in the future. This is true even if the average mean is currently profitable. But what makes a standard deviation too large? This is explained below when we attempt to use our confidence interval to estimate a likely range of average P&L values into the future.

What we wish to do with our confidence interval is estimate with 95% confidence, if our system will probably produce a negative average P&L into the future. In other words, is it likely our seemingly profitable trading model is based upon chance? We can estimate this with our CI formula.

CI = t * SD / squareroot( N )
CI = Confidence Interval
t = T-score (we estimate value to be 2 and the reasoning behind this is beyond this article)
SD = P&L Standard Deviation for all trades
N = number of trades in our sample

With our imaginary trading model we have a $100 average net profit and 60 trades in our sample. Please note that in order for this method to work, you must have a minimum of 60 trades in your sample. Let’s also state the standard deviation for all trades is $400. With this information we can compute our 95% confidence interval.

CI = 2 * $400 / squareroot( 60 )
CI = $800 / 7.746
CI = $103.28

For simplicity let’s round the confidence interval to the nearest dollar which is $103. What do we do with this value? We create a range or band around our average net profit value of $100 by both adding and subtracting the confidence interval value.

upper band = Average Net Profit + CI = $203
lower band = Average Net profit - CI = -$3

We have now created a range of -$3 to $203 for our average net profit. What does this mean? Based on our calculation we have estimated with 95% confidence that our trading model’s average net profit could be as low as -$3 or as high as $203. The important number is the lower band because this represents a worse case situation. In our example, we have a negative value which indicates a losing model. Or at least, a potential losing scenario.

In short, our hypothetical trading model’s average net profit could be based upon chance and in the future could produce a negative P&L. Suddenly what seems like a solid system appears more shaky. Does this mean our trading model should be abandoned? Not necessarily.

In the case of the confidence interval there are two critical factors at play. Those values are the number of trades (N) and the value of the standard deviation between trades. Modification of the standard deviation can be achieved by altering the trading model logic. Modifying stops, targets, and other trading rules will change the standard deviation value. The goal would be to tighten the variation of each trade to reduce the standard deviation. This in turn, would create a smaller confidence interval. However, if you don’t want to modify the system or if you are unable to modify the system there is another way.

Our example system was based on 60 trades. This is really not a lot of trades. Let’s say we find more data to test our system and we get up to 100 trades. Let’s also pretend all the other performance factors stay the same. If we recalculate our confidence interval value we now get a value of $80. This gives us a range of $20 and $180. In this case, we have a system which produces positive value for the lower band. So maybe before we make a judgment on a system that appears borderline we should collect more trades first.

I should also point out that our imaginary trading model we are looking at has $30 deduction for each trade to account for slippage and commissions. So this negative effect is already factored into our confidence interval calculation. If we did not take into account slippage and commissions during our back-testing we would have to deduct this from our final range which would give us -$10 and $150. The impact of commissions and slippage just puts us back into negative territory gain. But we have them accounted for in our back-tested results.

As you can see having enough data points (trades) can have a significant impact on the confidence interval calculation. For system trading there are many reasons for having a large number of trades. Of course continuing to add more and more trades is not going to turn a losing system into a good system. The point here is sometimes you need to have more data before making an informed decision. If you have what you believe is a good system, yet you only have a few data points, the confidence interval calculation may be warning you to get more trades in the test sample.

Our confidence interval calculation makes some assumptions about the data. That is, the data points (trades) have a normal distribution. This of course is not necessarily correct. In many cases our trade distribution looks somewhat normal with fat tails in our distribution. Thus, we must take our confidence interval results with a grain of salt.

A bigger issue, I think is the confidence interval calculation does not indicate if this system has been curve fitted. If we have a killer system with 1,000 trades with a confidence interval range of $100 – $200 that’s great. However, it’s pointless if the system is curve fitted to the historical data and there is no way our confidence interval calculation can tell us. So, what do we do?

Ideally you would validate your new trading model on out-of-sample data to help detect if your model is over-fit to the historical data. Furthermore, you would also perform the confidence interval calculation on the out-of-sample data of your trading model. By doing this, you’ll have more meaningful results. This step will help reduce the possibility of trading an over-fit trading model and give more meaning to your confidence interval test.

But even if we have a solid system that is not curve fitted to the historical data, our confidence interval calculations are no guarantee of success in the future. The markets are dynamic and changing and it’s possible that the distribution of trades will change thus altering our average trade and standard deviation. In the end, even if our system looks great on paper, validates on the out-of-sample data, and our confidence interval looks fantastic, our trading model could fail as soon as we trade it live. If this is the case, what is the point of all these testing and is it worth doing? The short answer is, yes.

In trading there is no guarantee for future results – ever. The point of testing a system is not to prove how much money it will make in the future. The goal is to find reasons why not to trade it. The purpose of any validating test is to find weaknesses so we can address those weaknesses now before we have money on the line. Our job as professional system traders is to manage risk which means eliminating risky actions. There is no certainty. This is one of the reasons this field is so psychologically difficult.

As a final point, by using confidence interval we have another tool to find weaknesses and ultimately gives us more confidence that a particular trading model will likely bring us success into the future.

RetrunVal = _CE_Expectancy( vExpectancy, vExpectancyScore, false );

Below is a video where I give a demonstration on using the expectancy function.

Expectancy Optimizer

In the previous article I hinted that you could use expectancy score as the key value to optimize a trading system against. Many people will use net profit or average profit per trade, but expectancy score might be useful as well. The expectancy score includes trading opportunity (as defined by the frequency of trading) its calculation. However, there is a problem. TradeStation provides no way to optimize over expectancy score. It must be done manually. To aid in this process I created a simple function called, _CE_Expectancy_Optimizer.

This function can be added to the bottom of any trading code and is designed to help you optimize a single parameter. For example, let’s say you wish to optimize the RSI look-back period for a particular strategy. After adding the function and configuring the required parameters, you perform your TradeStation optimization as you normally would. After each iteration of the TradeStation optimizer a line is written to an Excel file which contains the current RSI look-back period, expectancy and expectancy score. You can then use Excel’s sort feature to sort the different optimization iterations based upon the expectancy score.

There is nothing fancy about the Excel document generated. In fact, if you open up this file you will find  no headers or description. This is due to this function being called each time an optimization run is performed. I found no quick way to add headers only once.

Below are the function input parameters:

iWriteFlag( TrueFalseSeries ), // Should we write to a file or not?

iFileName(string), // File name

iSystemName( string ), // Name of system to test

iSystemVersionNumber( string ), // System's version number

iOptimize(NumericSimple); // The strategy input value you are optimizing

Here is an example of using the function within a strategy:

vRetValue = _CE_Expectancy_Optimizer( iWriteToFile, vFileName, RegimeLookback );

Below is a video demonstration of the _CE_Expectancy_Optimizer function.

Conclusion

With these two functions you will have two tools needed to calculate both the expectancy and expectancy score for the trading systems you are developing.

Expectancy = (AW * PW + AL * PL ) / | AL |  
Where:
AW = Average winning trade in dollars  
PW = Probability of winning trades  
| AL | = Absolute value of the average losing trade in dollars  
PL= Probability of losing trades

But if you look carefully at the calculation within the parenthesis you will see the value is nothing more than the trading system’s average net profit per trade. Substituting this value gives us the simplified formula of:

Expectancy = Average Net Profit Per Trade / | AL |

The two values you plug into the Expectancy formula can be found on most (if not all) strategy performance reports generated by backtesting. This is certainly true for TradeStation‘s strategy reports.

Now that we have our trading system’s Expectancy value are we ready to use this value to compare to other trading systems? Not yet. There is another step we must first take. Our Expectancy value simply tells us our historical profit per dollar risked for each trade. But we are missing something. Let’s imagine we have two trading systems that have two different Expectancy values:

Trading System #1 has an Expectancy of .25
Trading System #2 has an Expectancy of .50

Based on what we know it appears Trading System #2 produces more profit per dollar risked on each trade. In fact, it produces twice as much profit per dollar risked. Thus, if we risked $500 on each trade, Trading System 1 would generate $125 dollars while Trading System #2 would generate $250. But this is not the complete picture. We are missing the frequency at which each trading system operates. For example, maybe Trading System #1 trades once per day while Trading System #2 trades once per week. We need to take into account the number of times the trading system trades over the number of days the system was tested. In Van Tharp’s book he described that as Expectancy multiplied by opportunity. Opportunity is nothing more than how often does a given trading system trade. Opportunity times Expectancy leads us to our final calculation for Expectancy Score.

Expectancy Score

This value is an annualized Expectancy value which produces an objective number that can be used in comparing various trading systems. In essence the Expectancy Score factors in a trading system’s trade frequency. The higher the Expectancy Score the more profitable the system. This final score allows you to compare very different trading systems.

Expectancy Score = Expectancy * Number of Trades * 365 / Days in historical test

The Number of strategy trading days is nothing more than the number of days your backtesting was performed.

Conclusion

With the Expectancy Score in hand we have a metric to aid us in comparing different trading systems. Other uses for Expectancy Score might also include using this value as a target for optimization. Often optimization is performed on net profit, profit factor, Sharpe ratio or other metrics. Using Expectancy may also be something worth pursuing. But how would you do that using TradeStation? Unfortunately, there is no easy way to do it with TradeStation. However, I’m working on some EasyLanguage code that will help with a manual process. More on that in a future issue.