Comments on: System Performance and Confidence Interval

By: Jeff Swanson

Jeff Swanson — Mon, 02 May 2022 10:02:21 +0000

In reply to Marcello Chiuminatto. Hello Marcello. No, I've never done it that way.

By: Marcello Chiuminatto

Marcello Chiuminatto — Sat, 30 Apr 2022 14:36:54 +0000

Did you ever consider calculate a confidence interval using profit sample mean as your variable, so your samples are profit means produced by bootstraping?

By: Kevin Davey

Kevin Davey — Mon, 01 May 2017 17:59:19 +0000

Nice article Jeff. Lots of people want to develop a metric that “predicts” future performance. I’ve looked at tons of metrics, and nothing (yet) stands out as being predictive. Plus, I have a tough “gold standard” test – can the potential predictor metric identify an overoptimized, curve fit model, where all the performance metrics look super? We know that kind of strategy will perform poorly in the future. But no performance metric will say that.

By: DaveM

DaveM — Mon, 01 May 2017 12:18:23 +0000

This is a worthwhile approach, for both the testing of a strategy and also for after it has been trading for quite a while. Comparing it to other systems over the same period of time may also be quite revealing.
I do not think that the tests will give all of the answers until there is quite a history.
Thanks for this.

By: Jeff Swanson

Jeff Swanson — Thu, 03 Jul 2014 15:48:11 +0000

In reply to Lii.

Formula: “t = sqrt(number of trades) * avg trade / std deviation of avg trade”
For the T-score we like to see a value above 1.6. For the example above I round it up to 2.0. The higher the value the more statistically significant the results are. The value of 2.0 used raises the bar to a very high level.

By: Lii

Lii — Thu, 26 Jun 2014 14:20:58 +0000

Hi Jeff…
Very nice article…
Quick Question:
In the formula for CI there is “T-score” value in it.
How does one compute it? or /why did you use 2 in your example?
Could you pin-point me to a simple right direction regarding T-scores computation.
Thanks.

By: Jeff Swanson

Jeff Swanson — Wed, 02 Apr 2014 14:16:01 +0000

In reply to Mike. You're welcome Mike. Glad you found it helpful.

By: Mike

Mike — Mon, 31 Mar 2014 17:40:36 +0000

Killer article Jeff! I really love the way you think and view systems. Many, Many Thanks!

By: Jeff Swanson

Jeff Swanson — Tue, 11 Sep 2012 13:35:01 +0000

In reply to Bob. Of course. I did not created CI or was I the first to apply it to trading. In fact, using CI is a very common technique that I've read about in several books and/or web sites. What I'm showing here what is CI and how I personally utilize it.

By: Jeff Swanson

Jeff Swanson — Tue, 11 Sep 2012 13:31:34 +0000

In reply to Bob. This is true. Nor are future samples guaranteed to be identical in distribution.